@movq@www.uninformativ.de

Maybe I’m being a bit too purist/minimalistic here. As I said before (in one of the 1372739 posts on this topic – or maybe I didn’t even send that twt, I don’t remember 😅), I never really liked hashes to begin with. They aren’t super hard to implement but they are kind of against the beauty of the original twtxt – because you need special client support for them. It’s not something that you could write manually in your twtxt.txt file. With @sorenpeter@darch.dk’s proposal, though, that would be possible.

Tangentially related, I was a bit disappointed to learn that the twt subject extension is now never used except with hashes. Manually-written subjects sounded so beautifully ad-hoc and organic as a way to disambiguate replies. Maybe I’ll try it some time just for fun.

no my fault your client can’t handle a little editing ;)

@mckinley@twtxt.net

HTTPS is supposed to do [verification] anyway.

TLS provides verification that nobody is tampering with or snooping on your connection to a server. It doesn’t, for example, verify that a file downloaded from server A is from the same entity as the one from server B.

I was confused by this response for a while, but now I think I understand what you’re getting at. You are pointing out that with signed feeds, I can verify the authenticity of a feed without accessing the original server, whereas with HTTPS I can’t verify a feed unless I download it myself from the origin server. Is that right?

I.e. if the HTTPS origin server is online and I don’t mind taking the time and bandwidth to contact it, then perhaps signed feeds offer no advantage, but if the origin server might not be online, or I want to download a big archive of lots of feeds at once without contacting each server individually, then I need signed feeds.

feed locations [being] URLs gives some flexibility

It does give flexibility, but perhaps we should have made them URIs instead for even more flexibility. Then, you could use a tag URI, urn:uuid:*, or a regular old URL if you wanted to. The spec seems to indicate that the url tag should be a working URL that clients can use to find a copy of the feed, optionally at multiple locations. I’m not very familiar with IP{F,N}S but if it ensures you own an identifier forever and that identifier points to a current copy of your feed, it could be a great way to fix it on an individual basis without breaking any specs :)

I’m also not very familiar with IPFS or IPNS.

I haven’t been following the other twts about signatures carefully. I just hope whatever you smart people come up with will be backwards-compatible so it still works if I’m too lazy to change how I publish my feed :-)

@sorenpeter@darch.dk There was a client that would generate a unique hash for each twt. It didn’t get wide adoption.

Interesting.. QUIC isn’t very quick over fast internet.

QUIC is expected to be a game-changer in improving web application performance. In this paper, we conduct a systematic examination of QUIC’s performance over high-speed networks. We find that over fast Internet, the UDP+QUIC+HTTP/3 stack suffers a data rate reduction of up to 45.2% compared to the TCP+TLS+HTTP/2 counterpart. Moreover, the performance gap between QUIC and HTTP/2 grows as the underlying bandwidth increases. We observe this issue on lightweight data transfer clients and major web browsers (Chrome, Edge, Firefox, Opera), on different hosts (desktop, mobile), and over diverse networks (wired broadband, cellular). It affects not only file transfers, but also various applications such as video streaming (up to 9.8% video bitrate reduction) and web browsing. Through rigorous packet trace analysis and kernel- and user-space profiling, we identify the root cause to be high receiver-side processing overhead, in particular, excessive data packets and QUIC’s user-space ACKs. We make concrete recommendations for mitigating the observed performance issues.

https://dl.acm.org/doi/10.1145/3589334.3645323

So this is a great thread. I have been thinking about this too.. and what if we are coming at it from the wrong direction? Identity being tied to a given URL has always been a pain point. If i get a new URL its almost as if i have a new identity because not only am I serving at a new location but all my previous communications are broken because the hashes are all wrong.

What if instead we used this idea of signatures to thread the URLs together into one identity? We keep the URL to Hash in place. Changing that now is basically a no go. But we can create a signature chain that can link identities together. So if i move to a new URL i update the chain hosted by my primary identity to include the new URL. If i have an archived feed that the old URL is now dead, we can point to where it is now hosted and use the current convention of hashing based on the first url:

The signature chain can also be used to rotate to new keys over time. Just sign in a new key or revoke an old one. The prior signatures remain valid within the scope of time the signatures were made and the keys were active.

The signature file can be hosted anywhere as long as it can be fetched by a reasonable protocol. So say we could use a webfinger that directs to the signature file? you have an identity like frank@beans.co that will discover a feed at some URL and a signature chain at another URL. Maybe even include the most recent signing key?

From there the client can auto discover old feeds to link them together into one complete timeline. And the signatures can validate that its all correct.

I like the idea of maybe putting the chain in the feed preamble and keeping the single self contained file.. but wonder if that would cause lots of clutter? The signature chain would be something like a log with what is changing (new key, revoke, add url) and a signature of the change + the previous signature.

# chain: ADDKEY kex14zwrx68cfkg28kjdstvcw4pslazwtgyeueqlg6z7y3f85h29crjsgfmu0w 
# sig: BEGIN SALTPACK SIGNED MESSAGE. ... 
# chain: ADDURL https://txt.sour.is/user/xuu
# sig: BEGIN SALTPACK SIGNED MESSAGE. ...
# chain: REVKEY kex14zwrx68cfkg28kjdstvcw4pslazwtgyeueqlg6z7y3f85h29crjsgfmu0w
# sig: ...

@prologic@twtxt.net do that mean that for every new post (not replies) the client will have to generate a UUID or similar when posting and add that to to the twt?

@lyse@lyse.isobeef.org This looks like a nice way to do it.

Another thought: if clients can’t agree on the url (for example, if we switch to this new way, but some old clients still do it the old way), that could be mitigated by computing many hashes for each twt: one for every url in the feed. So, if a feed has three URLs, every twt is associated with three hashes when it comes time to put threads together.

A client stills need to choose one url to use for the hash when composing a reply, but this might add some breathing room if there’s a period when clients are doing different things.

(From what I understand of jenny, this would be difficult to implement there since each pseudo-email can only have one msgid to match to the in-reply-to headers. I don’t know about other clients.)

@movq@www.uninformativ.de @prologic@twtxt.net Another option would be: when you edit a twt, prefix the new one with (#[old hash]) and some indication that it’s an edited version of the original tweet with that hash. E.g. if the hash used to be abcd123, the new version should start “(#abcd123) (redit)”.

What I like about this is that clients that don’t know this convention will still stick it in the same thread. And I feel it’s in the spirit of the old pre-hash (subject) convention, though that’s before my time.

I guess it may not work when the edited twt itself is a reply, and there are replies to it. Maybe that could be solved by letting twts have more than one (subject) prefix.

But the great thing about the current system is that nobody can spoof message IDs.

I don’t think twtxt hashes are long enough to prevent spoofing.

@lyse@lyse.isobeef.org ah, if only you were to finally clean up that code, and make that client widely available…! One can only dream, right? :-)

@bender@twtxt.net Based on my experience so far, as a user, I would be upset if my client dropped someone from my follower list, i.e. stopped fetching their feed, without me asking for that to happen.

@prologic@twtxt.net I don’t know if this is new, but I’m seeing:

Jul 25 16:01:17 buc yarnd[1921547]: time="2024-07-25T16:01:17Z" level=error msg="https://yarn.stigatle.no/user/stigatle/twtxt.txt: client.Do fail: Get \"https://yarn.stigatle.no/user/stigatle/twtxt.txt\": dial tcp 185.97.32.18:443: i/o timeout (Client.Timeout exceeded while awaiting headers)" error="Get \"https://yarn.stigatle.no/user/stigatle/twtxt.txt\": dial tcp 185.97.32.18:443: i/o timeout (Client.Timeout exceeded while awaiting headers)"

I no longer see twts from @stigatle@yarn.stigatle.no at all.

I havnt seen any emails about the outage at work. I know i have the mac crowdstrike client though. My buddy that works at a hospital says they wernt affected.

@prologic@twtxt.net I was wondering if my reverse proxy could cause something but it’s pretty standard…

server {

    listen 80;
    server_name we.loveprivacy.club;

    location / {
            return 301 https://$host$request_uri;
            <a href="https://we.loveprivacy.club/search?q=%23proxy_pass">#proxy_pass</a> http://127.0.0.1:8000;
    }

}
server {

    listen 443 ssl http2;
    server_name we.loveprivacy.club;

    ssl_certificate /etc/letsencrypt/live/we.loveprivacy.club/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/we.loveprivacy.club/privkey.pem;

    client_max_body_size 8M;

    location / {
            proxy_pass http://127.0.0.1:8000;
    }

}

Its quite nice. I have been half tempted to make a twtxt client with it

@sorenpeter@darch.dk this makes sense as a quote twt that references a direct URL. If we go back to how it developed on twitter originally it was RT @nick: original text because it contained the original text the twitter algorithm would boost that text into trending.

i like the format (#hash) @<nick url> > "Quoted text"\nThen a comment
as it preserves the human read able. and has the hash for linking to the yarn. The comment part could be optional for just boosting the twt.

The only issue i think i would have would be that that yarn could then become a mess of repeated quotes. Unless the client knows to interpret them as multiple users have reposted/boosted the thread.

The format is also how iphone does reactions to SMS messages with +number liked: original SMS

I’m also more in favor of #reposts being human readable and writable. A client might implement a bottom that posts something simple like: #repost Look at this cool stuff, because bla bla [alt](url)

This will then make it possible to also “repost” stuff from other platforms/protocols.

The reader part of a client, can then render a preview of the link, which we talked about would be a nice (optional) feature to have in yarnd.

@adi@twtxt.net I think it is, and one benefit they have is that you can add third-party repositories to the F-Droid app as you discover them. So, for instance, if you know of a developer who pushes builds to an F-Droid compatible repository, you can add that to your F-Droid app and start tracking updates like you would for any other app in there. Can’t do that with Google Play!

F-Droid tends to focus on open source applications that can be built in a reproducible way, which limits the inventory (though of course tends to mean the apps are safer and don’t spy on you). There are non-free apps in there as well but they come with warnings so you’re informed about what you might be sacrificing by using them.

That said if you have a favorite app you get through Google Play, there’s a decent chance it won’t be in F-Droid. Many “big corporate” apps aren’t, and vendor-specific apps tend not to be either. But for most of the major functions you might want, like email clients, calendar apps, weather apps, etc etc, there are very good substitutes now in F-Droid. You’re definitely making a trade-off though.

What I did was go through the apps I had installed on my last phone, found as many substitutes in F-Droid as I could, started using those instead to see how they worked, and bit by bit replaced as much as I could from Google Play with a comparable app from F-Droid. I still have a few apps (mostly vendor-specific things that don’t have substitutes) that come from Google Play but I’m aiming to be rid of those before I need to replace this phone.

@jmjl@tilde.green I’m sorry that I’m not super knowledgeable about alternatives to jmp.chat but I’ll tell you what I know.

You’re probably right about jmp.chat not working for you, at least as it is now. You can only get US and Canadian phone numbers through it last time I checked, so if you’re not in either of those countries you’d be making international calls all the time and people who wanted to call you would be making international calls too.

I’ve seen people talk about using SIP as an intermediary: you can bridge SIP-to-XMPP, and bridge SIP-to-PSTN (PSTN = “packet switched telephone network”, meaning normal telephone). You can skip the SIP-to-XMPP side if you’re comfortable using a SIP client. I don’t know very much about SIP or PSTN so I am not sure what to recommend, but perhaps this helps your search queries.

There are a fair number of services like TextNow that let you sign up for a real telephone number that you can then use via their app (I wouldn’t use TextNow–they had tons of spyware in their app). I don’t know if that kind of service works for you but if it does perhaps you’d be able to find one of them that isn’t horrible. This page (https://alternativeto.net/software/jmp-chat/) has a bunch of alternatives; I can’t vouch for any of them but maybe it’s a starting point if you want to go this route.

Good luck!

I’m not super a fan of using json. I feel we could still use text as the medium. Maybe a modified version to fix any weakness.

What if instead of signing each twt individually we generated a merkle tree using the twt hashes? Then a signature of the root hash. This would ensure the full stream of twts are intact with a minimal overhead. With the added bonus of helping clients identify missing twts when syncing/gossiping.

Have two endpoints. One as the webfinger to link profile details and avatar like you posted. And the signature for the merkleroot twt. And the other a pageable stream of twts. Or individual twts/merkle branch to incrementally access twt feeds.

@mckinley@twtxt.net i use pass along with the android and browser-pass clients. it is very good and keeping in sync is pretty simple.

(cont.)

Just to give some context on some of the components around the code structure.. I wrote this up around an earlier version of aggregate code. This generic bit simplifies things by removing the need of the Crud functions for each aggregate.

A domain object can be used as an aggregate by adding the event.AggregateRoot struct and finish implementing event.Aggregate. The AggregateRoot implements logic for adding events after they are either Raised by a command or Appended by the eventstore Load or service ApplyFn methods. It also tracks the uncommitted events that are saved using the eventstore Save method.

type User struct {
  Identity string ```json:"identity"`

  CreatedAt time.Time

  event.AggregateRoot
}

// StreamID for the aggregate when stored or loaded from ES.
func (a *User) StreamID() string {
	return "user-" + a.Identity
}
// ApplyEvent to the aggregate state.
func (a *User) ApplyEvent(lis ...event.Event) {
	for _, e := range lis {
		switch e := e.(type) {
		case *UserCreated:
			a.Identity = e.Identity
			a.CreatedAt = e.EventMeta().CreatedDate
        /* ... */
		}
	}
}

Events are applied to the aggregate. They are defined by adding the event.Meta and implementing the getter/setters for event.Event

type UserCreated struct {
	eventMeta event.Meta

	Identity string
}

func (c *UserCreated) EventMeta() (m event.Meta) {
	if c != nil {
		m = c.eventMeta
	}
	return m
}
func (c *UserCreated) SetEventMeta(m event.Meta) {
	if c != nil {
		c.eventMeta = m
	}
}

With a domain object that implements the event.Aggregate the event store client can load events and apply them using the Load(ctx, agg) method.

// GetUser populates an user from event store.
func (rw *User) GetUser(ctx context.Context, userID string) (*domain.User, error) {
	user := &domain.User{Identity: userID}

	err := rw.es.Load(ctx, user)
	if err != nil {
		if err != nil {
			if errors.Is(err, eventstore.ErrStreamNotFound) {
				return user, ErrNotFound
			}
			return user, err
		}
		return nil, err
	}
	return user, err
}

An OnX command will validate the state of the domain object can have the command performed on it. If it can be applied it raises the event using event.Raise() Otherwise it returns an error.

// OnCreate raises an UserCreated event to create the user.
// Note: The handler will check that the user does not already exsist.
func (a *User) OnCreate(identity string) error {
    event.Raise(a, &UserCreated{Identity: identity})
    return nil
}

// OnScored will attempt to score a task.
// If the task is not in a Created state it will fail.
func (a *Task) OnScored(taskID string, score int64, attributes Attributes) error {
	if a.State != TaskStateCreated {
		return fmt.Errorf("task expected created, got %s", a.State)
	}
	event.Raise(a, &TaskScored{TaskID: taskID, Attributes: attributes, Score: score})
	return nil
}

The following functions in the aggregate service can be used to perform creation and updating of aggregates. The Update function will ensure the aggregate exists, where the Create is intended for non-existent aggregates. These can probably be combined into one function.

// Create is used when the stream does not yet exist.
func (rw *User) Create(
  ctx context.Context,
  identity string,
  fn func(*domain.User) error,
) (*domain.User, error) {
	session, err := rw.GetUser(ctx, identity)
	if err != nil && !errors.Is(err, ErrNotFound) {
		return nil, err
	}

	if err = fn(session); err != nil {
		return nil, err
	}

	_, err = rw.es.Save(ctx, session)

	return session, err
}

// Update is used when the stream already exists.
func (rw *User) Update(
  ctx context.Context,
  identity string,
  fn func(*domain.User) error,
) (*domain.User, error) {
	session, err := rw.GetUser(ctx, identity)
	if err != nil {
		return nil, err
	}

	if err = fn(session); err != nil {
		return nil, err
	}

	_, err = rw.es.Save(ctx, session)
	return session, err
}

I realized my twtxt client isn’t validating what it pulls once it gets a valid response when a domain started returning js-heavy parking pages for every URL. Oops. Weekend project, I guess. 🤦🏻

@stackeffect@twtxt.stackeffect.de
I am seeing this characters on your twts: )?â\200¨â\200¨. Which client are you using?

@stigatle@twtxt.net
A twtxt client would be nice! Or a very simple cgi script to print twts to web nicely—not a second Yarn, just something to show twts in a pretty form on the web.

@eldersnake@yarn.andrewjvpowell.com
RSS links are archaic. Clients discover them if properly linked, they do not need to be human visible.

I am noticing that Yarn doesn’t treat “outside” (that is, twts coming from a client other than Yarn) twts hashes right. Two examples:

• https://twtxt.net/twt/mwtnrna
  
• https://twtxt.net/twt/fl5dooq
  

There are many more, but those two will give you the gist. Yarn links the hash to the poster’s twtxt.txt, so conversation matching will not work.

@lyse@lyse.isobeef.org Unless you are stripping stuff on your twts, there is no much to implement. Things will be bold , italics , underlined , and so on, on a client that can render them. Since jenny uses Mutt, I can use my own regex in it to color them as I like. That’s pretty much it.

Fixed another bug in my finger client: rfc1288 says lines have to end with crlf, but I was just sending lf.

Indeed! I think the first “network protocol client” I ever wrote was something that just did the PING/PONG part and passed everything else raw.

Looking at raw IRC traffic streams to debug a client issue and it’s 1997 again.

Sure. I think search, if it’s going to exist, should be the client’s responsibility. But I also value the readability of the raw twtxt file a lot more than y’all do.

I want read-only iOS client that just does the simplest model: pull a list of feeds, make a timeline.

@xjix@xj-ix.luxe Saw your oldish note about wanting an offline/async twtxt workflow. Do you have something that works for you? My (very young!) client was designed with that in mind.

I agree clients should present things better (part of why I’m writing one!). But that should be additive. There’s a reason we’re not passing json around.

@prologic@twtxt.net Exactly, but that reduces the argument for URLs in the post. The client should figure out how to search based on the hashtag.

My silly Plan 9 rc twtxt client now has a web page: http://txtpunk.com/tw/index.html

@prologic@twtxt.netYes, I think tags should just be #foo, and let the client figure out searching if it cares.

@movq@www.uninformativ.de No argument that threading is an improvement. But I think (#hash) does that, and I think figuring out how to search should mostly be up to the client.

Hah… my silly twtxt client now has “stories” mode.☺

@prologic@twtxt.net deedum for android.

Kristall for OS X

Elaho for iOS

more here

though I can only vouch for the first two.

I have a working model for the reader portion of what I want this twtxt client to do.

@prologic@twtxt.net

Can we not have clients sign their own public keys before listing them on their Pod’s account?

Yeah.. we probably could. when they setup an account they create a master key that signs any subsequent keys. or chain of signatures like keybase does.

@prologic@twtxt.net (#gqg3gea) ha yeah. COVID makes for a timey-wimey mish-mash. Worked on some WKD and fought with my XMPP client a bit.

@prologic@twtxt.net huh.. true.. the email is md5/sha256 before storing.. if twtxt acted as provider you would store that hash and point the SRV record to the pod. .. to act as a client it would need to store the hash and the server that hosts the image.

Incomplete, not packaged for casual use, and this UI library won’t scroll, but my twtxt client kinda works https://github.com/jcolag/Uxuyu

Disregard this message - I’m testing a GUI client prototype. Hopefully this doesn’t wreck my feed…

The nice API makes me wonder how hard it would be to put together a lightweight twtxt desktop client; maybe I’ll look at that next week.