@prologic@twtxt.net I think printf is a more portable option than echo -e for interpreting \t as tab. E.g. printf ‘%s\t%s\t%s’ “$url” “$time” “$text”. In general I always prefer printf over echo for anything non-trivial in unix shell scripts. See last paragraph of https://en.wikipedia.org/wiki/Echo_(command)#History

@aelaraji@aelaraji.com easy as cake to get and
account here. Very reliable too!

Thanks @david@collantes.us, good to know, but we need to agree on what character we use, otherwise the hashes will not be the same:)

@aelaraji@aelaraji.com You could just remove the {getuser()} part because you added ~.

Ok, i know how to command working (not sure), but seems it only grab from cache. Maybe make fetch from twtxt.net if hash not found?

@prologic@twtxt.net Regarding the new way of generating twt-hashes, to me it makes more sense to use tabs as separator instead of spaces, since the you can just copy/past a line directly from a twtxt-file that already go a tab between timestamp and message. But tabs might be hard to “type” when you are in a terminal, since it will activate autocomplete…🤔

Another thing, it seems that you sugget we only use the domain in the hash-creation and not the full path to the twtxt.txt

$ echo -e "https://example.com 2024-09-29T13:30:00Z Hello World!" | sha256sum - | awk '{ print $1 }' | base64 | head -c 12

should i delete gemini support from twet? iirc in twtxt v2 it starts prohibited. And all of my fields are https

It has twts cache which used if timeline is set to jew. Maybe i.should fork twet to make wishes like newlines (i see two squares), showing conversations, showing twts if not found in cache and parsing medata to configure url, nick and followers (currenly it duplicated in config and twtxt file)

twet display twts in raw format with some formatting (sadly no newlines). And for reply messages i just seen (#hash). But which text hidden on hash? currenly im open twtxt.net/twt/hash to see this

Lol, im just join for several minutes. Wait, Merkle Trees in twtxt?

@prologic@twtxt.net YES James, it should be up to the client to deal with changes like edits and deletions. And putting this load on the clients, location-addressing with make this a lot easier since what is says it: Look in this file at this timestamp, did anything change or went missing? (And then threading will not break;)

@bender@twtxt.net It’s the experience of an ordinary person in a strange place where memories are disappearing with the help of the Memory Police. The setting feels contemporary (to the book’s 1994 publication date) rather than futuristic, except for some unexplained stuff about memories.

@prologic@twtxt.net

See https://dev.twtxt.net

Yes, that is exactly what I meant. I like that collection and “twtxt v2” feels like a departure.

Maybe there’s an advantage to grouping it into one spec, but IMO that shouldn’t be done at the same time as introducing new untested ideas.

See https://yarn.social (especially this section: https://yarn.social/#self-host) – It really doesn’t get much simpler than this 🤣

Again, I like this existing simplicity. (I would even argue you don’t need the metadata.)

That page says “For the best experience your client should also support some of the Twtxt Extensions…” but it is clear you don’t need to. I would like it to stay that way, and publishing a big long spec and calling it “twtxt v2” feels like a departure from that. (I think the content of the document is valuable; I’m just carping about how it’s being presented.)

More thoughts about changes to twtxt (as if we haven’t had enough thoughts):

1. There are lots of great ideas here! Is there a benefit to putting them all into one document? Seems to me this could more easily be a bunch of separate efforts that can progress at their own pace:
   

1a. Better and longer hashes.

1b. New possibly-controversial ideas like edit: and delete: and location-based references as an alternative to hashes.

1c. Best practices, e.g. Content-Type: text/plain; charset=utf-8

1d. Stuff already described at dev.twtxt.net that doesn’t need any changes.

1. We won’t know what will and won’t work until we try them. So I’m inclined to think of this as a bunch of draft ideas. Maybe later when we’ve seen it play out it could make sense to define a group of recommended twtxt extensions and give them a name.

2. Another reason for 1 (above) is: I like the current situation where all you need to get started is these two short and simple documents:
   https://twtxt.readthedocs.io/en/latest/user/twtxtfile.html
   https://twtxt.readthedocs.io/en/latest/user/discoverability.html
   and everything else is an extension for anyone interested. (Deprecating non-UTC times seems reasonable to me, though.) Having a big long “twtxt v2” document seems less inviting to people looking for something simple. (@prologic@twtxt.net you mentioned an anonymous comment “you’ve ruined twtxt” and while I don’t completely agree with that commenter’s sentiment, I would feel like twtxt had lost something if it moved away from having a super-simple core.)

3. All that being said, these are just my opinions, and I’m not doing the work of writing software or drafting proposals. Maybe I will at some point, but until then, if you’re actually implementing things, you’re in charge of what you decide to make, and I’m grateful for the work.

@prologic@twtxt.net Done. Also, I went ahead and made two changes: changed hexadecimal to base64 for hashes (wasn’t sure if anyone objected), and changed “MUST follow the chain” to “SHOULD follow the chain.

@prologic@twtxt.net Thanks for pointing out it lasts four hours. That’s a big window! I wonder when most people will be on. I might aim for halfway through unless I hear otherwise. (12:00Z is a bit early for me.)

@bender@twtxt.net I am also in camp no edit signals. deletes only breaks the head of a thread. all the replies are unaffected.

Yes. I have only twtxt and scp hook in twet and it enough

Wait, webfinger? Mandate this ruin philosophy “twtxt is just text file”

I dont think that is ruined twtxt. Twtxt v2 is just standartize twtxt and yarn extensions. What is bad?

@bender@twtxt.net I believe it is Unix-Unix Copy Protocol. Not Unix Copy-Copy Protocol.

@aelaraji@aelaraji.com ooooh! It’s that kind mission! /me stands, salutes, turns around, and exits the room. LOL.

@aelaraji@aelaraji.com why, having a party with lots of libations? LOL.

@david@collantes.us having offsets were nice because it gives you context of where the user is in relation to you.

@prologic@twtxt.net thanks. I hate it. Might as well use UUID

This is only first draft quality, but I made some notes on the #twtxt v2 proposal. http://a.9srv.net/b/2024-09-25

No, json is overhead. I love twtxt for simplicity where blog is just text file and not several json files where fields are repeated…

(#2024-09-24T12:53:35Z) What does this screenshot show? The resolution it too low for reading the text…

(#2024-09-24T12:45:54Z) @prologic@twtxt.net I’m not really buying this one about readability. It’s easy to recognize that this is a URL and a date, so you skim over it like you would we mentions and markdown links and images. If you are not suppose to read the raw file, then we might a well jam everything into JSON like mastodon

(#2024-09-24T12:44:35Z) There is a increase in space/memory for sure. But calculating the hashes also takes up CPU. I’m not good with that kind of math, but it’s a tradeoff either way.

(#2024-09-24T12:39:32Z) @prologic@twtxt.net It might be simple for you to run echo -e "\t\t" | sha256sum | base64, but for people who are not comfortable in a terminal and got their dev env set up, then that is magic, compared to the simplicity of just copy/pasting what you see in a textfile into another textfile – Basically what @movq@www.uninformativ.de also said. I’m also on team extreme minimalism, otherwise we could just use mastodon etc. Replacing line-breaks with a tab would also make it easier to handwrite your twtxt. You don’t have to hardwrite it, but at least you should have the option to. Just as i do with all my HTML and CSS.

(#2024-09-24T12:34:31Z) WebMentions does would work if we agreed to implement it correctly. I never figured out how yarnd’s WebMentions work, so I decide to make my own, which I’m the only one using…

I had a look at WebSub, witch looks way more complex than WebMentions, and seem to need a lot more overhead. We don’t need near realtime. We just need a way to notify someone that someone they don’t know about mentioned or replied to their post.

Aggred. But reading twtxt in raw form sounds… I can’t do this

Some more arguments for a local-based treading model over a content-based one:

1. The format: (#<DATE URL>) or (@<DATE URL>) both makes sense: # as prefix is for a hashtag like we allredy got with the (#twthash) and @ as prefix denotes that this is mention of a specific post in a feed, and not just the feed in general. Using either can make implementation easier, since most clients already got this kind of filtering.

2. Having something like (#<DATE URL>) will also make mentions via webmetions for twtxt easier to implement, since there is no need for looking up the #twthash. This will also make it possible to make 3th part twt-mentions services.

3. Supporting twt/webmentions will also increase discoverability as a way to know about both replies and feed mentions from feeds that you don’t follow.

Finally pubnix is alive! That’s im missing? Im only reading twtxt.net timeline because twtxt-v2.sh works slowly for displaying timeline…

@movq@www.uninformativ.de Yes, the tools are surprisingly fast. Still, magrep takes about 20 seconds to search through my archive of 140K emails, so to speed things up I would probably combine it with an indexer like mu, mairix or notmuch.

@falsifian@www.falsifian.org I believe the preserve means to include the original subject hash in the start of the twt such as (#somehash)

@bender@twtxt.net Ha! Maybe I should get on the Markdown train. You’re taking away my excuses.

@bender@twtxt.net

Sorry, you’re right, I should have used numbers!

I’m don’t understand what “preserve the original hash” could mean other than “make sure there’s still a twt in the feed with that hash”. Maybe the text could be clarified somehow.

I’m also not sure what you mean by markdown already being part of it. Of course people can already use Markdown, just like presumably nothing stopped people from using (twt subjects) before they were formally described. But it’s not universal; e.g. as a jenny user I just see the plain text.

@falsifian@www.falsifian.org The GDPR does not apply to the processing of data for a purely personal or household activity that is not connected to a professional or commercial activity.

@prologic@twtxt.net Do you feel the same about published vs. privately stored data?

For me there’s a distinction. I feel very strongly that I should be able to retain whatever private information I like. On the other hand, I do have some sympathy for requests not to publish or propagate (though I personally feel it’s still morally acceptable to ignore such requests).

@lyse@lyse.isobeef.org I’d suggest making the whole content-type thing a SHOULD, to accommodate people just using some hosting service they don’t have much control over. (The same situation could make detecting followers hard, but IMO “please email me if you follow me” is still legit twtxt, even if inconvenient.)

@prologic@twtxt.net Thanks for writing that up!

I hope it can remain a living document (or sequence of draft revisions) for a good long time while we figure out how this stuff works in practice.

I am not sure how I feel about all this being done at once, vs. letting conventions arise.

For example, even today I could reply to twt abc1234 with “(#abc1234) Edit: …” and I think all you humans would understand it as an edit to (#abc1234). Maybe eventually it would become a common enough convention that clients would start to support it explicitly.

Similarly we could just start using 11-digit hashes. We should iron out whether it’s sha256 or whatever but there’s no need get all the other stuff right at the same time.

I have similar thoughts about how some users could try out location-based replies in a backward-compatible way (append the replyto: stuff after the legacy (#hash) style).

However I recognize that I’m not the one implementing this stuff, and it’s less work to just have everything determined up front.

Misc comments (I haven’t read the whole thing):

• Did you mean to make hashes hexadecimal? You lose 11 bits that way compared to base32. I’d suggest gaining 11 bits with base64 instead.

• “Clients MUST preserve the original hash” — do you mean they MUST preserve the original twt?

• Thanks for phrasing the bit about deletions so neutrally.

• I don’t like the MUST in “Clients MUST follow the chain of reply-to references…”. If someone writes a client as a 40-line shell script that requires the user to piece together the threading themselves, IMO we shouldn’t declare the client non-conforming just because they didn’t get to all the bells and whistles.

• Similarly I don’t like the MUST for user agents. For one thing, you might want to fetch a feed without revealing your identty. Also, it raises the bar for a minimal implementation (I’m again thinking again of the 40-line shell script).

• For “who follows” lists: why must the long, random tokens be only valid for a limited time? Do you have a scenario in mind where they could leak?

• Why can’t feeds be served over HTTP/1.0? Again, thinking about simple software. I recently tried implementing HTTP/1.1 and it wasn’t too bad, but 1.0 would have been slightly simpler.

• Why get into the nitty-gritty about caching headers? This seems like generic advice for HTTP servers and clients.

• I’m a little sad about other protocols being not recommended.

• I don’t know how I feel about including markdown. I don’t mind too much that yarn users emit twts full of markdown, but I’m more of a plain text kind of person. Also it adds to the length. I wonder if putting a separate document would make more sense; that would also help with the length.

@movq@www.uninformativ.de I cases of these kind of “abuse” of social trust. Then I think people should just delete their replies, unfollow the troll and leave them to shouting in the void. This is a inter-social issue, not a technical issue. Anything can be spoofed. We are not building a banking app, we are just having conversation and if trust are broken then communication breaks down. These edge-cases are all very hypothetical and not something I think we need to solve with technology.

We’re happy to report that @burglar@yarnd.lyse.isobeef.org was taken into custody, @prologic@twtxt.net. Always remember, criminals cannot escape the law.

Our investigations revealed: https://lyse.isobeef.org/tmp/twtinjector.tar.bz2

Fear not, @prologic@twtxt.net, we’re deploying our helicopter and will arrive shortly.

Heads up, @prologic@twtxt.net! We’re seeing increased spate of burglaries in your neighbourhood. Please stay alert, while we keep you safe out there.

@prologic@twtxt.net I have no specifics, only hopes. (I have seen some articles explaining the GDPR doesn’t apply to a “purely personal or household activity” but I don’t really know what that means.)

I don’t know if it’s worth giving much thought to the issue unless either you expect to get big enough for the GDPR to matter a lot (I imagine making money is a prerequisite) or someone specifically brings it up. Unless you enjoy thinking through this sort of thing, of course.

@david@collantes.us Thanks, that’s good feedback to have. I wonder to what extent this already exists in registry servers and yarn pods. I haven’t really tried digging into the past in either one.

How interested would you be in changes in metadata and other comments in the feeds? I’m thinking of just permanently saving every version of each twtxt file that gets pulled, not just the twts. It wouldn’t be hard to do (though presenting the information in a sensible way is another matter). Compression should make storage a non-issue unless someone does something weird with their feed like shuffle the comments around every time I fetch it.

@movq@www.uninformativ.de I don’t think it has to be like that. Just make sure the new version of the twt is always appended to your current feed, and have some convention for indicating it’s an edit and which twt it supersedes. Keep the original twt as-is (or delete it if you don’t want new followers to see it); doesn’t matter if it’s archived because you aren’t changing that copy.