@wbknl@twtxt.net are you still in Russia? It could be hard mailing anything to there these days. I read your “russia is eternally cold”, and became curious. Patagonia is the only place I know on South America that it has rounded mountains, though they can be anywhere. Originally from Chile, or Argentina? My curiosity doesn’t need feeding, by the way. It’s all good if it doesn’t. :-)
@eapl.me@eapl.me here are my replies (somewhat similar to Lyse’s and James’)
Metadata in twts: Key=value is too complicated for non-hackers and hard to write by hand. So if there is a need then we should just use #NSFS or the alt-text file in markdown image syntax
if something is NSFWIDs besides datetime. When you edit a twt then you should preserve the datetime if location-based addressing should have any advantages over content-based addressing. If you change the timestamp the its a new post. Just like any other blog cms.
Caching, Yes all good ideas, but that is more a task for the clients not the serving of the twtxt.txt files.
Discovery: User-agent for discovery can become better. I’m working on a wrapper script in PHP, so you don’t need to go to Apaches log-files to see who fetches your feed. But for other Gemini and gopher you need to relay on something else. That could be using my webmentions for twtxt suggestion, or simply defining an email metadata field for letting a person know you follow their feed. Interesting read about why WebMetions might be a bad idea. Twtxt being much simple that a full featured IndieWeb sites, then a lot of the concerns does not apply here. But that’s the issue with any open inbox. This is hard to solve without some form of (centralized or community) spam moderation.
Support more protocols besides http/s. Yes why not, if we can make clients that merge or diffident between the same feed server by multiples URLs
Languages: If the need is big then make a separate feed. I don’t mind seeing stuff in other langues as it is low. You got translating tool if you need to know whats going on. And again when there is a need for easier switching between posting to several feeds, then it’s about building clients with a UI that makes it easy. No something that should takes up space in the format/protocol.
Emojis: I’m not sure what this is about. Do you want to use emojis as avatar in CLI clients or it just about rendering emojis?
@prologic@twtxt.net I’m not a yarnd user, so it doesn’t matter a whole lot to me, but FWIW I’m not especially keen on changing how I format my twts to work around yarnd’s quirks.
I wonder if this kind of postprocessing would fit better between composing (via yarnd’s UI) and publishing. So, if a yarnd user types ¼, it could get changed to ¼ in the twtxt.txt file for everyone to see, not just people reading through yarnd. But when I type ¼, meaning first out of four, as a non-yarnd user, the meaning wouldn’t get corrupted. I can always type ¼ directly if that’s what I really intend.
(This twt might be easier to understand if you read it without any transformations :-P)
Anyway, again, I’m not a yarnd user, so do what you will, just know you might not be seeing exactly what I meant.
Inversion by Aric McBay was another random library pick. Like The Fall of Io, it’s the most recent in a series, though I think this series is pretty loosely connected. In contrast, the villain in this book is simple and cartoonishly evil. The book presents a design for utopia which was interesting but a little cloying. I’m not sure if I’m supposed to want to live there, but I don’t think I do. I enjoyed the book as easy reading, and might try the others in the series some time. (4/4)
I read Starter Villain by John Scalzi. Enjoyable, like his other books that I’ve read. Somewhat sillier. (¾)
I’m enjoying Wesley Chu’s Tao and Io series. Spies, action, ancient aliens. Some funny parts, some interesting world-building parts, some action-filled parts. I picked up The Fall of Io at random from a library a few weeks ago, and it turned out to be the last in a series of six (technically two series), so after finishing that I read the first and am partway through the second. Usually I try to read series in order, but this way is interesting. One thing I liked about The Fall of Io was that it it followed many points of view with somewhat conflicting interests, some more evil than others, and I felt sympathy for most of them. (I was kind of hoping it would be about Jupiter’s moon Io, but it wasn’t, but I’m satisfied with what I ended up with.) (2/4)
Simplified twtxt - I want to suggest some dogmas or commandments for twtxt, from where we can work our way back to how to implement different feature like replies/treads:
It’s a text file, so you must be able to write it by hand (ie. no app logic) and read by eye. If you edit a post you change the content not the timestamp. Otherwise it will be considered a new post.
The order of lines in a twtxt.txt must not hold any significant. The file is a container and each line an atomic piece of information. You should be able to run
sorton a twtxt.txt and it should still work.Transport protocol should not matter, as long as the file served is the same. Http and https are preferred, so it is suggested that feed served via Gopher or Gemini also provide http(s).
Do we need more commandments?
@doesnm@doesnm.p.psf.lt finally someone read my blogpost ;)
@prologic@twtxt.net currently? it wouldnt :D.
we would need to come up with a way of registering with multiple brokers that can i guess forward to a reader broker. something that will retry if needed. need to read into how simplex handles multi brokers
Reading about browser security measures and getting sad we don’t live in a world where cross-site scripting is a feature instead of a bug.
Recent #fiction #scifi #reading:
The Memory Police by YĹŤko Ogawa. Lovely writing. Very understated; reminded me of Kazuo Ishiguro. Sort of like Nineteen Eighty-Four but not. (I first heard it recommended in comparison to that work.)
Subcutanean by Aaron Reed; https://subcutanean.textories.com/ . Every copy of the book is different, which is a cool idea. I read two of them (one from the library, actually not different from the other printed copies, and one personalized e-book). I don’t read much horror so managed to be a little creeped out by it, which was fun.
The Wind from Nowhere, a 1962 novel by J. G. Ballard. A random pick from the sci-fi section; I think I picked it up because it made me imagine some weird 4-dimensional effect (“from nowhere” meaning not in a normal direction) but actually (spoiler) it was just about a lot of wind for no reason. The book was moderately entertaining but there was nothing special about it.
Currently reading Scale by Greg Egan and Inversion by Aric McBay.
(#2024-09-24T12:53:35Z) What does this screenshot show? The resolution it too low for reading the text…
(#2024-09-24T12:45:54Z) @prologic@twtxt.net I’m not really buying this one about readability. It’s easy to recognize that this is a URL and a date, so you skim over it like you would we mentions and markdown links and images. If you are not suppose to read the raw file, then we might a well jam everything into JSON like mastodon
@prologic@twtxt.net Thanks for writing that up!
I hope it can remain a living document (or sequence of draft revisions) for a good long time while we figure out how this stuff works in practice.
I am not sure how I feel about all this being done at once, vs. letting conventions arise.
For example, even today I could reply to twt abc1234 with “(#abc1234) Edit: …” and I think all you humans would understand it as an edit to (#abc1234). Maybe eventually it would become a common enough convention that clients would start to support it explicitly.
Similarly we could just start using 11-digit hashes. We should iron out whether it’s sha256 or whatever but there’s no need get all the other stuff right at the same time.
I have similar thoughts about how some users could try out location-based replies in a backward-compatible way (append the replyto: stuff after the legacy (#hash) style).
However I recognize that I’m not the one implementing this stuff, and it’s less work to just have everything determined up front.
Misc comments (I haven’t read the whole thing):
Did you mean to make hashes hexadecimal? You lose 11 bits that way compared to base32. I’d suggest gaining 11 bits with base64 instead.
“Clients MUST preserve the original hash” — do you mean they MUST preserve the original twt?
Thanks for phrasing the bit about deletions so neutrally.
I don’t like the MUST in “Clients MUST follow the chain of reply-to references…”. If someone writes a client as a 40-line shell script that requires the user to piece together the threading themselves, IMO we shouldn’t declare the client non-conforming just because they didn’t get to all the bells and whistles.
Similarly I don’t like the MUST for user agents. For one thing, you might want to fetch a feed without revealing your identty. Also, it raises the bar for a minimal implementation (I’m again thinking again of the 40-line shell script).
For “who follows” lists: why must the long, random tokens be only valid for a limited time? Do you have a scenario in mind where they could leak?
Why can’t feeds be served over HTTP/1.0? Again, thinking about simple software. I recently tried implementing HTTP/1.1 and it wasn’t too bad, but 1.0 would have been slightly simpler.
Why get into the nitty-gritty about caching headers? This seems like generic advice for HTTP servers and clients.
I’m a little sad about other protocols being not recommended.
I don’t know how I feel about including markdown. I don’t mind too much that yarn users emit twts full of markdown, but I’m more of a plain text kind of person. Also it adds to the length. I wonder if putting a separate document would make more sense; that would also help with the length.
Could someone knowledgable reply with the steps a grandpa will take to calculate the hash of a twtxt from the CLI, using out-of-the-box tools? I swear I read about it somewhere, but can’t find it.
More:
Subject: The [tag URI scheme](https://en.wikipedia.org/wiki/Tag_URI_scheme) looks interesting. I like that it human read- and writable. And since we already got the timestamp in the twtxt.txt it would be
somewhat trivial to parse. But there are still the issue with what the name/id should be... Maybe it doesn't have to bee that stick? Instead of using `tag:` as the prefix/protocol, it would more it clear
what we are talking about by using `in-reply-to:` (https://indieweb.org/in-reply-to) or `replyto:` similar to `mailto:` 1. `(reply:sorenpeter@darch.dk,2024-09-15T12:06:27Z)' 2.
`(in-reply-to:darch.dk/twtxt.txt,2024-09-15T12:06:27Z)' 2. `(replyto:http://darch.dk/twtxt.txt,2024-09-15T12:06:27Z)' I know it's longer that 7-11 characters, but it's self-explaining when looking at the
twtxt.txt in the raw, and the cases above can all be caught with this regex: `\([\w-]*reply[\w-]*\:` Is this something that would work?
Subject: The [tag URI scheme](https://en.wikipedia.org/wiki/Tag_URI_scheme) looks interesting. I like that it human read- and writable. And since we already got the timestamp in the twtxt.txt it would be
somewhat trivial to parse. But there are still the issue with what the name/id should be... Maybe it doesn't have to bee that stick? Instead of using `tag:` as the prefix/protocol, it would more it clear
what we are talking about by using `in-reply-to:` (https://indieweb.org/in-reply-to) or `replyto:` similar to `mailto:` 1. `(reply:sorenpeter@darch.dk,2024-09-15T12:06:27Z)` 2.
`(in-reply-to:darch.dk/twtxt.txt,2024-09-15T12:06:27Z)` 3. `(replyto:http://darch.dk/twtxt.txt,2024-09-15T12:06:27Z)` I know it's longer that 7-11 characters, but it's self-explaining when looking at the
twtxt.txt in the raw, and the cases above can all be caught with this regex: `\([\w-]*reply[\w-]*\:` Is this something that would work?
Notice the difference? Soren edited, and broke everything.
@mckinley@twtxt.net Thanks for the feedback.
- Yeah I agrees that nick sound not be part of syntax. Any valid URL to a twtxt.txt-file should be enough and is more clear, so it is not confused with a email (one of the the issues with webfinger and fedivese handles)
- I think any valid URL would work, since we are not bound to look for exact matches. Accepting both http and https as well as a gemni and gophe could all work as long as the path to the twtxt.txt is the same.
- My idea is that you quote the timestamp as it is in the original twtxt.txt that you are referring to, so you can do it by simply copy/pasting. Also what are the change that the same human will make two different posts within the same second?!
Regarding the whole cryptographic keys for identity, to me it seems like an unnecessary layer of complexity. If you move to a new house or city you tell people that you moved - you can do the same in a twtxt.txt. Just post something like “I move to this new URL, please follow me there!” I did that with my feeds at least twice, and you guys still seem to read my posts:)
The tag URI scheme looks interesting. I like that it human read- and writable. And since we already got the timestamp in the twtxt.txt it would be somewhat trivial to parse. But there are still the issue with what the name/id should be… Maybe it doesn’t have to bee that stick?
Instead of using tag: as the prefix/protocol, it would more it clear what we are talking about by using in-reply-to: (https://indieweb.org/in-reply-to) or replyto: similar to mailto:
(reply:sorenpeter@darch.dk,2024-09-15T12:06:27Z)
(in-reply-to:darch.dk/twtxt.txt,2024-09-15T12:06:27Z)
(replyto:http://darch.dk/twtxt.txt,2024-09-15T12:06:27Z)
I know it’s longer that 7-11 characters, but it’s self-explaining when looking at the twtxt.txt in the raw, and the cases above can all be caught with this regex: \([\w-]*reply[\w-]*\:
Is this something that would work?
@prologic@twtxt.net Brute force. I just hashed a bunch of versions of both tweets until I found a collision.
I mostly just wanted an excuse to write the program. I don’t know how I feel about actually using super-long hashes; could make the twts annoying to read if you prefer to view them untransformed.
They’re in Section 6:
Receiver should adopt UDP GRO. (Something about saving CPU processing UDP packets; I’m a but fuzzy about it.) And they have suggestions for making GRO more useful for QUIC.
Some other receiver-side suggestions: “sending delayed QUICK ACKs”; “using recvmsg to read multiple UDF packets in a single system call”.
Use multiple threads when receiving large files.
@prologic@twtxt.net I believe you when you say registries as designed today do not crawl. But when I first read the spec, it conjured in my mind a search engine. Now I don’t know how things work out in practice, but just based on reading, I don’t see why it can’t be an API for a crawling search engine. (In fact I don’t see anything in the spec indicating registry servers shouldn’t crawl.)
(I also noticed that https://twtxt.readthedocs.io/en/latest/user/registry.html recommends “The registries should sync each others user list by using the users endpoint”. If I understood that right, registering with one should be enough to appear on others, even if they don’t crawl.)
Does yarnd provide an API for finding twts? Is it similar?
@movq@www.uninformativ.de, that would be a nice addition. :-) I would also love the ability to hide/not show the hash when reading twtxts (after all, that’s on the header on each “email”). Could that be added as a user configurable toggle?
Morphotrophic by Greg Egan is built around an idea for how life on Earth could have worked out differently. It gets increasingly strange and interesting as the story progresses. My partner and I finished it last night and thoroughly enjoyed it. The beginning is free online: https://gregegan.net/MORPHOTROPHIC/00/MorphotrophicExcerpt.html #scifi #reading
So updated. Seems to duplicate here in the ui. And what is this “Read More” on every twt now?
@bender@twtxt.net I have nothing against GoToSocial, but:
GoToSocial stores statuses, accounts, etc, in a database. This can be either SQLite or Postgres.
snac is simpler. Some JSON files and that’s it. I can read them with jq and less. I can use tar to back them up. I can hand edit them in a text editor.
@bender@twtxt.net ha! He goes his “poem”:
A string of letters, a forgotten name,
An email crafted, a message to claim.
We hit send with a click, a hopeful sigh,
But a bounce-back arrives, a tear in our eye.“Delivery failed,” the message reads cold,
The address it seems, is a story untold.
A ghost in the system, a memory’s trace,
Lost in the void of cyberspace.
:-D
Thanks @prologic@twtxt.net, I also just manage to get my own version of webmentions working. Please have a read at Webmentions vs. Custom Mentions Spec for Twtxt/Yarn - HedgeDoc and User Lookup for Twtxt/Yarn - Webfinger or Decentralized Identifiers (DIDs) - HedgeDoc for how it sorta works
Did another write up on #webfinger and DIDs for twtxt/yarn that you can read and edit/comment in: User lookup for twtxt/yarn - Webfinger or Decentralized Identifiers (DIDs) - HedgeDoc
> ?
@sorenpeter@darch.dk this makes sense as a quote twt that references a direct URL. If we go back to how it developed on twitter originally it was RT @nick: original text because it contained the original text the twitter algorithm would boost that text into trending.
i like the format (#hash) @<nick url> > "Quoted text"\nThen a comment
as it preserves the human read able. and has the hash for linking to the yarn. The comment part could be optional for just boosting the twt.
The only issue i think i would have would be that that yarn could then become a mess of repeated quotes. Unless the client knows to interpret them as multiple users have reposted/boosted the thread.
The format is also how iphone does reactions to SMS messages with +number liked: original SMS
@lyse@lyse.isobeef.org I have read the white papers for MLS before. I have put a lot of thought on how to do it with salty/ratchet. Its a very good tech for ensuring multiple devices can be joined to an encrypted chat. But it is bloody complicated to implement.
@movq@www.uninformativ.de I lasted for a long time.. Not sure where or when it was “got”. We had been having a cold go around with the kiddos for about a week when the wife started getting sicker than normal. Did a test and she was positive. We tested the rest of the fam and got nothing. Till about 2 days later and myself and the others were positive. It largely hasn’t been too bad a little feaver and stuffy noses.
But whatever it was that hit a few days ago was horrible. Like whatever switch in my head that goes to sleep mode was shut off. I would lay down and even though I felt sleepy, I couldn’t actually go to sleep. The anxiety hit soon after and I was just awake with no relief. And it persisted that way for three nights. I got some meds from the clinic that seemed to finally get me to sleep.
Now the morning after I realized for all that time a part of me was missing. I would close my eyes and it would just go dark. No imagination, no pictures, nothing. Normally I can visualize things as I read or think about stuff.. But for the last few days it was just nothing. The waking up to it was quite shocking.
Though its just the first night.. I guess I’ll have to see if it persists. 🤞
@johanbove@johanbove.info Sounds interesting. It is only for reading or also posting?
@prologic@twtxt.net in the article they say they have a p99 of 15ms reading historical data. Which is pretty nuts.. Aside from having close to 900TB of SSD…
Read this interesting retro about discords migration path from Mongo to Cassandra to now ScyllaDB.
https://discord.com/blog/how-discord-stores-trillions-of-messages
Found another example of Google stealing something I’ve written and putting it in a “featured snippet”.
What’s super annoying about this one is that the source is a course page at Tufts University, not the official page of the publication they’re taking this text from. I know the professor who taught that course and I’ve guest lectured for them before on this topic. They put this publication in their course readings, and I guess that’s where Google picked it up.
@prologic@twtxt.net I’ll second this–I find it very hard to read too.
I’ve only been using snac/the fediverse for a few days and already I’ve had to mute somebody. I know I come on strongly with my opinions sometimes and some people don’t like that, but this person had already started going ad hominem (in my reading of it), and was using what felt to me like sketchy tactics to distract from the point I was trying to make and to shut down conversation. They were doing similar things to other people in the thread so rather than wait for it to get bad for me I just muted them. People get so weirdly defensive so fast when you disagree with something they said online. Not sure I fully understand that.
The more I read from this guy, the more I come to believe he is a gigantic douchecanoe. What a profoundly stupid thing to say.
@abucci@anthony.buc.ci read my new skibloreet about why social meets payments is the next level idea! For just §5 bitshlongs a month on my serfdomage site!
@prologic@twtxt.net Maybe so, but that’s not because of the people who are objecting to Jordan Peterson, that’s for sure. You really need to read the articles I’ve posted before going there. Really.
Taking Jordan Peterson asn an example, the only thing he “preaches” (if you want to call it that) is to be honest with yourself and to take responsibility.
This is simply untrue. Read the articles I posted, seriously.
In a tweet in one of the articles I posted, Peterson states there is no white supremacy in Canada. This is blatantly false. It is disinformation. Peterson has made statements that rape is OK (he uses “fancy” language like “women should be naturally converted into mothers” but unpack that a bit–what he means is legalized rape followed by forced conception). He is openly anti-LGBTQ and refuses to use peoples’ preferred pronouns. He seems to believe that women who wear makeup at work are asking to be sexually harassed.
He’s using his platform in academia to pretend that straight, white men are somehow the most aggrieved group in the world and everyone else is just whining and can get fucked. The patron saint of Men’s Rights Activists and incels. I find him odious.
@prologic@twtxt.net nah, not inclined to do that. The articles suffice–have a read of those when you get the chance.
@prologic@twtxt.net I’ve read half, skimmed the others. Mostly I was going for scale–look at all those headlines. These are horrible people who say horrible things on a regular basis.
I’ve seen BlueSky referred to as BS (as in Blue Sky, but you know…), which seems apt.
CEO is a cryptocurrency fool, as is Jack Dorsey, so I don’t expect much from it. Then again I’m old and refuse to join any new hotness so take my curmudgeonly opinions with a grain of salt.
I read somewhere or another that the “decentralization” is only going to be there so that they can push content moderation onto users. They will happily welcome Nazis and fascists, leaving it up to end users to block those instances.
I wonder how they plan to handle the 4chan-level stuff, since that will surely come.
@abucci@anthony.buc.ci i have an old copy of the 2005 version from university if you want to give it a read through. its quite dry.
@prologic@twtxt.net What is the SMART reading for the disk?
An interesting read about testing code using nullable states instead of mocks.
https://www.jamesshore.com/v2/projects/testing-without-mocks/testing-without-mocks
(cont.)
Just to give some context on some of the components around the code structure.. I wrote this up around an earlier version of aggregate code. This generic bit simplifies things by removing the need of the Crud functions for each aggregate.
Domain ObjectsA domain object can be used as an aggregate by adding the event.AggregateRoot struct and finish implementing event.Aggregate. The AggregateRoot implements logic for adding events after they are either Raised by a command or Appended by the eventstore Load or service ApplyFn methods. It also tracks the uncommitted events that are saved using the eventstore Save method.
type User struct {
Identity string ```json:"identity"`
CreatedAt time.Time
event.AggregateRoot
}
// StreamID for the aggregate when stored or loaded from ES.
func (a *User) StreamID() string {
return "user-" + a.Identity
}
// ApplyEvent to the aggregate state.
func (a *User) ApplyEvent(lis ...event.Event) {
for _, e := range lis {
switch e := e.(type) {
case *UserCreated:
a.Identity = e.Identity
a.CreatedAt = e.EventMeta().CreatedDate
/* ... */
}
}
}
Events
Events are applied to the aggregate. They are defined by adding the event.Meta and implementing the getter/setters for event.Event
type UserCreated struct {
eventMeta event.Meta
Identity string
}
func (c *UserCreated) EventMeta() (m event.Meta) {
if c != nil {
m = c.eventMeta
}
return m
}
func (c *UserCreated) SetEventMeta(m event.Meta) {
if c != nil {
c.eventMeta = m
}
}
Reading Events from EventStore
With a domain object that implements the event.Aggregate the event store client can load events and apply them using the Load(ctx, agg) method.
// GetUser populates an user from event store.
func (rw *User) GetUser(ctx context.Context, userID string) (*domain.User, error) {
user := &domain.User{Identity: userID}
err := rw.es.Load(ctx, user)
if err != nil {
if err != nil {
if errors.Is(err, eventstore.ErrStreamNotFound) {
return user, ErrNotFound
}
return user, err
}
return nil, err
}
return user, err
}
OnX Commands
An OnX command will validate the state of the domain object can have the command performed on it. If it can be applied it raises the event using event.Raise() Otherwise it returns an error.
// OnCreate raises an UserCreated event to create the user.
// Note: The handler will check that the user does not already exsist.
func (a *User) OnCreate(identity string) error {
event.Raise(a, &UserCreated{Identity: identity})
return nil
}
// OnScored will attempt to score a task.
// If the task is not in a Created state it will fail.
func (a *Task) OnScored(taskID string, score int64, attributes Attributes) error {
if a.State != TaskStateCreated {
return fmt.Errorf("task expected created, got %s", a.State)
}
event.Raise(a, &TaskScored{TaskID: taskID, Attributes: attributes, Score: score})
return nil
}
Crud Operations for OnX Commands
The following functions in the aggregate service can be used to perform creation and updating of aggregates. The Update function will ensure the aggregate exists, where the Create is intended for non-existent aggregates. These can probably be combined into one function.
// Create is used when the stream does not yet exist.
func (rw *User) Create(
ctx context.Context,
identity string,
fn func(*domain.User) error,
) (*domain.User, error) {
session, err := rw.GetUser(ctx, identity)
if err != nil && !errors.Is(err, ErrNotFound) {
return nil, err
}
if err = fn(session); err != nil {
return nil, err
}
_, err = rw.es.Save(ctx, session)
return session, err
}
// Update is used when the stream already exists.
func (rw *User) Update(
ctx context.Context,
identity string,
fn func(*domain.User) error,
) (*domain.User, error) {
session, err := rw.GetUser(ctx, identity)
if err != nil {
return nil, err
}
if err = fn(session); err != nil {
return nil, err
}
_, err = rw.es.Save(ctx, session)
return session, err
}
Hi, I am playing with making an event sourcing database. Its super alpha but I thought I would share since others are talking about databases and such.
It’s super basic. Using tidwall/wal as the disk backing. The first use case I am playing with is an implementation of msgbus. I can post events to it and read them back in reverse order.

I plan to expand it to handle other event sourcing type things like aggregates and projections.
Find it here: sour-is/ev
@prologic@twtxt.net @movq@www.uninformativ.de @lyse@lyse.isobeef.org