I share I did write up an algorithm for it at some point I think it is lost in a git comment someplace. I’ll put together a pseudo/go code this week.
Super simple:
Making a reply:
- If yarn has one use that. (Maybe do collision check?)
- Make hash of twt raw no truncation.
- Check local cache for shortest without collision
- in SQL:
select len(subject) where head_full_hash like subject || '%'
- in SQL:
Threading:
- Get full hash of head twt
- Search for twts
- in SQL:
head_full_hash like subject || '%' and created_on > head_timestamp
- in SQL:
The assumption being replies will be for the most recent head. If replying to an older one it will use a longer hash.
🧮 USERS:1 FEEDS:2 TWTS:1111 ARCHIVED:79666 CACHE:2610 FOLLOWERS:17 FOLLOWING:14
🧮 USERS:1 FEEDS:2 TWTS:1110 ARCHIVED:79632 CACHE:2622 FOLLOWERS:17 FOLLOWING:14
🧮 USERS:1 FEEDS:2 TWTS:1109 ARCHIVED:79618 CACHE:2654 FOLLOWERS:17 FOLLOWING:14
Ok, i know how to command working (not sure), but seems it only grab from cache. Maybe make fetch from twtxt.net if hash not found?
🧮 USERS:1 FEEDS:2 TWTS:1108 ARCHIVED:79604 CACHE:2650 FOLLOWERS:17 FOLLOWING:14
It has twts cache which used if timeline is set to jew. Maybe i.should fork twet to make wishes like newlines (i see two squares), showing conversations, showing twts if not found in cache and parsing medata to configure url, nick and followers (currenly it duplicated in config and twtxt file)
🧮 USERS:1 FEEDS:2 TWTS:1107 ARCHIVED:79577 CACHE:2662 FOLLOWERS:17 FOLLOWING:14
🧮 USERS:1 FEEDS:2 TWTS:1106 ARCHIVED:79449 CACHE:2645 FOLLOWERS:17 FOLLOWING:14
🧮 USERS:1 FEEDS:2 TWTS:1105 ARCHIVED:79381 CACHE:2634 FOLLOWERS:17 FOLLOWING:14
🧮 USERS:1 FEEDS:2 TWTS:1104 ARCHIVED:79360 CACHE:2633 FOLLOWERS:17 FOLLOWING:14
🧮 USERS:1 FEEDS:2 TWTS:1103 ARCHIVED:79345 CACHE:2623 FOLLOWERS:17 FOLLOWING:14
x86 Embedded Controller with PC/104 Compatibility for Legacy Systems
The VDX3-6757 PC/104 family of low-power x86 embedded controllers meets PC/104 specifications, offering backward compatibility for projects facing end-of-life x86-based controllers. It is suited for applications like data acquisition, industrial automation, process control, and automotive control. Powered by a DM&P Vortex86DX3 1GHz dual-core CPU with 32KB L1 cache and 512KB L2 cache, the VDX3-6757 supports … ⌘ Read more
🧮 USERS:1 FEEDS:2 TWTS:1102 ARCHIVED:79309 CACHE:2611 FOLLOWERS:17 FOLLOWING:14
🧮 USERS:1 FEEDS:2 TWTS:1101 ARCHIVED:79243 CACHE:2591 FOLLOWERS:17 FOLLOWING:14
@prologic@twtxt.net Thanks for writing that up!
I hope it can remain a living document (or sequence of draft revisions) for a good long time while we figure out how this stuff works in practice.
I am not sure how I feel about all this being done at once, vs. letting conventions arise.
For example, even today I could reply to twt abc1234 with “(#abc1234) Edit: …” and I think all you humans would understand it as an edit to (#abc1234). Maybe eventually it would become a common enough convention that clients would start to support it explicitly.
Similarly we could just start using 11-digit hashes. We should iron out whether it’s sha256 or whatever but there’s no need get all the other stuff right at the same time.
I have similar thoughts about how some users could try out location-based replies in a backward-compatible way (append the replyto: stuff after the legacy (#hash) style).
However I recognize that I’m not the one implementing this stuff, and it’s less work to just have everything determined up front.
Misc comments (I haven’t read the whole thing):
Did you mean to make hashes hexadecimal? You lose 11 bits that way compared to base32. I’d suggest gaining 11 bits with base64 instead.
“Clients MUST preserve the original hash” — do you mean they MUST preserve the original twt?
Thanks for phrasing the bit about deletions so neutrally.
I don’t like the MUST in “Clients MUST follow the chain of reply-to references…”. If someone writes a client as a 40-line shell script that requires the user to piece together the threading themselves, IMO we shouldn’t declare the client non-conforming just because they didn’t get to all the bells and whistles.
Similarly I don’t like the MUST for user agents. For one thing, you might want to fetch a feed without revealing your identty. Also, it raises the bar for a minimal implementation (I’m again thinking again of the 40-line shell script).
For “who follows” lists: why must the long, random tokens be only valid for a limited time? Do you have a scenario in mind where they could leak?
Why can’t feeds be served over HTTP/1.0? Again, thinking about simple software. I recently tried implementing HTTP/1.1 and it wasn’t too bad, but 1.0 would have been slightly simpler.
Why get into the nitty-gritty about caching headers? This seems like generic advice for HTTP servers and clients.
I’m a little sad about other protocols being not recommended.
I don’t know how I feel about including markdown. I don’t mind too much that yarn users emit twts full of markdown, but I’m more of a plain text kind of person. Also it adds to the length. I wonder if putting a separate document would make more sense; that would also help with the length.
🧮 USERS:1 FEEDS:2 TWTS:1100 ARCHIVED:79197 CACHE:2589 FOLLOWERS:17 FOLLOWING:14
🧮 USERS:1 FEEDS:2 TWTS:1099 ARCHIVED:79147 CACHE:2577 FOLLOWERS:17 FOLLOWING:14
🧮 USERS:1 FEEDS:2 TWTS:1098 ARCHIVED:79066 CACHE:2530 FOLLOWERS:17 FOLLOWING:14
@david@collantes.us Well, I wouldn’t recommend using my code for your main jenny use anyway. If you want to try it out, set XDG_CONFIG_HOME and XDG_CACHE_HOME to some sandbox directories and only run my code there. If @movq@www.uninformativ.de is interested in any of this getting upstreamed, I’d be happy to try rebasing the changes, but otherwise it’s a proof of concept and fun exercise.
BTW this code doesn’t incorporate existing twts into jenny’s database. It’s best used starting from scratch. I’ve been testing it using a custom XDG_CACHE_HOME and XDG_CONFIG_HOME to avoid messing with my “real” jenny data.
🧮 USERS:1 FEEDS:2 TWTS:1097 ARCHIVED:79000 CACHE:2495 FOLLOWERS:17 FOLLOWING:14
🧮 USERS:1 FEEDS:2 TWTS:1096 ARCHIVED:78954 CACHE:2471 FOLLOWERS:17 FOLLOWING:14
🧮 USERS:1 FEEDS:2 TWTS:1095 ARCHIVED:78843 CACHE:2434 FOLLOWERS:17 FOLLOWING:14
🧮 USERS:1 FEEDS:2 TWTS:1094 ARCHIVED:78808 CACHE:2451 FOLLOWERS:17 FOLLOWING:14
🧮 USERS:1 FEEDS:2 TWTS:1093 ARCHIVED:78768 CACHE:2438 FOLLOWERS:17 FOLLOWING:14
🧮 USERS:1 FEEDS:2 TWTS:1092 ARCHIVED:78761 CACHE:2445 FOLLOWERS:17 FOLLOWING:14
🧮 USERS:1 FEEDS:2 TWTS:1091 ARCHIVED:78750 CACHE:2482 FOLLOWERS:17 FOLLOWING:14
🧮 USERS:1 FEEDS:2 TWTS:1090 ARCHIVED:78738 CACHE:2498 FOLLOWERS:17 FOLLOWING:14
🧮 USERS:1 FEEDS:2 TWTS:1089 ARCHIVED:78724 CACHE:2505 FOLLOWERS:17 FOLLOWING:14
🧮 USERS:1 FEEDS:2 TWTS:1088 ARCHIVED:78704 CACHE:2506 FOLLOWERS:17 FOLLOWING:14
🧮 USERS:1 FEEDS:2 TWTS:1087 ARCHIVED:78676 CACHE:2491 FOLLOWERS:17 FOLLOWING:14
🧮 USERS:1 FEEDS:2 TWTS:1086 ARCHIVED:78278 CACHE:2434 FOLLOWERS:17 FOLLOWING:14
🧮 USERS:1 FEEDS:2 TWTS:1085 ARCHIVED:78244 CACHE:2417 FOLLOWERS:17 FOLLOWING:14
🧮 USERS:1 FEEDS:2 TWTS:1084 ARCHIVED:78231 CACHE:2434 FOLLOWERS:17 FOLLOWING:14
🧮 USERS:1 FEEDS:2 TWTS:1083 ARCHIVED:78202 CACHE:2449 FOLLOWERS:17 FOLLOWING:14
🧮 USERS:1 FEEDS:2 TWTS:1082 ARCHIVED:78194 CACHE:2455 FOLLOWERS:17 FOLLOWING:14
🧮 USERS:1 FEEDS:2 TWTS:1081 ARCHIVED:78172 CACHE:2486 FOLLOWERS:17 FOLLOWING:14
🧮 USERS:1 FEEDS:2 TWTS:1080 ARCHIVED:78169 CACHE:2484 FOLLOWERS:17 FOLLOWING:14
🧮 USERS:1 FEEDS:2 TWTS:1079 ARCHIVED:78129 CACHE:2461 FOLLOWERS:17 FOLLOWING:14
🧮 USERS:1 FEEDS:2 TWTS:1078 ARCHIVED:78117 CACHE:2455 FOLLOWERS:17 FOLLOWING:14
🧮 USERS:1 FEEDS:2 TWTS:1077 ARCHIVED:78103 CACHE:2449 FOLLOWERS:17 FOLLOWING:14
🧮 USERS:1 FEEDS:2 TWTS:1076 ARCHIVED:77763 CACHE:2433 FOLLOWERS:17 FOLLOWING:14
@prologic@twtxt.net My pod, which is running the same commit you are, does not return an error like that. It returns the same HTML it always has. Try it. I nuked my cache before restarting.
Edit: Oh wait, the plot thickens. I do get an error if I use curl or if I use a web browser that isn’t logged in. That’s good!
@prologic@twtxt.net OK, I just updated to commit 77d527, which looks to be the same one you’re running right now. I forgot to blow away my cache before restarting, so I just deleted the cache file and restarted.
🧮 USERS:1 FEEDS:2 TWTS:1075 ARCHIVED:77751 CACHE:2431 FOLLOWERS:17 FOLLOWING:14
yarnd that's been around for awhile and is still present in the current version I'm running that lets a person hit a constructed URL like
@prologic@twtxt.net I deleted a file named cache in my yarn data and restarted. Problem persists.
yarnd that's been around for awhile and is still present in the current version I'm running that lets a person hit a constructed URL like
@prologic@twtxt.net “Refresh cache in Poderator Settings”
Is there some other way to do that?
yarnd that's been around for awhile and is still present in the current version I'm running that lets a person hit a constructed URL like
@prologic@twtxt.net This does not seem to fix the problem for me, or I’ve done something wrong. I did the following:
- Pull the latest version from
git(I have commit7ad848, same as ontwtxt.netI believe).
make buildandmake install
- Restart
yarnd
- Refresh cache in Poderator Settings
Yet I still see these bogus /external things on my pod when I hit URLs like the one I sent you recently. When I hit such a URL with curl I think it’s giving an error? But in a web browser, the (buggy) response is the same as it was before I updated.
So, this problem is not fixed for me.
🧮 USERS:1 FEEDS:2 TWTS:1074 ARCHIVED:77719 CACHE:2428 FOLLOWERS:17 FOLLOWING:14