Edit #issuecomment-18968

@eapl.me@eapl.me @andros@twtxt.andros.dev here’s another variant as per eapl’s suggestion (#issuecomment-18968)[https://git.mills.io/yarnsocial/twtxt.dev/issues/9#issuecomment-18968]

i made a little twtxt feed fixer for when a feed uses other whitespace instead of tabs.

https://git.mills.io/yarnsocial/go-lextwt/pulls/32

@prologic@twtxt.net we need to remove: https://git.mills.io/yarnsocial/go-lextwt/src/branch/main/ast.go#L776-L784

apparently i can’t make the edit via gitea.. i am guessing its hitting one of your firewall rules.

@prologic@twtxt.net I believe @andros@twtxt.andros.dev is referring to the one on the original twtxt docs . I’ve been meaning to contribute to the discussion on the git but I’m just lazy 😅 amma throw in a little something in a minute Poke a bee hive and run away style 😆

@prologic@twtxt.net Just in case… the git link is missing a c in prologic

I’ve started a draft over at: https://git.mills.io/yarnsocial/twtxt.dev/src/branch/main/exts/webfinger.md

@prologic@twtxt.net just rebuild my image.. though git says i am already at latest

@eapl.me@eapl.me Neat.

So for twt metadata the lextwt parser currently supports values in the form [key=value]

https://git.mills.io/yarnsocial/go-lextwt/src/branch/main/parser_test.go#L692-L698

I share I did write up an algorithm for it at some point I think it is lost in a git comment someplace. I’ll put together a pseudo/go code this week.

Super simple:

Making a reply:

1. If yarn has one use that. (Maybe do collision check?)
   
2. Make hash of twt raw no truncation.
   
3. Check local cache for shortest without collision
   
   • in SQL: select len(subject) where head_full_hash like subject || '%'
     

Threading:

1. Get full hash of head twt
   
2. Search for twts
   
   • in SQL: head_full_hash like subject || '%' and created_on > head_timestamp
     

The assumption being replies will be for the most recent head. If replying to an older one it will use a longer hash.

Diving into mblaze, I think I’ve nearly* reached peek email geek.

Just a bunch of shell commands I can pipe together to search, list, view and reply to email (after syncing it to a local Maildir).

EXAMPLES at https://git.vuxu.org/mblaze/tree/README

So far I’m using most of the tools directly from the command line, but I might take inspiration from https://sr.ht/~rakoo/omail/ to make my workflow a bit more efficient.

*To get any closer, I think I’d have to hand-craft my own SMTP client or something.

I forgot to git add a new test file. Added to the patch now at https://www.falsifian.org/a/oDtr/patch0.txt

Oh. looks like its 4 chars. git show 64bf

the stem matching is the same as how GIT does its branch hashes. i think you can stem it down to 2 or 3 sha bytes.

if a client sees someone in a yarn using a byte longer hash it can lengthen to match since it can assume that maybe the other client has a collision that it doesnt know about.

@prologic@twtxt.net Wikipedia claims sha1 is vulnerable to a “chosen-prefix attack”, which I gather means I can write any two twts I like, and then cause them to have the exact same sha1 hash by appending something. I guess a twt ending in random junk might look suspcious, but perhaps the junk could be worked into an image URL like

Image

git only uses sha1 because they’re stuck with it: migrating is very hard. There was an effort to move git to sha256 but I don’t know its status. I think there is progress being made with Game Of Trees, a git clone that uses the same on-disk format.

I can’t imagine any benefit to using sha1, except that maybe some very old software might support sha1 but not sha256.

Hey, @movq@www.uninformativ.de, a tiny thing to add to jenny, a -v switch. That way when you twtxt “That’s an older format that was used before jenny version v23.04”, I can go and run jenny -v, and “duh!” myself on the way to a git pull. :-D

@falsifian@www.falsifian.org yes, that happened around 2 years ago, on commit 5923078ea5.

Opened a couple of issues on twtxt2html. Maybe @prologic@twtxt.net will get to them after he has completed his luxurious recharging cycle. LOL.

@prologic@twtxt.net This does not seem to fix the problem for me, or I’ve done something wrong. I did the following:

1. Pull the latest version from git (I have commit 7ad848, same as on twtxt.net I believe).
   
2. make build and make install
   
3. Restart yarnd
   
4. Refresh cache in Poderator Settings
   

Yet I still see these bogus /external things on my pod when I hit URLs like the one I sent you recently. When I hit such a URL with curl I think it’s giving an error? But in a web browser, the (buggy) response is the same as it was before I updated.

So, this problem is not fixed for me.

@lyse@lyse.isobeef.org Interesting. The yarnd --help currently says (for me):

  -R, --open-registrations            whether or not to have open user registgration

meaning it doesn’t give the default setting or warn you that you need to use -R=false and not -R false. It also leaves unclear whether --open-registrations false would work or if you need to do --open-registrations=false. It’s also unclear whether the setting change in the user interface is overridden by the command line arguments, overrides the command line arguments, is persisted across restarts.

Maybe all this is worth posting an issue for additional documentation on the git repo if there isn’t one already.

“registgration” is misspelled that way in the help by the way.

@prologic@twtxt.net Righteo, so rookie error - I obviously had some untracked, rather important files for starting my pod and I ran a make clean. Why I originally had them in the git directory is anyone’s guess. Anyway it blew away those files including the database so that’s that. So your good self and @bender@twtxt.net etc - apologies but your profiles got nuked as well (as did my own but easily recreated).

Another thing I noticed which was the reason I ran make clean in the first place. I noticed my pod was being built with Go 1.22.4. Could this be a problem @prologic? preflight.sh actually errors out about it…

@lyse@lyse.isobeef.org

You can’t catch the kill signal. Should this be syscall.SIGTERM instead of os.Kill, xuu? https://git.sour.is/sour-is/go-paste/src/branch/main/main.go#L21

You are totally right.. i think i was going for SIGTERM and SIGQUIT

@prologic@twtxt.net why am I getting this on your git?

Twtxt spec enhancement proposal thread 🧵

Adding attributes to individual twts similar to adding feed attributes in the heading comments.

https://git.mills.io/yarnsocial/go-lextwt/pulls/17

The basic use case would be for multilingual feeds where there is a default language and some twts will be written a different language.

As seen in the wild: https://eapl.mx/twtxt.txt

The attributes are formatted as [key=value]

They can show up in the twt anywhere it is not enclosed by another element such as codeblock or part of a markdown link.

So, I finally got day 17 to under a second on my machine. (in the test runner it takes 10)

I implemented a Fibonacci Heap to replace the priority queue to great success.

https://git.sour.is/xuu/advent-of-code/src/branch/main/search.go#L168-L268

OH MY FREAKING HECK. So.. I made my pather able to run as Dijkstra or A* if the interface includes a heuristic.. when i tried without the heuristic it finished faster :|

So now to figure out why its not working right.

man… day17 has been a struggle for me.. i have managed to implement A* but the solve still takes about 2 minutes for me.. not sure how some are able to get it under 10 seconds.

Solution: https://git.sour.is/xuu/advent-of-code/src/branch/main/day17/main.go
A* PathFind: https://git.sour.is/xuu/advent-of-code/src/branch/main/search.go

some seem to simplify the seen check to only be horizontal/vertical instead of each direction.. but it doesn’t give me the right answer

@movq@www.uninformativ.de So.. i eventually made it to the end on this one.. was able to reuse code from days 8 and 9!

SSBzdGlsbCBkbyBub3QgdW5kZXJzdGFuZCB3aHkgdXNpbmcgdGhlIHJhdGUgb2YgY2hhbmdlIGlu
IHRoZSBwdXNoZXMgZ2l2ZXMgbWUgdGhlIGFuc3dlci4uIGJ1dCB5ZWFoLi4K

https://git.sour.is/xuu/advent-of-code/pulls/13/files

All brute force.

Its the latest ryzen 7 chipset for laptop/mini form factor.

I am very surprised about the times others are getting. I guess that’s the difference between interpreted and compiled showing.

@movq@www.uninformativ.de Dang. Really going overboard with this!

@prologic@twtxt.net I didn’t have to do much backtracking. I parsed into an AST-ish table and then just needed some lookups.

The part 2 was pretty easy to work into the AST after.

https://git.sour.is/xuu/advent-of-code-2023/commit/c894853cbd08d5e5733dfa14f22b249d0fb7b06c

My code is here. https://git.sour.is/xuu/advent-of-code-2023

@darch@neotxt.dk webmentions are dispatched from here https://git.mills.io/yarnsocial/yarn/src/branch/main/internal/post_handler.go#L160-L169

Gracias. Also the git repo now contain code that should actually work

@lyse@lyse.isobeef.org I wish more standardization around distributed issues and PRs within the repo ala git-bug was around for this. I see it has added some bridge tooling now.

It seems like https://proxy.vulpes.one/ runs a code that once was written by @prologic@twtxt.net. Its rendering looks quite nice. Sadly, I am unable to compile it (modified code at https://git.vulpes.one/gopherproxy/).

I have submitted this to be used as the hash tooling for Yarn. See it as a good example on using this in a production environment!

https://git.mills.io/yarnsocial/yarn/pulls/1095

Ah git-bug! Ive chatted with the creator when he was working on the graphql parts. Its working with git objects directly sorta like how git-repo does code reviews. Its a pretty neat idea for storing data along side the branches. I believe they don’t add a disconnected branch to avoid data getting corrupted by merging branches or something like that.

I have found the issue with this very subtle bug.. the cache was returning a slice that would be mutated. The mutation involved appending an item and then sorting. because the returned slice is just a pointer+length the sort would modify the same memory.

          CACHE         Returned slice          
original: [A B C D]     [A B C D]
add:      [A B C D] E   [A B C D E]
sort:     [E A B C] D   [A B C D E]

fix found here:
https://git.mills.io/yarnsocial/yarn/pulls/1072

Hi, I am playing with making an event sourcing database. Its super alpha but I thought I would share since others are talking about databases and such.

It’s super basic. Using tidwall/wal as the disk backing. The first use case I am playing with is an implementation of msgbus. I can post events to it and read them back in reverse order.

I plan to expand it to handle other event sourcing type things like aggregates and projections.

Find it here: sour-is/ev

@prologic@twtxt.net @movq@www.uninformativ.de @lyse@lyse.isobeef.org

@prologic@twtxt.net

#!/bin/sh

# Validate environment
if ! command -v msgbus > /dev/null; then
    printf "missing msgbus command. Use:  go install git.mills.io/prologic/msgbus/cmd/msgbus@latest"
    exit 1
fi

if ! command -v salty > /dev/null; then
    printf "missing salty command. Use:  go install go.mills.io/salty/cmd/salty@latest"
    exit 1
fi

if ! command -v salty-keygen > /dev/null; then
    printf "missing salty-keygen command. Use:  go install go.mills.io/salty/cmd/salty-keygen@latest"
    exit 1
fi

if [ -z "$SALTY_IDENTITY" ]; then
    export SALTY_IDENTITY="$HOME/.config/salty/$USER.key"
fi

get_user () {
    user=$(grep user: "$SALTY_IDENTITY" | awk '{print $3}')
    if [ -z "$user" ]; then
        user="$USER"
    fi
    echo "$user"
}

stream () {
    if [ -z "$SALTY_IDENTITY" ]; then
        echo "SALTY_IDENTITY not set"
        exit 2
    fi

    jq -r '.payload' | base64 -d | salty -i "$SALTY_IDENTITY" -d
}

lookup () {
    if [ $# -lt 1 ]; then
    printf "Usage: %s nick@domain\n" "$(basename "$0")"
    exit 1
    fi

    user="$1"
    nick="$(echo "$user" | awk -F@ '{ print $1 }')"
    domain="$(echo "$user" | awk -F@ '{ print $2 }')"

    curl -qsSL "https://$domain/.well-known/salty/${nick}.json"
}

readmsgs () {
    topic="$1"

    if [ -z "$topic" ]; then
        topic=$(get_user)
    fi

    export SALTY_IDENTITY="$HOME/.config/salty/$topic.key"
    if [ ! -f "$SALTY_IDENTITY" ]; then
        echo "identity file missing for user $topic" >&2
        exit 1
    fi

    msgbus sub "$topic" "$0"
}

sendmsg () {
    if [ $# -lt 2 ]; then
        printf "Usage: %s nick@domain.tld <message>\n" "$(basename "$0")"
        exit 0
    fi

    if [ -z "$SALTY_IDENTITY" ]; then
        echo "SALTY_IDENTITY not set"
        exit 2
    fi

    user="$1"
    message="$2"

    salty_json="$(mktemp /tmp/salty.XXXXXX)"

    lookup "$user" > "$salty_json"

    endpoint="$(jq -r '.endpoint' < "$salty_json")"
    topic="$(jq -r '.topic' < "$salty_json")"
    key="$(jq -r '.key' < "$salty_json")"

    rm "$salty_json"

    message="[$(date +%FT%TZ)] <$(get_user)> $message"

    echo "$message" \
        | salty -i "$SALTY_IDENTITY" -r "$key" \
        | msgbus -u "$endpoint" pub "$topic"
}

make_user () {
    mkdir -p "$HOME/.config/salty"

    if [ $# -lt 1 ]; then
        user=$USER
    else
        user=$1
    fi

    identity_file="$HOME/.config/salty/$user.key"

    if [ -f "$identity_file" ]; then
        printf "user key exists!"
        exit 1
    fi

    # Check for msgbus env.. probably can make it fallback to looking for a config file?
    if [ -z "$MSGBUS_URI" ]; then
        printf "missing MSGBUS_URI in environment"
        exit 1
    fi


    salty-keygen -o "$identity_file"
    echo "# user: $user" >> "$identity_file"

    pubkey=$(grep key: "$identity_file" | awk '{print $4}')

    cat <<- EOF
Create this file in your webserver well-known folder. https://hostname.tld/.well-known/salty/$user.json

{
  "endpoint": "$MSGBUS_URI",
  "topic": "$user",
  "key": "$pubkey"
}

EOF
}

# check if streaming
if [ ! -t 1 ]; then
    stream
    exit 0
fi

# Show Help
if [ $# -lt 1 ]; then
    printf "Commands: send read lookup"
    exit 0
fi


CMD=$1
shift

case $CMD in
    send)
        sendmsg "$@"
    ;;
    read)
        readmsgs "$@"
    ;;
    lookup)
        lookup "$@"
    ;;
    make-user)
        make_user "$@"
    ;;
esac

Yep! https://git.mills.io/yarnsocial/yarn/src/branch/master/internal/webmention/webmention.go#L150-L156

@movq@www.uninformativ.de love having fresh changes coming down with a git pull with my tea in the morning. Thank you much!

@fastidious@arrakis.netbros.com fixed this one yesterday. https://git.mills.io/yarnsocial/yarn/pulls/502

line in question: https://git.mills.io/yarnsocial/yarn/src/branch/master/types/lextwt/ast.go#L793

@movq@www.uninformativ.de Awesome! About to git pull, and check it out. Thanks!

@movq@www.uninformativ.de Nice! I am git pulling right after this. 🙈

@quark@ferengi.one If so, @movq@www.uninformativ.de, not quite https://www.uninformativ.de/git/jenny/commit/4a02eeec58317107c07e759733312d168e319f17.html#h0-0-5... Markdown needs single new lines for many things. Bulleted, numbered, code, etc. need them.

On the blog: Using git to Count Changed Words https://john.colagioia.net/blog/2021/10/13/words-changed.html #programming #techtips #blog #git

How would jenny handle multiline twts? Let’s find out! - One - Two - Three And: 1. One 2. Two 3. Three

@movq@www.uninformativ.de I did a git pull but that isn’t pulling jenny-posting.eml. Am I missing something?