@prologic@twtxt.net Thanks for the invitation. What time of day?
@prologic@twtxt.net Fair enough! I just added some metadata.
↳
In-reply-to
»
Hello twtxt! I'm James (or @falsifian). I live in Toronto. Recent interests include space complexity, simple software, and science fiction.
ā¤ Read More
Thanks @prologic@twtxt.net! I like the way Yarn.social is making all of twtxt stronger, not just Yarn.social pods.
Does anyone care about the 140-char limit recommended by the #twtxt spec? I have been trying to respect it but wonder if itās wasted effort.
↳
In-reply-to
»
Hello twtxt! I'm James (or @falsifian). I live in Toronto. Recent interests include space complexity, simple software, and science fiction.
ā¤ Read More
@movq@www.uninformativ.de Thanks!
Hello twtxt! Iām James (or @falsifian@www.falsifian.org). I live in Toronto. Recent interests include space complexity, simple software, and science fiction.
I would like to work on my Mastodon and TWTXT script to improve it.
@movq@www.uninformativ.de my bad man. I left off a return in the formatter func. I have a PR to fix waiting on @prologic@twtxt.net
@lyse@lyse.isobeef.org wow on my browser it shows up as all stars! ā¢ā¢ā¢ā¢ā¢ā¢ā¢
@prologic@twtxt.net āClownflareā š¤£š¤£š¤£ Love it.
But yes the idea of a cheap VPS as a tunnel and keeping home network all local is a good one I reckon.
↳
In-reply-to
»
Note to self, don't park on a hill in neutral with handbrake off
ā¤ Read More
It ended in a crash but I dreamt it thank God @lyse@lyse.isobeef.org
I learnt how to display a txt file in a html page in a pretty way: https://melyanna.ichi.city/twtxt.html
I just revisited my first script which posts to Mastodon and twtxt from CLI
↳
In-reply-to
»
(#icme3oq) @bender Is it so maxed out you couldn't fit a pretty small program like Headscale on it? Headscale by itself and only personal home type use as far as amount of peers go, it really isn't noticeable I don't think resource-wise. The Docker version I guess could be a different story.
ā¤ Read More
@prologic@twtxt.net Good to know. I must admit Iāve never actually used a Docker instance, probably as I just assumed the overhead might be a bit much for my usual very modest servers.
@bender@twtxt.net Is it so maxed out you couldnāt fit a pretty small program like Headscale on it? Headscale by itself and only personal home type use as far as amount of peers go, it really isnāt noticeable I donāt think resource-wise. The Docker version I guess could be a different story.
↳
In-reply-to
»
@bender on my hosted VPS, as I'm on Starlink which is CGNAT, I need some sort of external intermediary.
ā¤ Read More
@bender@twtxt.net Mine is about the same, though I have 20GB left š In terms of resources, Headscale is using next to nothing though.
↳
In-reply-to
»
I setup and switched to Headscale last night. It was relatively simple, I spent more time installing a web GUI to manage it to be honest, the actual server is simple enough. The native Tailscale Android app even works with it thankfully.
ā¤ Read More
@prologic@twtxt.net Yes I suppose that is true. There is an article on Tailscaleās site that explains it all quite a bit: https://tailscale.com/blog/how-nat-traversal-works
To me, with CGNAT, itās a small miracle that a direct connection can be made between peers (as opposed to going through a relay constantly) but it does indeed work. I guess to host it at home you would need to have it WAN accessible, and if youāve already gone to the trouble of port forwarding etcā¦ well š
Not that I could personally do that, but for those with static IPs etc.
↳
In-reply-to
»
I setup and switched to Headscale last night. It was relatively simple, I spent more time installing a web GUI to manage it to be honest, the actual server is simple enough. The native Tailscale Android app even works with it thankfully.
ā¤ Read More
@bender@twtxt.net on my hosted VPS, as Iām on Starlink which is CGNAT, I need some sort of external intermediary.
↳
In-reply-to
»
I must admit Tailscale is really cool and why I haven't used it before now is beyond me š
ā¤ Read More
@prologic@twtxt.net Interesting! Had no idea about that, but trust you to know of a self-hosted implementation š š
@prologic@twtxt.net I donāt think itās your code. As you said in one of your commit comments, the internet is a hostile place! Thatās partly why I reacted the way I did: all things considered itās usually better to react quickly and clean up the mess later, then it is to wait and risk further damage. Anyway it sucks @xuu@txt.sour.is got caught up in it. Hopefully itās all good now.
@xuu@txt.sour.is I hope everything is sorted out with your ISP. Please let me know if thereās anything I can do to help. I sincerely did not mean to cause you any trouble.
@stigatle@yarn.stigatle.no @xuu@txt.sour.is @lyse@lyse.isobeef.org āNot coolā? I was receiving many broken (HTTP 400 error) requests per second from an IP address I didnāt recognize, right after having my VPS crash because the hard drive filled up with bogus data. None of this had happened on this VPS before, so it was a new problem that I didnāt understand and I took immediate action to get it under control. Of course I reported the IP address to its abuse email. Thatās a 100% normal, natural, and ācoolā thing to do in such a situation. At the time I had no idea it was @xuu@txt.sour.is .
The moment I realized it was @xuu@txt.sour.is and definitely a false alarm, I emailed the ISP and told them this was a false positive and to not ban or block the IP in question because it was not abusive traffic. They havenāt yet responded but I do hope theyāve stopped taking action, and if thereās anything else I can do to certify to them that this is not abuse then I will do that.
I run numerous services on that VPS that I rely on, and I spent most of my day today cleaning up the mess all this has caused. I get that this caused @xuu@txt.sour.is a lot of stress and Iām sincerely sorry about that and am doing what I can to rectify the situation. But calling me ānot coolā isnāt necessary. This was an unfortunate situation that weāre trying to make right and thereās no need for criticizing anyone.
↳
In-reply-to
»
(#2rxkcca) he emailed my ISP about causing logging abuse. This is the only real ISP in my area, its gonna basically send me back to dialup.
ā¤ Read More
@bender@twtxt.net haha funny! though i just realized my ISP is the only one with fiber pulled to the property so i would have to get a phone line from them some how. The other ISP in the area is basically a mobile hotspot.
Hey so.. i just got an email from my ISP saying they will terminate my service. Did i break something @abucci@anthony.buc.ci ?
@prologic@twtxt.net Yes, I can do that.
@stigatle@yarn.stigatle.no @prologic@twtxt.net my /tmp is also fine now! Thanks for your help @prologic@twtxt.net!
@prologic@twtxt.net I unbanned a few IP address I had blocked before the bugfix. I wasnāt being careful and just blocked any IP I saw making a large number of requests to my pod. That slowed the problem down but I think I blocked your and @stigatle@yarn.stigatle.no ās pods in the process, oops.
@stigatle@yarn.stigatle.no Sweet, thank you! Iāve been shooting myself in the foot over here and want to make sure the situation is getting fixed!
@stigatle@yarn.stigatle.no @prologic@twtxt.net testing 1 2 3 can either of you see this?
@prologic@twtxt.net I donāt know if this is new, but Iām seeing:
Jul 25 16:01:17 buc yarnd[1921547]: time="2024-07-25T16:01:17Z" level=error msg="https://yarn.stigatle.no/user/stigatle/twtxt.txt: client.Do fail: Get \"https://yarn.stigatle.no/user/stigatle/twtxt.txt\": dial tcp 185.97.32.18:443: i/o timeout (Client.Timeout exceeded while awaiting headers)" error="Get \"https://yarn.stigatle.no/user/stigatle/twtxt.txt\": dial tcp 185.97.32.18:443: i/o timeout (Client.Timeout exceeded while awaiting headers)"
I no longer see twts from @stigatle@yarn.stigatle.no at all.
@prologic@twtxt.net Have you been seeing any of my replies?
↳
In-reply-to
»
This is a test. I am not seeing twts from @stigatle and it seems like @prologic might not be seeing twts from me. Do people see this?
ā¤ Read More
It shows up in my twtxt feed so thatās good.
This is a test. I am not seeing twts from @stigatle@yarn.stigatle.no and it seems like @prologic@twtxt.net might not be seeing twts from me. Do people see this?
@prologic@twtxt.net I am not seeing twts from @stigatle@yarn.stigatle.no anymore. Are you seeing twts from me?
./tools/dump_cache.sh: line 8: bat: command not found
No Token Provided
I donāt have bat on my VPS and there is no package for installing it. Is cat a reasonable alternate?
@prologic@twtxt.net Try hitting this URL:
https://twtxt.net/external?nick=nosuchuser&uri=https://foo.com
Change nosuchuser to any phrase at all.
If you hit https://twtxt.net/external?nick=nosuchuser , youāre given an error. If you hit that URL above with the uri parameter, you can a legitimate-looking page. I think that is a bug.
@prologic@twtxt.net Hitting that URL returns a bunch of HTML even though there is no user named lovetocode999 on my pod. I think it should 404, and maybe with a delay, to discourage whatever this abuse is. Basically this can be used to DDoS a pod by forcing it to generate a hunch of HTML just by doing a bogus GET like this.
@stigatle@yarn.stigatle.no I used the following hack to keep my VPS from running out of space: watch -n 60 rm -rf /tmp/yarn-avatar-*, run in tmux so it keeps running.
↳
In-reply-to
»
@prologic 10 Gbytes has accumulated since I made that last post. It's coming in at a rate of 55 Mbits/second !
ā¤ Read More
@prologic@twtxt.net There are a lot of logs being generated by yarnd, which is something I havenāt seen before too:
Jul 25 14:32:42 buc yarnd[1911318]: [yarnd] 2024/07/25 14:32:42 (162.211.155.2) "GET /twt/ubhq33a HTTP/1.1" 404 29 643.251Āµs
Jul 25 14:32:43 buc yarnd[1911318]: [yarnd] 2024/07/25 14:32:43 (162.211.155.2) "GET /twt/112073211746755451 HTTP/1.1" 400 12 505.333Āµs
Jul 25 14:32:44 buc yarnd[1911318]: [yarnd] 2024/07/25 14:32:44 (111.119.213.103) "GET /twt/whau6pa HTTP/1.1" 200 37360 35.173255ms
Jul 25 14:32:44 buc yarnd[1911318]: [yarnd] 2024/07/25 14:32:44 (162.211.155.2) "GET /twt/112343305123858004 HTTP/1.1" 400 12 455.069Āµs
Jul 25 14:32:44 buc yarnd[1911318]: [yarnd] 2024/07/25 14:32:44 (168.199.225.19) "GET /external?nick=lovetocode999&uri=http%3A%2F%2Fwww.palapa.pl%2Fbaners.php%3Flink%3Dhttps%3A%2F%2Fwww.dwnewstoday.com HTTP/1.1" 200 36167 19.582077ms
Jul 25 14:32:44 buc yarnd[1911318]: [yarnd] 2024/07/25 14:32:44 (162.211.155.2) "GET /twt/112503061785024494 HTTP/1.1" 400 12 619.152Āµs
Jul 25 14:32:46 buc yarnd[1911318]: [yarnd] 2024/07/25 14:32:46 (162.211.155.2) "GET /twt/111863876118553837 HTTP/1.1" 400 12 817.678Āµs
Jul 25 14:32:46 buc yarnd[1911318]: [yarnd] 2024/07/25 14:32:46 (162.211.155.2) "GET /twt/112749994821704400 HTTP/1.1" 400 12 540.616Āµs
Jul 25 14:32:47 buc yarnd[1911318]: [yarnd] 2024/07/25 14:32:47 (103.204.109.150) "GET /external?nick=lovetocode999&uri=http%3A%2F%2Fampurify.com%2Fbbs%2Fboard.php%3Fbo_table%3Dfree%26wr_id%3D113858 HTTP/1.1" 200 36187 15.95329ms
Iāve seen that nick=lovetocode999 a bunch.
↳
In-reply-to
»
Hack of the day: running
ā¤ Read More
watch -n 60 rm -rf /tmp/yarn-avatar-* in a tmux because all of a sudden, without warning, yarnd started throwing hundreds of gigabytes of files with names like yarn-avatar-62582554 into /tmp, which filled up the entire disk and started crashing other services.
@prologic@twtxt.net Inspect? Whatās sift? What would you like to know about the files?
↳
In-reply-to
»
Hack of the day: running
ā¤ Read More
watch -n 60 rm -rf /tmp/yarn-avatar-* in a tmux because all of a sudden, without warning, yarnd started throwing hundreds of gigabytes of files with names like yarn-avatar-62582554 into /tmp, which filled up the entire disk and started crashing other services.
@prologic@twtxt.net 10 Gbytes has accumulated since I made that last post. Itās coming in at a rate of 55 Mbits/second !
@prologic@twtxt.net I think thereās more to it than that. Iāve updated, yet hundreds of gigabytes of junk is still accumulating.
↳
In-reply-to
»
Hack of the day: running
ā¤ Read More
watch -n 60 rm -rf /tmp/yarn-avatar-* in a tmux because all of a sudden, without warning, yarnd started throwing hundreds of gigabytes of files with names like yarn-avatar-62582554 into /tmp, which filled up the entire disk and started crashing other services.
@prologic@twtxt.net Iām still getting this crap:
abucci@buc:~/yarnd/yarn$ ls -lh /tmp/yarnd-avatar-*
-rw------- 1 abucci abucci 863M Jul 25 14:19 /tmp/yarnd-avatar-1594499680
-rw------- 1 abucci abucci 7.8G Jul 25 14:19 /tmp/yarnd-avatar-2144295337
-rw------- 1 abucci abucci 9.8G Jul 25 14:19 /tmp/yarnd-avatar-2334738193
-rw------- 1 abucci abucci 10G Jul 25 14:14 /tmp/yarnd-avatar-2494107777
-rw------- 1 abucci abucci 9.5G Jul 25 13:59 /tmp/yarnd-avatar-2619243454
-rw------- 1 abucci abucci 11G Jul 25 14:04 /tmp/yarnd-avatar-2922187513
-rw------- 1 abucci abucci 7.5G Jul 25 14:14 /tmp/yarnd-avatar-349775570
-rw------- 1 abucci abucci 10G Jul 25 14:09 /tmp/yarnd-avatar-3640724243
-rw------- 1 abucci abucci 901M Jul 25 14:19 /tmp/yarnd-avatar-3921595598
-rw------- 1 abucci abucci 9.5G Jul 25 13:59 /tmp/yarnd-avatar-609094539
-rw------- 1 abucci abucci 9.3G Jul 25 14:04 /tmp/yarnd-avatar-755173392
-rw------- 1 abucci abucci 7.9G Jul 25 14:09 /tmp/yarnd-avatar-984061000
Something like 100 Gbytes of this junk has accumulated since I updated and re-started the server. Iām now running the latest version of yarnd, so the update did not fix the problem. Something else is going wrong.
How are temporary files growing to 10 Gbytes in size? The name of the file is āyarn-avatarā, but why would avatars be so large?
↳
In-reply-to
»
Hack of the day: running
ā¤ Read More
watch -n 60 rm -rf /tmp/yarn-avatar-* in a tmux because all of a sudden, without warning, yarnd started throwing hundreds of gigabytes of files with names like yarn-avatar-62582554 into /tmp, which filled up the entire disk and started crashing other services.
@prologic@twtxt.net Alright, running yarnd 0.15.1 now. I stopped my hack so weāll see if the VPS gets clogged with junk š
↳
In-reply-to
»
Hack of the day: running
ā¤ Read More
watch -n 60 rm -rf /tmp/yarn-avatar-* in a tmux because all of a sudden, without warning, yarnd started throwing hundreds of gigabytes of files with names like yarn-avatar-62582554 into /tmp, which filled up the entire disk and started crashing other services.
abucci@buc:~/yarnd/yarn$ make preflight
Checking Go version ... [ ERR ]
Go 1.16+ is required, found go1.22.5
FATAL: š preflight failed
make: *** [Makefile:33: preflight] Error 1
š¤
↳
In-reply-to
»
Hack of the day: running
ā¤ Read More
watch -n 60 rm -rf /tmp/yarn-avatar-* in a tmux because all of a sudden, without warning, yarnd started throwing hundreds of gigabytes of files with names like yarn-avatar-62582554 into /tmp, which filled up the entire disk and started crashing other services.
@prologic@twtxt.net Aha, got it. Thanks for looking into it. Iām updating now and weāll see if that stops it.
↳
In-reply-to
»
Hack of the day: running
ā¤ Read More
watch -n 60 rm -rf /tmp/yarn-avatar-* in a tmux because all of a sudden, without warning, yarnd started throwing hundreds of gigabytes of files with names like yarn-avatar-62582554 into /tmp, which filled up the entire disk and started crashing other services.
@prologic@twtxt.net Sure, but why would this start happening all of a sudden today? Nothing like this has happened before. Is this a known bug?
↳
In-reply-to
»
Hack of the day: running
ā¤ Read More
watch -n 60 rm -rf /tmp/yarn-avatar-* in a tmux because all of a sudden, without warning, yarnd started throwing hundreds of gigabytes of files with names like yarn-avatar-62582554 into /tmp, which filled up the entire disk and started crashing other services.
@prologic@twtxt.net 0.15.1, looks like.
↳
In-reply-to
»
Hack of the day: running
ā¤ Read More
watch -n 60 rm -rf /tmp/yarn-avatar-* in a tmux because all of a sudden, without warning, yarnd started throwing hundreds of gigabytes of files with names like yarn-avatar-62582554 into /tmp, which filled up the entire disk and started crashing other services.
@bender@twtxt.net I hope so too. Iāve never seen anything like this before. Whatever it is, itās strange.
@prologic@twtxt.net This is weird, but today, out of nowhere, yarnd filled up the disk on the VPS where I run it. Itās never done anything like this before and I have no idea why it would start. But it threw almost 700 Gbytes of data into /tmp in files like this:
yarnd-avatar-1087570772 yarnd-avatar-1599127133 yarnd-avatar-2042956376 yarnd-avatar-2562946212 yarnd-avatar-3274766535 yarnd-avatar-3931929859 yarnd-avatar-553201529
yarnd-avatar-1089125452 yarnd-avatar-1606826819 yarnd-avatar-2089122560 yarnd-avatar-2611944556 yarnd-avatar-3310922372 yarnd-avatar-3938996661 yarnd-avatar-556240195
yarnd-avatar-1101228867 yarnd-avatar-1618755765 yarnd-avatar-2104107259 yarnd-avatar-2641384948 yarnd-avatar-3326285269 yarnd-avatar-3939402047 yarnd-avatar-559344463
yarnd-avatar-1112165824 yarnd-avatar-1650827505 yarnd-avatar-2142824779 yarnd-avatar-2680659340 yarnd-avatar-3340682113 yarnd-avatar-3998621883 yarnd-avatar-570292705
yarnd-avatar-1119886894 yarnd-avatar-1656673647 yarnd-avatar-2160786463 yarnd-avatar-271923479 yarnd-avatar-3374584613 yarnd-avatar-4005102536 yarnd-avatar-595490106
yarnd-avatar-1131417623 yarnd-avatar-1685698239 yarnd-avatar-2165405940 yarnd-avatar-2793562275 yarnd-avatar-3380606954 yarnd-avatar-4016872095 yarnd-avatar-679251850
yarnd-avatar-1160959085 yarnd-avatar-1746759128 yarnd-avatar-2171489899 yarnd-avatar-2842068287 yarnd-avatar-3416352997 yarnd-avatar-4110048378 yarnd-avatar-679950970
yarnd-avatar-1231649265 yarnd-avatar-1752278279 yarnd-avatar-2251317422 yarnd-avatar-2843868670 yarnd-avatar-3468636088 yarnd-avatar-4116552474 yarnd-avatar-737874628
164 files. Some are empty, some are 7 or even 10 Gbyte.
Any idea what would cause that? And why now, after running yarnd for so long with nothing like this happening?