My very strong opinion on the use of Twtxt is if you intend to use it, you should be prepared to let people pull your feed or at least check it and regular rentals.

Otherwise get out and go use something that’s either a distributed (Mastodon, AT, etc) or centralized (Facebook, X, etc) network.

@Codebuzz@www.codebuzz.nl I agree. At the end of the day it’s just a text file served by some means, we should get more upset by crawlers that don’t respect our robots.tzt rules 🤣

@prologic@twtxt.net Yeah, that’s what I don’t understand about this behavior. Once you put up a web server on the internet, you’ll be hit by so many bots instantly – the few hits on my twtxt file hardly matter … 🤷

🤣 Indeed. It’s like using disallow is only pointing them to look for more stuff to index. Those few kB’s for a twtxt file, meh.. shouldnt even be a discussion.

Yeah 😔