@eldersnake@we.loveprivacy.club What version are you running btw? It’s probably time you upgraded and time I released a new version finally 😂 If you’re running a version that’s pre-SQLite-cache, then yeah I’m not surprised. The SQLite cache version is honestly much better 🤣

I corrupted my SQLite test database with sed -i s/… $(find …). Clearly, I found too many files. That’s the signal to go to bed.

There are 5 of such “Twters” on this pod that have periods in their nick 😢

sqlite> select count(distinct(nick)) from twters where nick like '%.%';
count(distinct(nick)) = 5
sqlite> select distinct(nick) from twters where nick like '%.%';
 nick = @marado@ciberlandia.pt

 nick = eapl.me

 nick = eapl.mx

 nick = grumpygordie.great-site.net

 nick = @chyrp.doesnm.cc

Hmmm there’s a bug somewhere in the way I’m ingesting archived feeds 🤔

sqlite> select * from twts where content like 'The web is such garbage these days%';
      hash = 37sjhla
  feed_url = https://twtxt.net/user/prologic/twtxt.txt/1
   content = The web is such garbage these days 😔 Or is it the garbage search engines? 🤔
   created = 2024-11-14T01:53:46Z
created_dt = 2024-11-14 01:53:46
   subject = #37sjhla
  mentions = []
      tags = []
     links = []
sqlite>

@abucci@anthony.buc.ci Apologies, the basic summary is as follows:

• Decided to rewrite the cache backend.
  
• It will now be a SQLite backend going forward.
  
• I’m planning on no data migration.

Doesn’t look like it Hmmm

sqlite> select * from twts where content LIKE '%Linux installation%';
    hash = znf6csa
feed_url = https://www.uninformativ.de/twtxt.txt
 content = I wonder if my current Linux installation will actually make it to 20 years:

    $ head -n 1 /var/log/pacman.log
    [2011-07-07 11:19] installed filesystem (2011.04-1)

It’s not toooo far into the future.

It would be crazy … 20 years without reinstalling once … phew. 🥴
 created = 2025-04-07T19:59:51Z
 subject = (#znf6csa)
mentions = []
    tags = []
   links = []

neat! my watcher is currently sitting at about 75 MB following over 1500 feeds. only about 200 are currently somewhat active.

-rw-r--r--. 1 xuu  xuu   69M Mar 25 20:46 twt.db
-rw-r--r--. 1 xuu  xuu   32K Mar 25 21:34 twt.db-shm
-rw-r--r--. 1 xuu  xuu  5.6M Mar 25 21:34 twt.db-wal

sqlite> select state, count(*) n from feeds group by 1;
hot|7
warm|8
cold|183
frozen|743
permanantly-dead|857

re reading so NewRAMStorage(…) is just something that setups your storage and initial data.. that can probably live with storage/sqlite. The point is the storage package does not import the implementations of storage.Storage It just defines the contract for things that use that interface. Now storage/sqlite CAN import storage and not have a circle dep.

It kinda works in reverse for import directions. usually you have your root package that imports things from deeper in the directory structures.. but for the case of interfaces it reverses where the deeper can import from parents but parents cannot import from children.

- app < storage
      < storage/sqlite
      < controller < storage
                   < storage/sqlite
 
- sqlite < storage

- storage X storage/sqlite

@lyse@lyse.isobeef.org OK. So how I have worked things like this out is to have the interface in the root package from the implementations. The interface doesn’t need to be tested since it’s just a contract. The implementations don’t need to import storage.Storage

• storage/ defines the Storage interface (no tests!)
  
  • storage/sqlite for the sqlite implementation tests for sqlite directly
    
  • storage/ram for the ram implementation and tests for RAM directly
    
• controller/ can now import both storage and the implementation as needed.
  

So now I am guessing you wanted the RAM test for testing queries against sqlite and have it return some query response?

For that I usually would register a driver for SQL that emulates sqlite. Then it’s just a matter of passing the connection string to open the registered driver on setup.

https://github.com/glebarez/go-sqlite?tab=readme-ov-file#connection-string-examples

I wrote some code to try out non-hash reply subjects formatted as (replyto ), while keeping the ability to use the existing hash style.

I don’t think we need to decide all at once. If clients add support for a new method then people can use it if they like. The downside of course is that this costs developer time, so I decided to invest a few hours of my own time into a proof of concept.

With apologies to @movq@www.uninformativ.de for corrupting jenny’s beautiful code. I don’t write this expecting you to incorporate the patch, because it does complicate things and might not be a direction you want to go in. But if you like any part of this approach feel free to use bits of it; I release the patch under jenny’s current LICENCE.

Supporting both kinds of reply in jenny was complicated because each email can only have one Message-Id, and because it’s possible the target twt will not be seen until after the twt referencing it. The following patch uses an sqlite database to keep track of known (url, timestamp) pairs, as well as a separate table of (url, timestamp) pairs that haven’t been seen yet but are wanted. When one of those “wanted” twts is finally seen, the mail file gets rewritten to include the appropriate In-Reply-To header.

Patch based on jenny commit 73a5ea81.

https://www.falsifian.org/a/oDtr/patch0.txt

Not implemented:

• Composing twts using the (replyto …) format.
  
• Probably other important things I’m forgetting.

@bender@twtxt.net I have nothing against GoToSocial, but:

GoToSocial stores statuses, accounts, etc, in a database. This can be either SQLite or Postgres.

snac is simpler. Some JSON files and that’s it. I can read them with jq and less. I can use tar to back them up. I can hand edit them in a text editor.

All the scripts on my Gemini capsule (except chess) have now been rewritten using Python and storing data in a SQLite database. This is the first time I’ve ever worked with database in a “production” environment, and I’m inordinately excited.

Posted to Entropy Arbitrage: Small Technology Notes https://john.colagioia.net/blog/2020/02/05/recutils.html #techtips #recutils #linux #sqlite #export