@feld

> I spent so many hours fighting with IPFS I think it's a dead end.

Yeah, that's my conclusion, too. It should be great but is not even good. FSE had the start of a patch but I didn't get too far actually integrating it. (
https://git.freespeechextremist.com/gitweb/?p=fse;a=commit;h=2dec0a138612eebe79f757b5bc04810b81d65951 )
> Nobody working on Kubo has any interest in fixing the storage scalability problems it seems.

I think it's a little past them, if I'm being honest. They could probably get away with just integrating tkrzw.

> To use it for fedi we'd really want the garbage collection to work and not spin 100% CPU and IO for hours but that's what it does when you get enough data in there 🤬

Yeah, you wanna see a real horror, look at the directory where it stores the data. I don't know if it's still doing that but it was storing different permutations of the same hash as different files last I checked. So the Qm$whatever or the bafy$whatever, those were just stored as separate files instead of normalizing the hash.

8MB is too large for you to get fortuitous collisions, it's not a very good chunk size for transfer, just the entire thing is like a parody.