block_accounting.png

Notices where this attachment appears

1. pistolero (p@fsebugoutzone.org)'s status on Thursday, 15-Aug-2024 14:22:09 JST pistolero

   in reply to
   You can see the periodic pauses in the Apotheosis Stream, those are caused by flushing the IDIOT (which should possibly be done more frequently, as at present it actually makes a couple of the child processes time out when it does this).
   
   Visible in the Prometheus stats is the count of blocks and the count of activities, from which we can get the number of blocks required per activity. Since it's content-addressed, the number of blocks per activity gradually goes down. (It'll go down faster when I finish up that stupid JSON thing I was threatening to do.)
   
   Since screamshitter.club handles FSE's media uploads, its average block size stays near the maximum (8192 bytes): videos/images/mp3s are already compressed so they don't usually get deflated, and because the files are bigger than activities in JSON, you've got, like, a thousand full blocks and one at the end, dragging the average towards the maximum. When more posts arrive there than uploads on FSE, then it pulls the average down.
   storage_accounting.png
   ingestion_process.png
   block_accounting.png