Conversation

Notices

1. Tadano (tadano@mt.watamelon.win)'s status on Monday, 03-Feb-2025 23:36:38 JST Tadano

   • silverpill

   @silverpill if you're able to see this post, for whatever reason federation's a tad borked on my server at the moment. All the posts I'm getting on the TL are from 3 days ago minimum and no new posts seem to be coming in. Where should I look to check for anything wrong?

   • silverpill (silverpill@mitra.social)'s status on Monday, 03-Feb-2025 23:36:37 JST silverpill

     in reply to

     @tadano Run mitractl instance-report, it will show how many messages you have in the incoming federation queue. Maybe something is blocking it.
     And check your log, is there any activity? Any errors?

   • Tadano ❄️🎅 (tadano@amala.schwartzwelt.xyz)'s status on Tuesday, 04-Feb-2025 00:08:00 JST Tadano ❄️🎅

     in reply to

     • silverpill
     @silverpill @tadano Posting from here because the response hasn't federated yet. Checking journalctl gives a whole bunch of requests, and I don't see anything in /var/log for mitra. Remind me where the logs are again?
   • silverpill (silverpill@mitra.social)'s status on Tuesday, 04-Feb-2025 00:08:00 JST silverpill

     in reply to

     • Tadano ❄️🎅

     @Tadano @tadano

     journalctl -u mitra -o cat

     if you installed the deb package.

     Instance report confirms that incoming message queue was blocked for some reason. Normally its size is less than 10.

     Now we need to somehow find out why it didn't progress. Maybe this will provide some clues:

     journalctl -u mitra -o cat | grep timeout
   • Tadano ❄️🎅 (tadano@amala.schwartzwelt.xyz)'s status on Tuesday, 04-Feb-2025 00:49:28 JST Tadano ❄️🎅

     in reply to

     • silverpill

     @silverpill @tadano Ran with journalctl --since "2025-01-29" -u mitra -o cat | grep timeout and it came back blank.

   • silverpill (silverpill@mitra.social)'s status on Tuesday, 04-Feb-2025 00:49:28 JST silverpill

     in reply to

     • Tadano ❄️🎅

     @Tadano @tadano Try grepping for "ERROR" and "panic".
     What did you do before it started federating again? A restart?

   • silverpill (silverpill@mitra.social)'s status on Tuesday, 04-Feb-2025 01:01:57 JST silverpill

     in reply to

     • Tadano ❄️🎅

     @Tadano @tadano I think it was fixed by restarting mitra. This means there was a "panic" that killed the part of mitra responsible for processing incoming messages

     In that is true, you should see this in your journalctl output:

     thread 'main' panicked at ...
   • Tadano ❄️🎅 (tadano@amala.schwartzwelt.xyz)'s status on Tuesday, 04-Feb-2025 01:01:58 JST Tadano ❄️🎅

     in reply to

     • silverpill

     @silverpill @tadano Before, it wasn't fetching any new posts whatsoever. So I stopped the mitra service, cleared some extraneous posts from the last day, and restarted both it and postgres. Also, before restarting postgres, while the mitra service was off I ran VACUUM; in psql. Nothing happened, however. After the restarts it started federating, though at the slow rate and in the chronological order you see now.

   • silverpill (silverpill@mitra.social)'s status on Tuesday, 04-Feb-2025 02:42:00 JST silverpill

     in reply to

     • Tadano ❄️🎅

     @Tadano @tadano Thanks, I'll fix that.
     Do you want to add polls to your replies? I removed the "poll" icon from the reply form in the development version to save some space on small screens, but may bring it back if you need it.

   • Tadano ❄️🎅 (tadano@amala.schwartzwelt.xyz)'s status on Tuesday, 04-Feb-2025 02:42:01 JST Tadano ❄️🎅

     in reply to

     • silverpill
     @silverpill @tadano Found this so far