> Next time it happens, I will have a look at the DB.
Okay, it is two rows in the DB. Activity AdVKpA5RgC3jKDypGa occupied rows 9680118 and 9680119 in the DB, both with my user_id and type "mention". So it's definitely a backend problem. It coincided with a burst of traffic. (The traffic was Applebot and archive.org, and apparently some dipshits are hitting an RSS feed on the blog every 30 seconds so I should probably trim it instead of just putting everything in there.)
> Worth noting that Pleroma also sends out notifications to user's websocket transitiently,
If I've seen it happen, then it can't be that, because I'm using bloat. Now that I think about it, for all I know, though, there could be an off-by-one error in the paging of notifications: muted threads always create moderate flakiness. I haven't seen this in the ssh client, which ignores thread-muting.
Next time it happens, I will have a look at the DB.
@colonelj@Terry Something weird going on with bae.st, started a few weeks ago but there were some weird incidents before that with them sending double posts.
It's been hitting both the public inbox endpoint and the user-specific inboxes. I don't know why it's doing that, but it's even doing it for users that no longer exist.
I don't think it's that, though, I think it's probably that it takes us longer to do the insert than bae.st's local timeout. (cc @sjw ) Delivering a post is several steps¹, most of which have overlapping timers on both ends, so something can get delivered but might not be viewed as a success on the other end and they try to deliver it again.
There should probably just be a uniqueness constraint on (user_id,type,activity_id) for notifications. That would slow things down (explanation in the footnote), though, and it's already slow.
¹ Abridged and simplified:
1. Worker receives job to deliver post to FSE. Baest DB connection pool checkout timer starts. 2. Worker does DNS lookup. This is fast usually but should be locally cached because a lot of them happen and it is holding a DB connection from the pool. 3. Worker eventually sends the signed request to FSE to deliver the post. Web request timer starts. 4. FSE receives the post by HTTP. nginx and Pleroma both start incoming request timers. 5. FSE starts processing the post. The post gets verified, goes through the MRF chain, etc. FSE's DB connection pool checkout timer starts. 6. FSE completes processing, including inserting the notification into the notifications table, which is huge. 7. FSE finishes processing the request. FSE's DB connection pool timer stops. 8. FSE sends the request back out through nginx. Pleroma and nginx request timers stop. 9. Baest receives the response. Baest's web request timer stops. 10. Baest worker processes the request, noting that it is a success, and recording that to the DB. Baest DB connection pool checkout timer stops. 11. The job queue records the job's success and the job stops getting retried.
If any of the timers expires after step 6 is over but before step 11 is over, double deliveries could be attempted and, although the post would not get recorded twice, double notifications might be created. A number of these steps are slower than they might be, like step 6: FSE has received 9,679,872 notifications so far; several purges have occurred, it currently holds 3,385,727 rows, but is frequently and repeatedly accessed by reads/writes and is indexed all to hell (which makes writes to the table more expensive). Like Poast, Baest's outgoing requests go through proxies and some of these proxies (e.g., M247) have been responsible for spam accounts so FSE doesn't treat all IP ranges the same. (There are a lot of weird things going on, because bae.st and FSE are both old servers that have accumulated quirks and scar tissue, plus both servers are kind of, like, outliers in terms of use.)
Typically, the way you solve this kind of thing is a two-phase commit, but ActivityPub doesn't have a provision for that kind of thing. It's just "toss the activity at the other server and let's all hope for the best". Granted, the network is not designed to be bulletproof and it's difficult to design a decentralized protocol that *is* bulletproof, and even if it's designed well, it is more complicated to implement that sort of thing than the naïve version, so a two-phase commit might have seemed like overkill. (Something tells me that The Mastadan Netwark's German dictator was closer to "didn't know what a two-phase commit is" than "prudently decided not to complicate the protocol". In any case, if you want lots of implementations and broad adoption, it's better not to require more complicated implementations than you have to.)
So you end up having to deduplicate on both ends. (Content-addressed storage makes deduplication trivial. :revolvertan:)
> I am not going to lie and pretend I know what duganism is Pete I would have to actually look it up and I will (swear I will lol)
I think he qualifies as "moderately obscure" in the Anglosphere so I wouldn't worry about that. Fringe internet politics here; he is a somewhat bigger deal in Russia. I think this is maybe the third time I have ever heard someone mention him. I had to look him up first time. Everyone's gotta look up everything.
Wikipedia's not exactly kind to the guy: https://en.wikipedia.org/wiki/Aleksandr_Dugin . He seems like Ayn Rand but if she hated American capitalism instead of the USSR. I don't know how popular he is beyond being a punchline (like Ayn Rand) and he has said some stuff that's unobjectionable to anybody (like Ayn Rand), some stuff that is appealing mainly to people on his side of the fence (like Ayn Rand), and then some stuff that nobody wants to stand by (like Ayn Rand). It's just that he hates globalism and capitalism and atheism.
> do you happen to know whether it is possible at all to do IP bridging on KVM/QEMU with a WiFi adapter?
Yes. Depends on how you want to do it, but yes. You can put everything on a bridge interface, or you can do a virtual bridge and then there's effectively a simulated network and you can have the host machine do the routing. You can also do both:
I think that wifi adapters have trouble being a bridge interface, so the virtual bridge might be the easiest way to do it.
This is due to MACs being used for associating with APs. Bridge interfaces usually require the interface to support "promiscuous mode" and (I think) most wifi drivers do that but getting the AP to recognize the fake MACs might be a problem. On a regular wired network, you don't (typically!)
(Typically, because you actually can do this. Fyodor's chapter in "Stealing the Network" talked about this. I don't have a copy, unfortunately.)
BOFH of freespeechextremist.com, and former admin. The usual alt if FSE is down: @p@shitposter.club, and others. I am no longer the admin. FSE has no admins now. Welcome to the FSE Autonomous Zone.I'm not angry with you, I'm just disappointed.I am physically in Los Angeles but I exist in a permanent state of 3 a.m.I have dropped a bytebeat album, feel free to DM me for a download code or a link to a tarball: https://finitecell.bandcamp.com/album/villain . There is a chiptunes album there, too.Revolver is coming: https://blog.freespeechextremist.com/blog/revolver-kickoff.html