Conversation

Notices

1. silverpill (silverpill@mitra.social)'s status on Tuesday, 21-Mar-2023 03:32:04 JST silverpill

   • pistolero :thispersondoesnotexist:

   @p I have FediList feature request.

   At http://demo.fedilist.com there is a "TOR?" dropdown menu containing "TOR only" and "Exclude TOR" items. Could you add a new category that represents clearnet instances connected to Tor federation? An instance belongs to that category if it has a clearnet domain name AND its peer list (api/v1/instance/peers) contains at least one .onion instance.

   This should help people on Tor network make new connections.

   • silverpill (silverpill@mitra.social)'s status on Tuesday, 21-Mar-2023 04:29:07 JST silverpill

     in reply to

     • pistolero :thispersondoesnotexist:

     @p But what about simply parsing /api/v1/instance/peers response? According to Details page, the crawler already visits this endpoint (~twice per day), so you can walk through the list and find domain names ending with .onion This might not be very accurate but having a list of such instances would be useful.
     For example, my instance is connected to Tor and its peer list contains four .onion instances: https://mitra.social/api/v1/instance/peers

   • pistolero :thispersondoesnotexist: (p@freespeechextremist.com)'s status on Tuesday, 21-Mar-2023 04:29:08 JST pistolero :thispersondoesnotexist:

     in reply to
     @silverpill
     
     > Could you add a new category that represents clearnet instances connected to Tor federation?
     
     This is actually difficult to detect! Automatically dedup/merge is going to either be slow and result in some false positives or reasonably fast and result in a more false positives. It's not intractable, but getting it right (and making sure it doesn't go off the rails) would be non-trivial.
     
     All the data in the system is a moving target, so it'd have to be fuzzed a little. Maybe I could do something like generate a fingerprint based on a hash of the instance description plus (truncated to account for wobble) ratios of users/posts, posts/peers. So then you can flag pairs by matching the fingerprint, and then compare the numbers, see if they're all within a certain range of each other, then mark as a duplicate. It's something that has to be done carefully to be done correctly.
   • pistolero :thispersondoesnotexist: (p@freespeechextremist.com)'s status on Tuesday, 21-Mar-2023 05:50:41 JST pistolero :thispersondoesnotexist:

     in reply to
     @silverpill
     
     > But what about simply parsing /api/v1/instance/peers response?
     
     Oh, it does. That's how it found the .onion domains it has: http://demo.fedilist.com/instance?q=&ip=&software=&registrations=&onion=only
     
     The only parsing it does of domain names is that it treats .onion domains specially: it doesn't try to resolve their hostnames before sending requests, it uses a different proxy, and it uses http instead of https.
     
     If there are Tor sites that are in your peers list but that don't typically appear anywhere, you could try just going to http://demo.fedilist.com/instance/$domain. Most of the things the site shows are just instances that are up, and Tor sites seem to bounce around a lot. For example:
     
     http://demo.fedilist.com/instance/recent-changes?host=rawrxd4mden7rmbobaftao3qjyxbrvj4rrooehkqxlqcsdtnnn2hndid.onion
     vs.
     http://demo.fedilist.com/instance/recent-changes?host=rawr-xd.club
   • silverpill (silverpill@mitra.social)'s status on Tuesday, 21-Mar-2023 05:51:10 JST silverpill

     in reply to

     • pistolero :thispersondoesnotexist:

     @p My Tor peers are on FediList already. What I'm thinking about is a separate bucket that will contain instances like mine. Basically, a filter:

     http://demo.fedilist.com/instance?q=&ip=&software=&registrations=&onion=combo

     >onion=combo

     This filter will show instances that have Tor peers but otherwise are normal clearnet instances.

   • pistolero :thispersondoesnotexist: (p@freespeechextremist.com)'s status on Tuesday, 21-Mar-2023 08:30:21 JST pistolero :thispersondoesnotexist:

     in reply to
     @silverpill Yeah, that's the thing, there's no good way to determine which two hosts are duplicate instances.
   • silverpill (silverpill@mitra.social)'s status on Tuesday, 21-Mar-2023 08:30:21 JST silverpill

     in reply to

     • pistolero :thispersondoesnotexist:

     @p I don't quite understand why this is needed. Is it some implementation quirk?
     The instances I'm talking about do not have .onion mirrors, so there shouldn't be any duplicates. They are Tor clients, but not Tor servers.

   • pistolero :thispersondoesnotexist: (p@freespeechextremist.com)'s status on Tuesday, 21-Mar-2023 09:04:53 JST pistolero :thispersondoesnotexist:

     in reply to
     @silverpill
     
     > They are Tor clients, but not Tor servers.
     
     Eh? I think that's why what I'm saying doesn't make sense, I don't know what you mean.
   • silverpill (silverpill@mitra.social)'s status on Tuesday, 21-Mar-2023 09:04:53 JST silverpill

     in reply to

     • pistolero :thispersondoesnotexist:

     @p A "combo" instance is a clearnet service, and it federates with other clearnet instances as usual, by sending HTTPS requests. There's no associated onion/hidden service. However, when it sees .onion actor, it sends federation requests via locally running Tor proxy.
     Tor instances still can talk back to it via exit nodes, even though there's no onion service.

     So, if I understand correctly, such "combo" instance creates only one entry in FediList database and there shouldn't be any duplicates.

   • silverpill (silverpill@mitra.social)'s status on Tuesday, 21-Mar-2023 09:11:56 JST silverpill

     in reply to

     • pistolero :thispersondoesnotexist:

     @p Here's a post that explains how to achieve this with Pleroma: https://pleromanonx86.wordpress.com/2022/10/29/tor-over-fediverse-and-how-to-proxy-it/

   • silverpill (silverpill@mitra.social)'s status on Tuesday, 21-Mar-2023 09:36:12 JST silverpill

     in reply to

     • pistolero :thispersondoesnotexist:

     @p

     >*That's* what you meant about finding .onion domains in your peers list, right?

     Yes, exactly. Thank you!

   • pistolero :thispersondoesnotexist: (p@freespeechextremist.com)'s status on Tuesday, 21-Mar-2023 09:36:13 JST pistolero :thispersondoesnotexist:

     in reply to
     @silverpill
     
     > A "combo" instance is a clearnet service, and it federates with other clearnet instances as usual, by sending HTTPS requests.
     
     Oh, okay, I think I get what you mean. Detecting instances that federate with Tor but are not themselves Tor instances. (*That's* what you meant about finding .onion domains in your peers list, right?)
     
     That actually shouldn't be that hard to implement, I think.
   • silverpill (silverpill@mitra.social)'s status on Tuesday, 21-Mar-2023 09:43:30 JST silverpill

     in reply to

     • pistolero :thispersondoesnotexist:
     • meso

     @meso @p Admins can block or limit Tor instances in the same way as clearnet instances. But Tor domains are very cheap so yes, it could become an issue. In that case we can switch to allowlist-only federation with onions.

   • meso (meso@asbestos.cafe)'s status on Tuesday, 21-Mar-2023 09:43:31 JST meso

     in reply to

     • pistolero :thispersondoesnotexist:
     @silverpill @p what about child porn wouldn't that be an issue with tor federation
   •  (mint@ryona.agency)'s status on Tuesday, 21-Mar-2023 20:12:38 JST 

     in reply to

     • pistolero :thispersondoesnotexist:
     • meso
     @silverpill @meso @p Also, if you don't pipe all remote images through mediaproxy, there's no way to tell what they contain unless you use a browser/proxy configured to open clearnet and Tor in the same profile. Which leaves enough time to react and nuke the offending instance, so it's nothing that isn't an issue already.