@Moon@mint@p@silverpill W3C specs born after 1990 don't know which part of the URL is supposed to be opaque to the client and which is supposed to be opaque to the server, they only know produce ill-conceived wishlist, make everyone else charge they phone, eat bandwidth, and lie.
@p@p@Moon From what I heard, you split objects into pieces and transfer them via IPFS... Is that correct?
Making objects self-contained is one of my goals in FEP-ef61. You can use regular HTTP to deliver them and fetch them, but you can also use other transports.
@silverpill@p@Moon Initial glance, looks like it. I was going to add supplemental keys named with an "rvl" prefix, so like {"url":"...","rvlid":"..."} like I've been doing with the HTTP headers, but that was going to be slightly later. Same thing with the attachment digests.
It just occurred to me now that I should probably have been subscribed to the RSS feed for the FEP stuff; the way that stupid hardware has been (and other personal things like the dog's cancer and a few other things that have been resolved finally) I had no time for my own stuff so I slacked on following the Mitra development.
There is an alternate proposal I have where you just wrap a DID-based variant inside of a standard AP object, using the parent object as an envelope, thus being able to support both as a transitory thing. It just makes it a bit more bloated/jank in appearance.
Either way, I think there needs to be more people involved in the discussion, especially if there’s better ideas.
> Yes, but server can send different activities to different peers.
This is behavior that I wouldn't want to rely on. You end up with bad auto-negotiation and someone checks a header instead of probing for capabilities and that ends up the way `User-Agent:` ended up, plus it compels you to keep a lot of state around.
> Software that supports FEP-ef61 would recognize a resolver path and process the object accordingly.
> From what I heard, you split objects into pieces and transfer them via IPFS... Is that correct?
No. Kind of. IPFS is an alternative supplemental channel that works transparently by fortuitous coincidence: IPFS and Revolver both use SHA-256. So Revolver can kind of optimistically request blocks from IPFS and it can send them to IPFS as a fire-and-forget, but I have IPFS disabled on most of the nodes because it slows down the network. It has its own protocol for moving the slush and it can make use of side-channels, like it uses HTTP as one side-channel, so it can fetch blocks over HTTP/HTTPS/Tor and probably I2P, and eventually other stuff. The idea is that it should be easy to move data into and out of your node regardless of network-level censorship, so you set the protocols you want your node to speak and it looks for peers that advertise that they speak that protocol. IPFS is almost only accidentally a transport layer because it's a pluggable backend layer that happens to speak to a network; there's no reason it couldn't support Tahoe-LAFS or something as another storage backend (venti is planned). For network propagation, at some point it might even support bittorrent at some point (looking into it is planned).
> You can use regular HTTP to deliver them and fetch them, but you can also use other transports.
I would prefer the resolver http url for all DID objects, because the behavior of a server that doesn't understands DIDs remains correct (HTTP GET to the url returns the requested object as long as that server responds). Servers that understand and support DIDs can process objects that it understands in the way prescribed by the DID standards and use the HTTP GET for all that it doesn't.
It works like venti, kind of, except that the metadata is stored in the pointer blocks. So the individual blocks where the data is stored, they are nearly unidentifiable on their own, you can safely propagate them without liability. Then the pointer blocks say things like "block $x, size $y, zlib-compressed; block $x+1, size $z, uncompressed" and they're reassembled like that. Unlike IPFS, the block size is 8kB max, so there are more collisions (which are good in this case) and it's easier to distribute individual blocks. Just empirically, it's also much faster to read data that you have locally and acquire unknown blocks, but we'll see how that holds up under heavier use. FSE's emoji and media storage use it (so media.fse is still up and running and it's got most of the uploads) and it's handled the load just fine.
> How node determines what blocks are needed?
The pointer blocks contain references to the blocks under them, so if you have one, you can tell what blocks are needed to assemble the full chunk of data. Someone performs an activity, and effectively they manufacture the blocks required, then sign the pointer block and a sequence number, and new blocks are fed to the rest of the network. The big hosted nodes can eat everything they hear about, smaller nodes only fetch blocks that are required to reassemble the streams of people that someone on the node is interested in, and individual nodes don't have to keep anything but blocks that haven't propagated. Usually, keeping them in memory is fine, but this might not be the case once it gets heavier use.
@p@arcanicanis@mint@p@Moon Thanks, I'll think about it. I should probably add a new section called "Alternatives" to FEP-ef61 where different solutions can be listed, with pros and cons.
Data-puns rarely work out how you might hope. It's better for it to be unambiguous. It may seem unlikely that the DID is accidentally generated, but consider a maliciously crafted URL, for example.
One of the things Revolver does is just put an ID in the headers so that, like how IPFS does it, if the data is large enough that it hasn't all arrived by the time you've parsed the headers, you can consult the local storage first. (For example, `curl -I https://media.freespeechextremist.com/emoji/custom/revolvertan.png`.)
> I want to limit protocol changes to IDs because this should make the implementation simpler, at least in theory.
This is a good idea, but extending it when it's an extension is probably the best. If you're sending things based on what you think the other server understands, then that's harder to debug and easier to craft maliciously. Easiest way to do it is to just give everyone what you have (or at least what they ask for) and it keeps it more flexible, so maybe someone does an experimental Pleroma that supports fetching these DIDs but no one is delivering them because they saw "Pleroma" and guessed "Doesn't understand DIDs", or Pleroma starts to fully support them but all of the old posts were delivered without DIDs.
> And we could support even more URI types in the future.
If you do something like adding `"cas":{"ipfs":"hash", "did:ap":"other-hash", "rvl":"third-hash"}`, that might enable implementations to kind of branch off. Or maybe call it "alt" and add whatever Nostr uses to the list. Then an implementation's free to scan the "alt" list and find something it understands, and fetch otherwise.
(Working right now so it is possible I have rushed and have read carelessly.)
@lanodan@p@p I was going to say this about the CF'd site too. Just block that crap. There are cloudflare blockers out there of all types and sizes, get one, save yourself, time, money, sanity.
>I use one (cloudflare blocker), it's called "Tor".
That's not exactly a CF blocker though. Sure, its protecting you from being trivially identified by hiding your IP address from CF, but they can still aggregate the data you are feeding it by using a service they control. And can probably do worse things.
I prefer to be informed that a site is CF so you can avoid using it at all. A handy add-on for Tor Browser is Block Cloudflare MITM Attack. The default settings block CF immediately, though which is a behavior-based fingerprint problem, I use the setting that turns the addon icon red. That doesn't protect from Amazon Cloudfront or Akamai though. The tool I use the most is the Antitrust Firewall.
If CloudFlare hates Tor and I don't run their challenge JS or they just don't like my UA string, then yes, it totally blocks CloudFlare. This is why, on instances where media proxy is disabled (e.g., FSE), I can't see some people's avatars/attachments.
> I prefer to be informed that a site is CF so you can avoid using it at all.
Cloudfed handles that for me.
> A handy add-on for Tor Browser
Oh, I just use Firefox/Seamonkey/mothra and send all the HTTP traffic through Tor; I don't run their browser. iptables trumps plugins anyway, then it extends to any device on the network. Come hop on my wifi, there are no ads, Facebook doesn't work.
> methinks you may be shocked by how many sites are paying extortion money to CF.
It is unlikely I'd be surprised, I get greeted by their "Access Denied" pages often enough. According to demo.fedilist.com, we're just short of 28k instances (27,911) and about 20% of those are behind Cloudfed. (I just stopped clicking boats forever. Their boats are automated now but they still don't like to let me past. Fine with me.)
> its gotta be one of the most useful things I've ever read.
I stopped somewhere in the part where they start defining things and skimmed a while. It's kind of a lengthy screed, accomplishes something of questionable utility that definitely will not work on my machine, and with a goal that this thing will not achieve. (Of course the author does not think Bytedance/Reddit/Netflix are worth including on the list; I could speculate as to why.) So I start to smell "aimless bloviating activist" and then the TODO says they plan at some point to turn it into a bash script, which kind of sealed it. They picked the wrong target, the wrong criteria, an unachievable goal of stopping someone from owning too many IP addresses. (Seriously, is the problem that some people have too many IP addresses? How many IP addresses did Cambridge Analytica have? That MBA dickhead from MIT Sloan that scraped fedi to make censorship easier, he's got like a /29 or something, almost nothing.) I think this kind of thing does more harm than good: there's actual, productive stuff people can do but the grandiose, demonstrably useless wheel-spinning actually gets in the way. Like, it's a fun hack, no need to dress it up with a manifesto that actually hurts the alleged goal. It's performance art, but frustratingly close to something real.
(If it's a friend of yours that wrote it, maybe leave off the mean parts and tell him that he can stop scraping whois and just use https://ftp.ripe.net/pub/stats/ripencc/nro-stats/latest/ and that since IPv6 is 128-bit and a lot of ISPs hand out /64s, the limit of 2^24 is exceeded by most SIM cards.)
The free tier of CF blocks Tor in most circumstances yes, but methinks you may be shocked by how many sites are paying extortion money to CF.
>iptables trumps plugins
Yes! Agree 100%! Iptables is awesome. That firewall setup I linked above uses iptables. See line 4378. Blocks CF ip addresses in AS13335, 14789, 132892, 133877, 139242, 202623, 203898, 209242, 394536, 395747.
Its not the best bash script I've read, in fact its not even a proper bash script but its gotta be one of the most useful things I've ever read. If you dislike oligarchy as much as I do then you'll appreciate the Amazon and Akamai ip address lists too. :)