Conversation
Notices
-
erlang devs position is maps are not suitable as a portable representation of data. nostr uses arrays, but in json
reading some msgpack notes about the futility of a canonical representation are refreshing my memory. some languages enforce encoding rules on utf8 and some normalize it or reject it so you may not even be able to parse it and verify the signature at all
-
@Moon
> erlang devs position is maps are not suitable as a portable representation of data
really? why?
-
@shpuld @Moon i guess i'm just not sure what 'portable' is supposed to mean here. if that includes different representations, then i get it.
-
@lain @Moon key ordering making same logical data have multiple binary and ascii representations?
-
@shpuld @lain my interpretation is that arrays are already consistent to handle wrt to serialization and deserialization but how you serialize and deserialize a map has all kinds of assumptions that are different between languages and implementations of maps which ends up making using them with serialization fraught with risk exactly because you do not know how an application is going to use it. ways to manage that risk are using serialization formats that enforce canonical representations of data, or are low-level and you can do your specific validation at a higher level before you stick it into whatever data structure you need. all these projects end up reimplementing json because the normal json parser won't give you something that is consistent. again this is an argument against signing anything but the raw bytes but matrix and secure scuttlebutt people aren't dumb so maybe they're doing it to hedge bets or deal with problems somewhere else in the code that helps with, i don't know.
-
@lain @Moon yeah I'm just guessing too, really depends on the application
-
@Moon @shpuld the problem with signing just the bytes is that you have to keep them around forever in the exact form if you ever want to check them again (or forward them at a later point, for example for repeating).
-
@hj @lain @shpuld tons of applications it might not matter and json is fine but if it matters then should have used something else. i realize this sounds like a "everybody should change" argument, it's just an evolving realization that everything is really wrong and we have to live with it but new systems shouldn't have to.
-
@Moon @hj @shpuld json-ld signatures define a canonical json representation so you can deserialize and serialize and get the same thing. We never implemented it because the algorithm looks like a nightmare, but I didn't look into it too deeply. I think mastodon stopped doing it too, but for different reasons (i think privacy / deniability)
-
@Moon @lain @shpuld the JS part of its name might clue you in why it's like that, but really i'm not talking about json itself, more that somebody using it like that.
-
@hj @lain @shpuld json? yes
-
@Moon @lain @shpuld sounds like bad design to me
-
@hj @lain @shpuld I know this very basic computer science fact. if you're passing json and you need the map data to be ordered or there are security implications to duplicate keys then either you shouldn't have used json or you write your own json parser, and then somebody else just uses the built in one for their implementation and gets fucked.
-
@Moon @lain @shpuld arrays are ordered by definition. Maps are literally mapping keys to values, i.e. linking one thing to another thing, order isn't even in definition of the data structure. There are multiple ways you could "order" maps - order by insertion time, order by modification time, sorted by value, sorted by key, random. I mean one of the basic implementations, HashMap just hashes the key and uses that hash as an index in some internal array. SortedMap is a thing but that's specific implementation of it, much like multimaps and such. If you're passing JSON or some sort of serialized data you need to explicitly state if some maps are supposed to be ordered or not, either by documentation or by using more explicit data structures such as nested/2d arrays, i.e. [[k,v],[k,v],[k,v]]
-
@lain @shpuld understood, that's a big downside. doesn't pleroma repeat remote posts under some circumstances?
-
@Moon @shpuld when pleroma repeats / Announces things, it just announces the id as the object, not the full object itself. The other server then has to dereference it and maybe fetch it.
-
@lain @hj @shpuld one of my friends pointed out to me that matrix also has a json canonicalization, the rules look pretty simple for it, I should compare the two.
-
@teknomunk @hj @Moon @shpuld i think it's just that
-
@lain @hj @Moon @shpuld
> canonical JSON-LD
Is this something different that the canonical RDF quads used as part of RSASignature2017 that mastodon uses/used to authenticate relayed measages?
Moon
LS
御shp :blobshp:
Proud owner of a potato sack :flag:
teknomunk
All ０７６萌ＳＮＳ content and data are available under the