So my original comment on the ion package was a bit dubious about its use on microcontrollers mainly because it converts everything to text including blobs (JSON must be pure text and cannot have binary data in it). As I said probably using base64. This would make the blob (binary) data a lot bigger and would require a base64 decoder on the MCU. I don't consider this to be a wise choice, though simple formats may work fine.
On Mon, Nov 22, 2021 at 3:59 PM Jon Wolfe <[email protected]> wrote: > nanopb's readme just says it's an implementation of Protocol Buffers for > embedded use, without really describing what Protocol Buffers is. > > The readme for the main Protocol Buffers says it's "Google's > language-neutral, platform-neutral, extensible mechanism for serializing > structured data" > > In other words, a system for converting data structures to and from a > block of bytes. That description also fits Ion. > > They just seem to differ in how they do that and in some of the aspects of > the serialized data. Ion supports text or binary format for serialized > data, and it's text format is a superset of json. protobufs are primarily a > binary format, but the original google developed version does also support > json (nanopb presently does not). Protobufs require a well-defined schema > for the data structures, and there is tooling to process that schema into > code. Ion does not appear to require a schema (that's the 'self describing' > part of it). I'm guessing Ion has a rich set of API's for interrogating the > de-serialized data for structure. (Protobuf has reflection API,s but they > are populated based on code generated by the tooling from the schema > definition, not from the data object itself). > > If Amazon Ion does support embedded systems, I'd imagine it comes with a > lot of limitations on using the text formats. Parsing json on embedded > devices can be pretty expensive, memory wise. > > > On 11/22/2021 2:33 PM, Carl Nobile wrote: > > Jon, none of what you are talking about is in the GitHub README file where > it says it's buffering code. The external docs, after looking at them, > hardly mention buffering at all, so there's a definite misunderstanding if > one only reads the README. They need to put a bit more info in the README. > > With 'amazon.ion' there is no encoding or decoding as it would be in > 'nanopb', you just pull data out of the user-defined structure, maybe this > is what you mean when you say it's "self describing". JSON is just > keywords and values and/or a list of items, that's it. > JSON is an acronym for JavaScript Object Notation because this is where > the format was first created, however, it almost directly matches how the > internals of Python works also, so it can be parsed with Python extremely > easily. > > ~Carl > > > On Mon, Nov 22, 2021 at 1:56 PM Jon Wolfe <[email protected]> wrote: > >> Carl, that is not correct. I have worked with nanopb specifically with >> two micros passing data encoded with nanopb over UART and I2C. Also, I'm >> 99.9% certain that the on-the-wire format used by nanopb is compatible with >> the mainstream protocol buffers format used on desktop/servers. I've used >> the full protocol buffers libraries for both communication over sockets and >> for serializing data structures to disk on PC. >> >> nanopb does not deal directly with the transport of the data (eg UART, >> I2C, or sockets), it just can convert a data structure back and forth from >> a block of bytes. What you do with that block of bytes is up to you. It's >> my understanding that that is pretty much what the Amazon library does as >> well. >> >> The main difference from my brief reading about Ion as that it's "self >> describing" where the data contains a description of itself. Protobuf >> doesn't do that, but you share ".proto" files between both sides and the >> protoc compiler generates wrapper code in the language you're using. It has >> capability to handle backward compatibility, so you can modify the data >> structures, but both sides do need to have some basic idea about the >> structure, it's not inherent to the data stream. Proto bufs let you have a >> strongly typed contract on both ends of a communication channel, and it >> sounds like Amazon Ion lets you have that contract more loosely defined. >> >> The main Protocol Buffers project has a companion project called GRPC >> that builds on top of Protocol Buffers and is geared toward "client server" >> communications. That library does handle the transport of data as well as >> the packaging of it. It generates server and client code for you to handle >> the transport. Think of that like REST, but with a binary format and >> strongly typed contract. >> >> >> On 11/22/2021 1:37 PM, Carl Nobile via TriEmbed wrote: >> >> So the two packages mentioned in this thread do not do the same thing and >> cannot replace each other. >> The 'amason.ion' package is a data format structure implemented using >> JSON, whereas 'nanopb' is a buffering system specifically for >> microcontrollers. In other words, 'nanopb' CANNOT be sent over a wire >> protocol where amazon.ion can be. >> Interestingly they can be used together where amazon.ion can be buffered >> by 'nanopb' which may help with larger 'amazon.ion' data packets. >> ~Carl >> >> >> On Mon, Nov 22, 2021 at 11:30 AM Peter Soper via TriEmbed < >> [email protected]> wrote: >> >>> Nanopb looks way cool. Thanks! >>> _______________________________________________ >>> Triangle, NC Embedded Computing mailing list >>> >>> To post message: [email protected] >>> List info: >>> http://mail.triembed.org/mailman/listinfo/triembed_triembed.org >>> TriEmbed web site: http://TriEmbed.org >>> To unsubscribe, click link and send a blank message: mailto: >>> [email protected]?subject=unsubscribe >>> >>> >> >> -- >> -------------------------------------------------------------- >> Carl J. Nobile (Software Engineer/API Design) >> [email protected] >> -------------------------------------------------------------- >> >> _______________________________________________ >> Triangle, NC Embedded Computing mailing list >> >> To post message: [email protected] >> List info: http://mail.triembed.org/mailman/listinfo/triembed_triembed.org >> TriEmbed web site: http://TriEmbed.org >> To unsubscribe, click link and send a blank message: >> mailto:[email protected]?subject=unsubscribe >> <[email protected]?subject=unsubscribe> >> >> > > -- > -------------------------------------------------------------- > Carl J. Nobile (Software Engineer/API Design) > [email protected] > -------------------------------------------------------------- > > -- -------------------------------------------------------------- Carl J. Nobile (Software Engineer/API Design) [email protected] --------------------------------------------------------------
_______________________________________________ Triangle, NC Embedded Computing mailing list To post message: [email protected] List info: http://mail.triembed.org/mailman/listinfo/triembed_triembed.org TriEmbed web site: http://TriEmbed.org To unsubscribe, click link and send a blank message: mailto:[email protected]?subject=unsubscribe
