Hello, We use Protocol Buffers for a portion of our settings on one of our products, in particular those that the device rewrites.
One issue we encountered was that, as we are in an embedded setting with limited RAM, writing to flash memory with 256 byte pages, we did not want structures to overlap page boundaries. Doing so would complicate writing and require more buffering. So, we extended the binary protocol by making Wire Type 7 (Field Number 0) -- effectively just the byte 0x07 -- padding. All the parser needs to do is ignore it and skip to the next byte. Having done this, once we recognize that our next part of the message will overlap a page boundary, we pad with 0x07 to that boundary, write the page to flash memory, reset our write pointer, and continue writing. As you can see, with this small extension, it's now possible to trivially stream Protocol Buffers data *while maintaining alignment*. I think in a PC context, this would also have some uses, because for a large binary field for a picture or audio, written into a field as binary, the writer could optionally pad to an 8 or 16 byte boundary, and then memcpy out of the stream would be drastically faster. Or similarly, because the data is already aligned, a game could use the contained art resources directly from the Protocol Buffer memory, without any copy at all. (As an aside, one possibility would be to include a pad length in the Field Number, to tell the parser how many bytes to skip. We did not do this, because it invites a writer to not write those bytes at all and leak arbitrary data into the stream. Also, most padding is very short, so the extra complication when reading would not likely gain any speed.) Please consider this. It's been very useful to us, and I think it would be useful to everyone else too. Thank you! God bless. James Bellinger Dimension Engineering -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion visit https://groups.google.com/d/msgid/protobuf/6db5d4a0-bce8-42e9-8396-f354a5276d58n%40googlegroups.com.
