Re: [protobuf] Serialized Message Field Order
On Jul 6, 2010, at 7:33 , Srivats P wrote:
@evan: That's what I'm doing currently - serializing and writing the
bytestream for the magic, checksum and actual content messages
separately and in that order - I was just wondering if I could put
them all into one message { ... } and just serialize one message
instead of three.
Well, the checksum must be a separate "message" from the content,
since you need the content bytes in order to compute the checksum. At
least with the "official" implementation. But you could use the
content message for the "magic" bytes part.
I would just use a CodedOutputStream/CodedInputStream directly to do
this, with something like the following:
CodedOutputStream out = ...;
ByteString msgBytes = msg.toByteString();
byte[] checksum = computeChecksum(msgBytes);
out.writeRawBytes(magicBytes);
out.writeRawBytes(checksum);
out.writeBytesNoTag(msgBytes);
--
Evan Jones
http://evanjones.ca/
--
You received this message because you are subscribed to the Google Groups "Protocol
Buffers" group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Serialized Message Field Order
@evan: That's what I'm doing currently - serializing and writing the
bytestream for the magic, checksum and actual content messages
separately and in that order - I was just wondering if I could put
them all into one message { ... } and just serialize one message
instead of three.
- Srivats
On Mon, Jul 5, 2010 at 5:56 PM, Evan Jones wrote:
> On Jul 5, 2010, at 7:25 , Srivats P wrote:
>>
>> when a message is serialized its known fields should be written
>> sequentially by field number, as in the provided C++, Java, and Python
>> serialization code
>>
>>
>> Like the encoded wire format, is the above guaranteed for the C++
>> Serialization API (assuming no unknown fields)?
>
> The "official" protobuf C++, Java, and Python implementations all write the
> fields out in tag number order (lowest to highest). However, their parsers
> handle messages where the fields are out of order.
>
>
>> I'm designing a file format based on protobuf. I plan to have the
>> first field as a fixed size file type magic value and second field as
>> a fixed size checksum value, followed by other fields. While opening
>> such a file, I'd like to match the magic value and verify the
>> checksum, before parsing the whole file.
>
> From this description, it doesn't sound like the field order matters?
> Serialize the protocol buffer, compute the checksum, then write out the
> (magic bytes)(checksum)(protocol buffer bytes)?
>
> Evan
>
> --
> Evan Jones
> http://evanjones.ca/
>
>
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Serialized Message Field Order
>From the encoding spec, it is required (or at least: recommended) to handle out-of-order fields. I guess *in part* this helps with message concatenation. I also know of at least one case where I knowingly emit fields out of order (in my implementation; not related to the core Google implementation). Marc On 5 July 2010 12:25, Srivats P wrote: > Hi, > > From http://code.google.com/apis/protocolbuffers/docs/encoding.html > > when a message is serialized its known fields should be written > sequentially by field number, as in the provided C++, Java, and Python > serialization code > > > Like the encoded wire format, is the above guaranteed for the C++ > Serialization API (assuming no unknown fields)? > > Background: > I'm designing a file format based on protobuf. I plan to have the > first field as a fixed size file type magic value and second field as > a fixed size checksum value, followed by other fields. While opening > such a file, I'd like to match the magic value and verify the > checksum, before parsing the whole file. > > Regards, > Srivats > > -- > You received this message because you are subscribed to the Google Groups > "Protocol Buffers" group. > To post to this group, send email to proto...@googlegroups.com. > To unsubscribe from this group, send email to > protobuf+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/protobuf?hl=en. > > -- Regards, Marc -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Serialized Message Field Order
On Jul 5, 2010, at 7:25 , Srivats P wrote: when a message is serialized its known fields should be written sequentially by field number, as in the provided C++, Java, and Python serialization code Like the encoded wire format, is the above guaranteed for the C++ Serialization API (assuming no unknown fields)? The "official" protobuf C++, Java, and Python implementations all write the fields out in tag number order (lowest to highest). However, their parsers handle messages where the fields are out of order. I'm designing a file format based on protobuf. I plan to have the first field as a fixed size file type magic value and second field as a fixed size checksum value, followed by other fields. While opening such a file, I'd like to match the magic value and verify the checksum, before parsing the whole file. From this description, it doesn't sound like the field order matters? Serialize the protocol buffer, compute the checksum, then write out the (magic bytes)(checksum)(protocol buffer bytes)? Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
