[protobuf] Re: Issue 429 in protobuf: Support ByteBuffer in CodedInputStream and CodedOutputStream to avoid redundant copies for direct ByteBuffer.
Updates: Summary: Support ByteBuffer in CodedInputStream and CodedOutputStream to avoid redundant copies for direct ByteBuffer. Status: New Labels: -Type-Defect -Priority-Medium Type-Enhancement Priority-Low Comment #2 on issue 429 by xiaof...@google.com: Support ByteBuffer in CodedInputStream and CodedOutputStream to avoid redundant copies for direct ByteBuffer. http://code.google.com/p/protobuf/issues/detail?id=429 Reopened this issue and changed the title to be more specific. The interface of messages and builders is already fixed and shouldn't be changed easily. Adding ByteBuffer support in CodedInputStream and CodedOutputStream should be good enough to support direct ByteBuffers efficiently. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscr...@googlegroups.com. To post to this group, send email to protobuf@googlegroups.com. Visit this group at http://groups.google.com/group/protobuf?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
[protobuf] Again about delimiting multiple messages in streams
Hello! I read a lot about delimiting messages by VarInt32 number before each serialized message. It requires some additional coding. In the same time I guess that parsing code of GPB must understand when it must stop deserialization. Maybe someone who is close to sources can help me. I assume two conditions for that: 1) End of stream 2) End of message 3) Other? Option 1) we do not consider since our stream must receive multiple messages like socket stream. Option 2) well can be detected. GPB encode data in order of TAG numbers. So if we add the last field in message (say) required bool eom = MAX_POSSIBLE_TAG_NUMBER a message parser must stop parsing after reading this field since no fields more expected. I have several message types and union them into one. It looks like this: message M1 { required int32 ri32 = 1; optional bool b = 2; } message M2 { optional string s = 1; optional int64 i64 = 2; } message ServerMessage { enum Type { M1 = 1; M2 = 2; } optional M1 m1 = 1; optional M2 m2 = 2; required eom bool = 1000; } So can I do the following: // On sender CodedOutputStream os; ServerMessage m1; m1.set_type(ServerType::M1); m1.set_eom(true); ... m1.SerializeToCodedStream(os); ServerMessage m2; m2.set_type(ServerType::M2); m2.set_eom(true); ... m2.SerializeToCodedStream(os); // On receiver CodedInputStream os; ServerMessage m1; m1.ParseFromCodedStream(os); ServerMessage m2; m2.ParseFromCodedStream(os); Opinions? -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscr...@googlegroups.com. To post to this group, send email to protobuf@googlegroups.com. Visit this group at http://groups.google.com/group/protobuf?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [protobuf] Again about delimiting multiple messages in streams
On Sat, Feb 2, 2013 at 12:02 PM, Roman roman.sima...@gmail.com wrote: Hello! I read a lot about delimiting messages by VarInt32 number before each serialized message. It requires some additional coding. In the same time I guess that parsing code of GPB must understand when it must stop deserialization. Maybe someone who is close to sources can help me. I assume two conditions for that: 1) End of stream 2) End of message 3) Other? Option 1) we do not consider since our stream must receive multiple messages like socket stream. Well, unfortunately, it is effectively option (1) that is actually used. There is nothing in the encoding itself that tells you where the message ends. This is why you need a separate message delimiting mechanism. Option 2) well can be detected. GPB encode data in order of TAG numbers. So if we add the last field in message (say) required bool eom = MAX_POSSIBLE_TAG_NUMBER a message parser must stop parsing after reading this field since no fields more expected. Fields don't have to be encoded in any particular order, so you can have a valid message encoding where the encoding of 'eom' is followed by other fields. Opinions? What you are describing is not how the current encoding works. I'm not sure whether you are asking whether this would work with the current encoding (it won't) or whether you are suggesting changes to the encoding (which I suspect will be a very uphill battle - why change what already works and break all the current users of the code? Remember there are multiple implementations of the protocol, too..) Oliver -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscr...@googlegroups.com. To post to this group, send email to protobuf@googlegroups.com. Visit this group at http://groups.google.com/group/protobuf?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [protobuf] Again about delimiting multiple messages in streams
Thanks, 2013/2/2 Oliver Jowett oliver.jow...@gmail.com: Fields don't have to be encoded in any particular order, so you can have a valid message encoding where the encoding of 'eom' is followed by other fields. I.e. I cannot be sure eom is the last field? I'm not sure whether you are asking whether this would work with the current encoding (it won't) Exactly this one. I see. Thanks. I will go with standard practice and delimit by VarInt32 before message data. -- Roman Simakov -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscr...@googlegroups.com. To post to this group, send email to protobuf@googlegroups.com. Visit this group at http://groups.google.com/group/protobuf?hl=en. For more options, visit https://groups.google.com/groups/opt_out.