Re: [protobuf] Again about delimiting multiple messages in streams

2013-02-02 Thread Oliver Jowett
On Sat, Feb 2, 2013 at 12:02 PM, Roman roman.sima...@gmail.com wrote:

 Hello!

 I read a lot about delimiting messages by VarInt32 number before each
 serialized message. It requires some additional coding. In the same time I
 guess that parsing code of GPB must understand when it must stop
 deserialization. Maybe someone who is close to sources can help me. I
 assume two conditions for that:
 1) End of stream
 2) End of message
 3) Other?

 Option 1) we do not consider since our stream must receive multiple
 messages like socket stream.


Well, unfortunately, it is effectively option (1) that is actually used.
There is nothing in the encoding itself that tells you where the message
ends. This is why you need a separate message delimiting mechanism.

Option 2) well can be detected. GPB encode data in order of TAG numbers. So
 if we add the last field in message (say)
 required bool eom = MAX_POSSIBLE_TAG_NUMBER
 a message parser must stop parsing after reading this field since no
 fields more expected.


Fields don't have to be encoded in any particular order, so you can have a
valid message encoding where the encoding of 'eom' is followed by other
fields.


 Opinions?


What you are describing is not how the current encoding works. I'm not sure
whether you are asking whether this would work with the current encoding
(it won't) or whether you are suggesting changes to the encoding (which I
suspect will be a very uphill battle - why change what already works and
break all the current users of the code? Remember there are multiple
implementations of the protocol, too..)

Oliver

-- 
You received this message because you are subscribed to the Google Groups 
Protocol Buffers group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at http://groups.google.com/group/protobuf?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: [protobuf] Again about delimiting multiple messages in streams

2013-02-02 Thread Roman Simakov
Thanks,

2013/2/2 Oliver Jowett oliver.jow...@gmail.com:
 Fields don't have to be encoded in any particular order, so you can have a
 valid message encoding where the encoding of 'eom' is followed by other
 fields.

I.e. I cannot be sure eom is the last field?

 I'm not sure
 whether you are asking whether this would work with the current encoding (it
 won't)

Exactly this one. I see. Thanks. I will go with standard practice and
delimit by VarInt32 before message data.


--
Roman Simakov

-- 
You received this message because you are subscribed to the Google Groups 
Protocol Buffers group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at http://groups.google.com/group/protobuf?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.