Just in case it's helpful, here is the Hadoop JIRA that tracks the
progress of integrating Protocol Buffers into Hadoop:



On Sep 11, 4:29 pm, Chris <[EMAIL PROTECTED]> wrote:
> I would like to note that at this other thread in this same mailing list:
> http://groups.google.com/group/protobuf/browse_thread/thread/19ab6bbb...
> This is about Alex integrating with Hadoop and:> Now, when I see the stream 
> coming in on the deserialization side, I get
> > "<binary>my_string<binary>"  The leading binary is the same as the
> > original,
> > however the trailing binary is something new entirely.
> Where Kenton replies:> No, it won't work.  Protocol buffers are not 
> self-delimiting.  They assume
> > that the input you provide is supposed to be one complete message, not a
> > message possibly followed by other stuff.
> > You will need to somehow communicate the size of the message and make sure
> > to limit the input to that size.
> And so we have a customer for a delimited message API to use in a mixed
> protocol binary stream.
> I have just posted a message in that thread pointing at this thread.
> It looks like (Length + Message) on the wire would work.
> I would also like to note that there is another (probably silly) way to
> delimit a message: a trailing byte of value 0 to 7. The 0-7, as a wire
> tag, decodes to 0 as a field number and 0-7 as the wire encoding.  A
> field number of 0 is disallowed by the ".proto" specification.  Thus the
> 0-7 cannot be for the next field and could be used as punctuation after
> the message by a new API.
> I still prefer Tag+Length+Message or Length+Message.  But there have
> been long threads here with those that think precomputing the Length is
> expensive and/or want a streaming write capability.  These people might
> want a punctuation delimited API.
> Cheers,
>   Chris
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 

Reply via email to