Re: Deserializing Messages of unknown type at compile-time

Even if the message contains only one, non-repeated field, ParseFrom*() will keep reading until EOF or an error. At Google, we have lots of various container formats, for streaming, record-based files, database tables, etc., where each record is a protocol buffer. All of these formats store the si

Re: Deserializing Messages of unknown type at compile-time

Kenton, > No, it won't work. Protocol buffers are not self-delimiting. They assume > that the input you provide is supposed to be one complete message, not a > message possibly followed by other stuff. There are a couple of related threads about delimiting the outer message (with either a mar

Re: Deserializing Messages of unknown type at compile-time

Hi Chris, Once I learned that Messages are not self-delimiting (thanks, Kenton!), I started working with Hadoop's source to stop the trailing bits from being included in the InputStream. I've since fixed this issue, kind of at least ;). Perhaps a good general solution is to allow a user to put a

Re: Deserializing Messages of unknown type at compile-time

Hi Alex, Kenton Varda wrote: > On Mon, Sep 8, 2008 at 9:11 PM, Alex Loddengaard > <[EMAIL PROTECTED] > wrote: > > I have a follow-up question: > > Will using > /messageInstance.newBuilderForType().mergeFrom(input).build();/ > work for a stream that conta

Re: Deserializing Messages of unknown type at compile-time

Thanks for your feedback, Kenton! You've answered all of my questions. Alex On Wed, Sep 10, 2008 at 1:00 AM, Kenton Varda <[EMAIL PROTECTED]> wrote: > On Mon, Sep 8, 2008 at 10:52 PM, Alex Loddengaard < > [EMAIL PROTECTED]> wrote: > >> I should revise my problem slightly. I had said that I am

Re: Deserializing Messages of unknown type at compile-time

On Mon, Sep 8, 2008 at 10:52 PM, Alex Loddengaard <[EMAIL PROTECTED] > wrote: > I should revise my problem slightly. I had said that I am given an > instance of a Message class when deserializing. This is true, though > sometimes that instance is null. In the cases when it's null, I'm not able

Re: Deserializing Messages of unknown type at compile-time

On Mon, Sep 8, 2008 at 9:11 PM, Alex Loddengaard <[EMAIL PROTECTED]>wrote: > I have a follow-up question: > > Will using > *messageInstance.newBuilderForType().mergeFrom(input).build();*work for a > stream that contains trailing binary information? > No, it won't work. Protocol buffers are not

Re: Deserializing Messages of unknown type at compile-time

On more follow-up (sorry for all these follow-ups): I should revise my problem slightly. I had said that I am given an instance of a Message class when deserializing. This is true, though sometimes that instance is null. In the cases when it's null, I'm not able to call newBuilderForType() on i

Re: Deserializing Messages of unknown type at compile-time

I have a follow-up question: Will using *messageInstance.newBuilderForType().mergeFrom(input).build();*work for a stream that contains trailing binary information? I'm asking this question for the following reason: I'm using a very simple example where my Message just contains a single String. W

Re: Deserializing Messages of unknown type at compile-time

After taking my code out of Hadoop, it looks as though my deserializing mechanism is working fine. My problem lies with my integration with Hadoop. Thanks for resolving this issue, Kenton! Alex On Tue, Sep 9, 2008 at 9:42 AM, Alex Loddengaard <[EMAIL PROTECTED]>wrote: > On Tue, Sep 9, 2008 at

Re: Deserializing Messages of unknown type at compile-time

On Tue, Sep 9, 2008 at 9:28 AM, Kenton Varda <[EMAIL PROTECTED]> wrote: > To clarify: In my original message I was saying that you should call > isInitialized() on the builder returned by mergeFrom(), to make sure the > parsed message is complete, before you call build(). > Ah. Now isInitialize

Re: Deserializing Messages of unknown type at compile-time

On Mon, Sep 8, 2008 at 6:27 PM, Kenton Varda <[EMAIL PROTECTED]> wrote: > > > On Mon, Sep 8, 2008 at 6:18 PM, Alex Loddengaard < > [EMAIL PROTECTED]> wrote: > >> On Tue, Sep 9, 2008 at 1:16 AM, Kenton Varda <[EMAIL PROTECTED]> wrote: >> >>> That won't work. DynamicMessage is a different class; it

Re: Deserializing Messages of unknown type at compile-time

On Mon, Sep 8, 2008 at 6:18 PM, Alex Loddengaard <[EMAIL PROTECTED]>wrote: > On Tue, Sep 9, 2008 at 1:16 AM, Kenton Varda <[EMAIL PROTECTED]> wrote: > >> That won't work. DynamicMessage is a different class; it does not know >> how to instantiate the protocol-compiler-generated version of the cla

Re: Deserializing Messages of unknown type at compile-time

On Tue, Sep 9, 2008 at 1:16 AM, Kenton Varda <[EMAIL PROTECTED]> wrote: > That won't work. DynamicMessage is a different class; it does not know how > to instantiate the protocol-compiler-generated version of the class. > Instead, you should do: > > Message result = > messageInstance.newBu

Re: Deserializing Messages of unknown type at compile-time

On Mon, Sep 8, 2008 at 4:33 AM, <[EMAIL PROTECTED]> wrote: > More tricky. Given a stream and an instance, I'm trying to get the > Descriptor by calling Message#getDescriptorForType() on the instance > and passing the return value, along with an input stream, to > DynamicMessage#parseFrom(Descript

Deserializing Messages of unknown type at compile-time

I have a scenario where I'm trying to create a Serializer and Deserializer class that can handle any general Message, given a stream (InputStream or OutputStream) and an instance of a particular Message implementation. How can I use this information to serialize and deserialize? I will break thi