On Mon, Mar 23, 2009 at 1:38 AM, ode <fuji...@gmail.com> wrote:

>
> There is api in c++, ParseFromIstream, but is there any similar api in
> python?


No, there's no Python equivalent right now.

But, the parsed objects are bigger than the original serialized data, so if
the original serialized data can't fit in memory, then the parsed objects
definitely can't.  In general, protocol buffers are designed to encode small
to medium-sized messages, generally less than 1MB (usually much less).  If
your data is larger than that, you should split it up into multiple small
messages and devise some higher-level container format to wrap them so you
can parse one at a time.

In your case, you might try separating the messages from the payload.  That
is, remove the blk_data field from Block, and instead write all of the data
to the stream *after* the DifferUpload message.  Then on the receiving end,
you can parse the whole protocol message first and then use it to write the
data directly to the final destination as you read it.


>
>
> On Mar 23, 4:06 pm, ode <fuji...@gmail.com> wrote:
> > hi,
> >
> >    I'm going to use protocol buffers in http post data, seems
> > SerializeToString can be used to generate binary string, but what if
> > the data is very large, is all data serialize in memory?
> >
> >    The following is the proto file I defined, It is used for file
> > upload.
> >
> > /////////////////////////////////////
> > package fileupload;
> >
> > message Range {
> >   required uint64 start = 1;
> >   required uint32 len = 2;
> >
> > }
> >
> > message Block {
> >   required Range r = 1;
> >   required bytes blk_hash = 2;
> >   required bytes blk_data = 3;
> >
> > }
> >
> > message DifferUpload {
> >   repeated Block blk = 1;
> >
> > }
> >
> > /////////////////////////////////////
> >
> >    Any solutions>
> >
> >   Thanks in advance
> >
>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply via email to