OK, that makes sense. Thanks for the quick reply.

I work at a seismic earthquake data center. We're looking at using
protocol buffers as a means of internally moving around processed
chunks of data. Seems to work pretty well, as long as the chunks
aren't too large
(which is a problem one way or another). But working with ~5 million
data points
doesn't seem to be any problem.

"some other container format.".... Not exactly sure what that would
look like.

Thanks again.

On Oct 22, 3:26 pm, "Kenton Varda" <[EMAIL PROTECTED]> wrote:
> The "Partial" serialize and parse routines actually do something completely
> unrelated:  they allow the message to be missing required fields.  So, that
> doesn't help you.
> I'm afraid protocol buffers are not designed for storing very large
> collections in a single message.  Instead, you should be thinking about
> representing each item in a collection using a protocol message, but then
> using some other container format.  Using protocol buffers here makes the
> container format simpler because it only needs to deal with raw strings
> rather than having to worry about structured data.
> On Wed, Oct 22, 2008 at 10:19 AM, <[EMAIL PROTECTED]> wrote:
> > Is there a general strategy/methodology for dealing with very large
> > collections so that they do not need to be
> > completely held in memory before serializing and de-serializing? I
> > see, for example, SerializePartialToOstream()
> > and ParsePartialFromIstream() but no documentation of how to use it.
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 

Reply via email to