OK, that makes sense. Thanks for the quick reply. I work at a seismic earthquake data center. We're looking at using protocol buffers as a means of internally moving around processed chunks of data. Seems to work pretty well, as long as the chunks aren't too large (which is a problem one way or another). But working with ~5 million data points doesn't seem to be any problem.
"some other container format.".... Not exactly sure what that would look like. Thanks again. -B On Oct 22, 3:26 pm, "Kenton Varda" <[EMAIL PROTECTED]> wrote: > The "Partial" serialize and parse routines actually do something completely > unrelated: they allow the message to be missing required fields. So, that > doesn't help you. > I'm afraid protocol buffers are not designed for storing very large > collections in a single message. Instead, you should be thinking about > representing each item in a collection using a protocol message, but then > using some other container format. Using protocol buffers here makes the > container format simpler because it only needs to deal with raw strings > rather than having to worry about structured data. > > On Wed, Oct 22, 2008 at 10:19 AM, <[EMAIL PROTECTED]> wrote: > > > Is there a general strategy/methodology for dealing with very large > > collections so that they do not need to be > > completely held in memory before serializing and de-serializing? I > > see, for example, SerializePartialToOstream() > > and ParsePartialFromIstream() but no documentation of how to use it. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~----------~----~----~----~------~----~------~--~---
