Re: [protobuf] Re: Protocol buffers and large data sets

2010-05-27 Thread Terri Kamm
that can be serialized. You can certainly use protocol buffers in large data sets, but it's not recommended to have your entire data set be represented by a single message. Instead, see if you can break it up into smaller messages. On Mon, May 17, 2010 at 1:05 PM, sanikumbh saniku

[protobuf] Re: Protocol buffers and large data sets

2010-05-24 Thread Terri
on the size of data that can be serialized. You can certainly use protocol buffers in large data sets, but it's not recommended to have your entire data set be represented by a single message. Instead, see if you can break it up into smaller messages. On Mon, May 17, 2010 at 1:05 PM, sanikumbh

Re: [protobuf] Re: Protocol buffers and large data sets

2010-05-24 Thread Kenton Varda
if you really need to parse larger messages, but it is generally not recommended. Additionally, ByteSize() returns a 32-bit integer, so there's an implicit limit on the size of data that can be serialized. You can certainly use protocol buffers in large data sets, but it's not recommended

[protobuf] Protocol buffers and large data sets

2010-05-17 Thread sanikumbh
I wanted to get some opinion on large data sets and protocol buffers. Protocol Buffer project page by google says that for data 1 megabytes, one should consider something different but they don’t mention what would happen if one crosses this limit. Are there any known failure modes when it comes

Re: [protobuf] Protocol buffers and large data sets

2010-05-17 Thread Jason Hsueh
an implicit limit on the size of data that can be serialized. You can certainly use protocol buffers in large data sets, but it's not recommended to have your entire data set be represented by a single message. Instead, see if you can break it up into smaller messages. On Mon, May 17, 2010 at 1

Large data sets

2009-10-06 Thread Brenden Matthews
Hi, In the documentation here: http://code.google.com/apis/protocolbuffers/docs/techniques.html#large-data it specifies that if you are dealing in messages larger than a megabyte each, it may be time to consider an alternate strategy. My question is: does this apply to messages which are

Re: Large data sets

2009-10-06 Thread Kenton Varda
On Tue, Oct 6, 2009 at 9:34 AM, Brenden Matthews bren...@diddyinc.comwrote: it specifies that if you are dealing in messages larger than a megabyte each, it may be time to consider an alternate strategy. My question is: does this apply to messages which are large because they themselves