Re: [protobuf] Any j2me implementations that support extensions?

2010-01-27 Thread Jason Hsueh
It appears the ones listed at http://code.google.com/p/protobuf/wiki/ThirdPartyAddOns do not, though I've only taken a cursory glance at their pages. Anyone on the mailing list holding out on us? :-) On Wed, Jan 27, 2010 at 2:11 AM, Juan Amiguet jamig...@gmail.com wrote: Hi All, Does any one

Re: [protobuf] analysing field sizes

2010-01-27 Thread Jason Hsueh
(Since you said getserializedsize I'm assuming Java) There isn't a precanned way to get a breakdown of the serialized sizes, but I don't think you need to reimplement the entire parser: if you take a serialized message, you could set up a CodedInputStream and use a fairly simple loop of

[protobuf] Re: Issue 57 in protobuf: null values should be treated as no value

2010-01-27 Thread protobuf
Comment #38 on issue 57 by christophedehlinger: null values should be treated as no value http://code.google.com/p/protobuf/issues/detail?id=57 Kenton, I'm in the process of migrating from JSON to Protobuf, and I must say I was very surprised and disappointed when I came across this

[protobuf] Re: Issue 57 in protobuf: null values should be treated as no value

2010-01-27 Thread protobuf
Comment #39 on issue 57 by christophedehlinger: null values should be treated as no value http://code.google.com/p/protobuf/issues/detail?id=57 D'oh ! The last block of code should read: ProtoPerson.Builder builder = ProtoPerson.Builder.newBuilder(); String eFirstName =

[protobuf] Re: Issue 57 in protobuf: null values should be treated as no value

2010-01-27 Thread protobuf
Updates: Owner: ken...@google.com Comment #40 on issue 57 by ken...@google.com: null values should be treated as no value http://code.google.com/p/protobuf/issues/detail?id=57 OK, look: The vast majority of the people I talk to about this agree that the current behavior is

Re: [protobuf] Re: How can I reset a FileInputStream?

2010-01-27 Thread Kenton Varda
Your new results seem to indicate that the extra malloc/free can cost between 300ns and 1500ns. At the low end, the overhead is negligible and not worth complicating the interface. At the high end, your results seem hard to believe, unless your memory allocator is extremely poor. And even if

[protobuf] Arbitrary corruption of repeated fields

2010-01-27 Thread Stefan
Hello everybody, I have a small dilemma with regards to protocol buffers. I read the documentation but I still do not see a clear answer (I only use the Java version of protocol buffers). I hope I am not missing something really obvious here ... I have the following setup: message Item {

Re: [protobuf] Arbitrary corruption of repeated fields

2010-01-27 Thread Kenton Varda
Sorry, it is a non-goal of protocol buffers to provide message integrity -- this is left to a higher layer. One byte of corruption in a protocol message can very easily make it impossible to parse the remainder of the message, or even make the rest of the message appear as parseable garbage.

Re: [protobuf] Arbitrary corruption of repeated fields

2010-01-27 Thread Michael Poole
Stefan writes: What could I do reduce the risk of losing the entire list due to arbitrary corruption? What if corruption only occurs at the end of the file, would it be simpler to recover all the elements up to the corruption point? If you serialize the elements inside the Bag to the disk

Re: [protobuf] Re: Arbitrary corruption of repeated fields

2010-01-27 Thread Madhav Ancha
Stefan, Under what circumstances does persistence occur? If the message does not break into smaller ones naturally and it makes sense to keep the message in one piece, you can also use some checksum algorithm to verify your persistence. -Madhav Ancha On Wed, Jan 27, 2010 at 11:57 PM, Stefan

Re: [protobuf] Arbitrary corruption of repeated fields

2010-01-27 Thread Kenton Varda
On Wed, Jan 27, 2010 at 9:05 PM, Michael Poole mdpo...@troilus.org wrote: If you serialize the elements inside the Bag to the disk individually, you could prefix them with a synchronizing marker and length. A marker would typically be a fixed-length pattern that is unlikely to appear in

Re: [protobuf] Re: Arbitrary corruption of repeated fields

2010-01-27 Thread Kenton Varda
On Wed, Jan 27, 2010 at 9:57 PM, Stefan sneg...@gmail.com wrote: Next step would be to discard corrupted messages from the file (a corrupt message is one that does not parse) This is not a good way to detect corruption. It is very easy for a corrupt message to still parse correctly, and then