On Wed, Apr 15, 2015 at 9:56 PM, Gianluca Borello <[email protected]> wrote:
> Hello, > > At my current company we're heavy protobuf users, but we started > facing a big limitation: in the current Java implementation, since > there's no lazy parsing, or ability to do direct sub-message parsing, > we pay a huge penalty of time and, especially, memory (we have an > articulated schema and the memory footprint is often 5X+ the > serialized size) even when we just want to read a small subset of > fields of a big protobuf (~10-100 KB). I just checked the code and it seems lazy field for Java is already included: The protoc compiler code: https://github.com/google/protobuf/blob/master/src/google/protobuf/compiler/java/java_lazy_message_field.h Java runtime support: https://github.com/google/protobuf/blob/master/java/src/main/java/com/google/protobuf/LazyField.java And tests: https://github.com/google/protobuf/blob/master/java/src/test/java/com/google/protobuf/LazyMessageLiteTest.java It's either already working for Java, or only requires small tweaks for it to work. Could you check if marking a field lazy makes any difference in Java? If not please help file an issue on our github site. > > I understand that, according to [1] and [2], such feature has not yet > been released and it will come at some point, but nonetheless I'd like > to ping again to see if there's a more defined ETA (of course I > realize this is an open source project so support and roadmap are > shared at a best-effort). > If you are looking for lazy field support in C++, I think it mostly likely will become a part of 3.0 official release (along with other possible performance tweaks for the release). > > I am aware that there are projects (like flatbuffers) that might > better suited for our use case, but we would really like to stick to > protobuf because we have a huge code base already working on that, and > we consider it the most stable and robust solution available. > > Thanks. > > [1] > https://groups.google.com/forum/#!searchin/protobuf/lazy|sort:date/protobuf/2P4ge8MfYKw/dcsmXDPaE84J > [2] > https://groups.google.com/forum/#!searchin/protobuf/lazy|sort:date/protobuf/G9Jhv_xrxqg/tYcXzyy6zV8J > > -- > You received this message because you are subscribed to the Google Groups > "Protocol Buffers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/protobuf. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/protobuf. For more options, visit https://groups.google.com/d/optout.
