[protobuf] Using GZIP compression with ofstream and ifstream

2017-07-06 Thread sin-chia . vincent-tan
Hi guys, I am having problem in serialization with GZIP and ofstream and ifstream. I suspect the problem might be due to misuse of API due to unfamiliarity with protobuf because i cant seems to find example working with ofstream/fstream with GZIP. The problem happens when trying to serialize obj

[protobuf] Protobuf ParseFromZeroCopyStream incurs high memory usage with repeated field and then hung

2017-07-27 Thread sin-chia . vincent-tan
I have encountered a problem of high memory usage when using ParseFromZeroCopyStream to load file in which a large buffer is written. Besides, the code snippet below uses 60Gb++ of RAM but failed as the system froze after reaching its RAM limit. FYI, I am using protobuf as DLL. syntax = "proto

Re: [protobuf] Protobuf ParseFromZeroCopyStream incurs high memory usage with repeated field and then hung

2017-08-04 Thread sin-chia . vincent-tan
Hi Adam, Sorry for my late reply, I agreed that such large data should be broken up into smaller pieces as i have also encountered error of "Exceeded maximum protobuf size of 2GB" which could only be solved by breaking things up. Thanks a lot for your reply Vincent On Friday, July 28, 2017 at