Re: [protobuf] Re: Python with PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=cpp

2013-06-25 Thread Feng Xiao
Yes, the package uploaded to PyPI only contains the pure python implementation. To use the cpp implementation you'll need to download the source tar from the download page and build it from source. On Mon, Jun 24, 2013 at 4:49 PM, William Rowan william.ro...@gmail.comwrote: I'm having the same

[protobuf] Byte Offsets into File with Multiple Messages

2013-06-25 Thread jonathan . wolk
Hi All, I know the topic of multiple [protobuff] messages in a single file has been covered a bunch, but I have a slightly different question. Most of the answers to multiple messages in a single file has been, use a CodedInputStream to do the reading and writing and to write the size of the

Re: [protobuf] Byte Offsets into File with Multiple Messages

2013-06-25 Thread Ilia Mirkin
You also need to keep track of how big your header is. Otherwise when reading the header, it will happily just keep on going, accumulating unknown tags/etc. When you write the file, keep track of the actual offsets you write the protos at. When reading, compare them to what you had when written

Re: [protobuf] Byte Offsets into File with Multiple Messages

2013-06-25 Thread jonathan . wolk
On Tuesday, June 25, 2013 2:55:44 PM UTC-7, Ilia Mirkin wrote: You also need to keep track of how big your header is. Otherwise when reading the header, it will happily just keep on going, accumulating unknown tags/etc. When you write the file, keep track of the actual offsets you write

Re: [protobuf] Byte Offsets into File with Multiple Messages

2013-06-25 Thread jonathan . wolk
After checking the offsets written to the header in the file and the offsets being read at (to what I seekg to), everything seems to match up 100%. The offsets for the entries aren't suspicious either (so there is actual data being written there). Maybe I'm not doing something else properly

Re: [protobuf] Byte Offsets into File with Multiple Messages

2013-06-25 Thread jonathan . wolk
After checking the offsets written to the header in the file and the offsets being read at (to what I seekg to), everything seems to match up 100%. The offsets for the entries aren't suspicious either (so there is actual data being written there). Maybe I'm not doing something else properly

Re: [protobuf] Byte Offsets into File with Multiple Messages

2013-06-25 Thread Ilia Mirkin
More code certainly couldn't hurt. Are you sure that the data is making it into the file OK? Are you opening the file with ios::binary or whatever the flag is? On Tue, Jun 25, 2013 at 6:16 PM, jonathan.w...@gree.co.jp wrote: After checking the offsets written to the header in the file and the

Re: [protobuf] Byte Offsets into File with Multiple Messages

2013-06-25 Thread jonathan . wolk
Alright, some code incoming..be warned This is code which creates the file, I'll only paste the important parts code void WriteEntriesToFile(const string path, .other params here..) { ofstream* stream = new ofstream(path.c_str(), ios::out | ios::binary); // check for

Re: [protobuf] Byte Offsets into File with Multiple Messages

2013-06-25 Thread jonathan . wolk
More code. This is the parsing code code void GetMessagesFromFile(const string path) { ifstream* stream = new ifstream(path.c_str(), ios::in | ios::binary); unsigned int endOfHeaderOffset = 0; // read_header_from_file just reads the HeaderPB and computes endOfHeaderOffset,

Re: [protobuf] Byte Offsets into File with Multiple Messages

2013-06-25 Thread jonathan . wolk
The number of entries in the header is the correct number. The print outs of the byte offsets when encoding and when parsing are the exact same. The parsing from the CodedInputStream doesn't fail. I'm at a loss as to what I can do here. For the time being, I can work around my problem by doing

[protobuf] how to describe Map and List in protobuf?

2013-06-25 Thread Minxuan Zhuang
I have a c++ program, wrapping a List type, it looks like python list: type of element is arbitrary within a list. So with the Map : both key and value are arbitrary type, how can I describe them in protobuf? -- You received this message because you are subscribed to the Google Groups