[protobuf] Detecting end of CodedInputStream
Hi, I'm using a CodedInputStream, based on a FileInputStream in C++ to read a sequence of size delimited messages (as discussed in recent posts, I'm building a new CodedInputStream object for each new message, in order to allow parsing large number of events without reaching the total bytes limit). Before reading the size of the message and then its body, I'm calling the ExpectAtEnd() method to check whether we reached end of stream. However, it always returns false (which causes my program to report a warning about junk at end of stream...) Looking at source code, my understanding is that this method will never return true, unless we have an explicit limit pushed on the CodedInputStream. My question is then: how can I safely detect end of file? I guess I could do something like calling Next() on the underlying FileInputStream until it returns false (end of file) or a non empty buffer (and then call BackUp() to re-queue this buffer before creating the CodedInputStream), but it seems a bit overkill (and probably not the best thing from a performance point of view...) Thanks for your advices, Louis-Marie -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Detecting end of CodedInputStream
On Jun 25, 2010, at 4:40 , Louis-Marie wrote:
My question is then: how can I safely detect end of file? I guess I
could do something like calling Next() on the underlying
FileInputStream until it returns false (end of file) or a non empty
buffer (and then call BackUp() to re-queue this buffer before creating
the CodedInputStream), but it seems a bit overkill (and probably not
the best thing from a performance point of view...)
I think that detecting the end of file may depend on your underlying
input stream. I have some code that uses the built-in FileInputStream,
and I simply keep trying to read values until I get an error:
bool success = in.ReadVarint32(size);
if (!success) {
// we are probably at EOF
close();
return;
}
Then my close() method looks like:
assert(input_-GetErrno() == 0);
bool success = input_-Close();
assert(success);
This works for me.
Evan
--
Evan Jones
http://evanjones.ca/
--
You received this message because you are subscribed to the Google Groups Protocol
Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
[protobuf] Compile libprotobuf.a with a cross compiler?
Hi, is it possible to compile libprotobuf.a with a cross compiler and the rest (protoc for example) with the native compiler? -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Compile libprotobuf.a with a cross compiler?
This is covered in README.txt. On Fri, Jun 25, 2010 at 1:46 AM, sh seb...@gmail.com wrote: Hi, is it possible to compile libprotobuf.a with a cross compiler and the rest (protoc for example) with the native compiler? -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/protobuf?hl=en. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] FileDescriptor module error and Python library path
Hello everyone! I am a beginner of protocol buffers on python environment and installed recent version on two machines: - Ubuntu 10.04 (my local laptop) - CentOS (school's cluster) As I have root account in my local laptop, I installed this package into user folder with the following commands. 1. /configure --prefix=/usr 2. make install 3. (in python folder of package) python setup.py install (Also tried this command : python setup.py install --prefix=/usr and make clean) In contrast, I installed the same version into local folder on cluster machine because I didn't have a permission. 1. ./configure --prefix=(my home)/gpb 2. make install 3. (in python folder of package) python setup.py install (Also tried this command : python setup.py install --prefix=(my home)/ gpb and make clean) * I used (my home) as my absolute path getting from pwd command. From my local machine, there comes an error : 'module' object has no attribute 'FileDescriptor'. Also, in the cluster machine, an error : no module named google.protobuf When I was working on my local machine, I realized that even though I used --prefix=/usr option, the final python library was installed on /usr/local/lib/python2.6/dist-packages/google NOT /usr/lib/ pymodules/python2.6/google. Since Ubuntu 10.02 had builtin protocol buffer of version 2.2.0 and distance.py did not contain FileDescriptor attribute, this error may be caused when the python automatically loaded the module from /usr/lib/pymodules/python2.6/google NOT newly installed 2.3.0) So My questions are the following two things. (1) How to change module folder to new one in python (in both my laptop and cluster machine) (I already tried to add them in LD_LIBRARY_PATH and PYTHONPATH. But no effect!) (2) How to renew my previous 2.2.0 version to new one? (only in my laptop) Please give me an advice. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] FileDescriptor module error and Python library path
The flags you pass to configure only affect the C++ package -- configure/make don't know anything about the python subdirectory (except to include it in the dist), and setup.py doesn't know anything about the stuff outside that directory. Think of them as completely separate packages. setup.py is based on setuptools. So, the answers are probably in the setuptools docs. Also try setup.py --help if you haven't already. On Fri, Jun 25, 2010 at 4:26 PM, Kala canum...@gmail.com wrote: Hello everyone! I am a beginner of protocol buffers on python environment and installed recent version on two machines: - Ubuntu 10.04 (my local laptop) - CentOS (school's cluster) As I have root account in my local laptop, I installed this package into user folder with the following commands. 1. /configure --prefix=/usr 2. make install 3. (in python folder of package) python setup.py install (Also tried this command : python setup.py install --prefix=/usr and make clean) In contrast, I installed the same version into local folder on cluster machine because I didn't have a permission. 1. ./configure --prefix=(my home)/gpb 2. make install 3. (in python folder of package) python setup.py install (Also tried this command : python setup.py install --prefix=(my home)/ gpb and make clean) * I used (my home) as my absolute path getting from pwd command. From my local machine, there comes an error : 'module' object has no attribute 'FileDescriptor'. Also, in the cluster machine, an error : no module named google.protobuf When I was working on my local machine, I realized that even though I used --prefix=/usr option, the final python library was installed on /usr/local/lib/python2.6/dist-packages/google NOT /usr/lib/ pymodules/python2.6/google. Since Ubuntu 10.02 had builtin protocol buffer of version 2.2.0 and distance.py did not contain FileDescriptor attribute, this error may be caused when the python automatically loaded the module from /usr/lib/pymodules/python2.6/google NOT newly installed 2.3.0) So My questions are the following two things. (1) How to change module folder to new one in python (in both my laptop and cluster machine) (I already tried to add them in LD_LIBRARY_PATH and PYTHONPATH. But no effect!) (2) How to renew my previous 2.2.0 version to new one? (only in my laptop) Please give me an advice. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/protobuf?hl=en. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
