Re: [protobuf] Total bytes limit on coded input stream in C++?

2010-08-04 Thread Evan Jones

On Aug 3, 2010, at 16:44 , Julian González wrote:
I used the approach you mentioned and it worked. I just have a  
problem I am writing 10,000 little messages in a file, first I write  
the size of the message and then the message as it follows:


codedOutput->WriteVarint32(sample.ByteSize());
res = sample.SerializeToCodedStream(codedOutput);

The problem is that when I try to read the 10,000 messages I just  
wrote I just can read 9984 messages, when I try to read the 9885 an  
error is thrown:


libprotobuf ERROR c:\active\protobuf-2.3.0\src\google\protobuf 
\message_lite.cc:1
23] Can't parse message of type "apm.Sample" because it is missing  
required fiel

ds: timestamp

what is happening? It looks like only 9886 messages were written  
into the file, why the last 16 messages were not written?


It shouldn't be happening. Since the sender checks that all required  
fields are present, this indicates that some mismatch is occurring  
between the serialization and deserialization code. Are you sure the  
data that is being sent is exactly the same as the data being  
received? Normally these errors occur because the data is being  
truncated or changed in transit somehow (eg. truncating at a null  
byte? truncating at some buffer limit?).


The other thing that could be happening is that you could be mis- 
parsing the earlier messages. To parse multiple messages from a  
stream, you need to limit the number of bytes read (eg. using  
CodedInputStream::PushLimit, or MessageLite::  
ParseFromBoundedZeroCopyStream).


Good luck,

Evan

--
Evan Jones
http://evanjones.ca/

--
You received this message because you are subscribed to the Google Groups "Protocol 
Buffers" group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en.



Re: [protobuf] Total bytes limit on coded input stream in C++?

2010-08-03 Thread Evan Jones

On Aug 3, 2010, at 12:46 , Jon Schewe wrote:

I know that I could create a new coded input stream for each message,
but this seems rather wasteful and slow compared with just resetting a
counter.


I complained about the same thing a little while ago:

http://groups.google.com/group/protobuf/browse_thread/thread/a4bc2a3788d356f6


Read that thread for details, but the summary is: patches welcomed.  
CodedInputStream is pretty lightweight though, so creating and  
destroying one per message should be pretty efficient.


Evan

--
Evan Jones
http://evanjones.ca/

--
You received this message because you are subscribed to the Google Groups "Protocol 
Buffers" group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en.