Re: [protobuf] DecodeError (Truncated Message)
Into the parseString you can pass the serialized omitting the last char, like ParseFromString(serialized[0:-1]). That seemed to work for me. -S On Thursday, March 28, 2013 2:27:43 AM UTC-7, Oliver wrote: > > On Thu, Mar 28, 2013 at 12:54 AM, W. David Jarvis > wrote: > >> If I compare the len() of the two strings (pre-writing/reading v. >> post-writing/reading) I tend to get two add'l chars on the reading, so I >> guess I'm confused by how this is getting truncated? Is python running into >> a newline char and interpreting it literally somehow? > > > Don't use 'print', use file.write(). Using print adds a newline (or > probably CRLF in your case). The encoded form is not self-delimiting so if > you include an extra CRLF in the bytes given to the decoder, it'll happily > try to interpret them, and then run out of data because the decoder's > interpretation of the extra bytes implies there should be more data > following them. > > I wouldn't rely on sys.stdout being binary-safe either. The encoded forms > are binary, not text - you should treat them as such - if any conversion of > linefeeds happens, it will corrupt the message. > > Oliver > > -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscr...@googlegroups.com. To post to this group, send email to protobuf@googlegroups.com. Visit this group at http://groups.google.com/group/protobuf. For more options, visit https://groups.google.com/d/optout.
Re: [protobuf] DecodeError (Truncated Message)
On Thu, Mar 28, 2013 at 12:54 AM, W. David Jarvis wrote: > If I compare the len() of the two strings (pre-writing/reading v. > post-writing/reading) I tend to get two add'l chars on the reading, so I > guess I'm confused by how this is getting truncated? Is python running into > a newline char and interpreting it literally somehow? Don't use 'print', use file.write(). Using print adds a newline (or probably CRLF in your case). The encoded form is not self-delimiting so if you include an extra CRLF in the bytes given to the decoder, it'll happily try to interpret them, and then run out of data because the decoder's interpretation of the extra bytes implies there should be more data following them. I wouldn't rely on sys.stdout being binary-safe either. The encoded forms are binary, not text - you should treat them as such - if any conversion of linefeeds happens, it will corrupt the message. Oliver -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscr...@googlegroups.com. To post to this group, send email to protobuf@googlegroups.com. Visit this group at http://groups.google.com/group/protobuf?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
[protobuf] DecodeError (Truncated Message)
Howdy all --
I'm looking to use protobuf objects as a convenient and lightweight object
framework for some cross-language stuff I'm doing between Cascading and
Hadoop Streaming, but I'm running into some low-level problems just parsing
messages when I'm writing the message out somewhere.
So, more concretely:
I've got a simple python script to generate an object for me, which looks
like this:
def main():
user = test_tools.generate_user() # creates a 'user' protobuf object
user.id = 427293847234
v1_activity = user.v1_activities.add()
v1_activity.CopyFrom(test_tools.generate_v1_activity())
test_string = user.SerializeToString()
# Just to prove that the string can be parsed at all--this part works
fine
sample_user_2 = user_pb2.User()
sample_user_2.ParseFromString(test_string)
# Now let's push the string to sys.stdout
print test_string
So I run that with python test_usergen.py > sample_user.txt, everything
works fine, the sample_user_2 object parses just fine, and I've got some
sort of string stored in sample_user.txt. The problem begins here.
Now, on the receiving end, I've got a script where I want to be able to
parse this string. Only when I do this:
user_file = open('sample_user.txt', 'rb').read()
user = User()
user.ParseFromString(user_file)
I get the following:
Traceback (most recent call last):
File "test_mta.py", line 112, in
main(sys.argv[1:])
File "test_mta.py", line 59, in main
user.ParseFromString(user_file)
File
"/Users/jarvis/.virtualenvs/ak-sci-env/lib/python2.7/site-packages/ak_user_dev/protowrapper.py",
line 146, in ParseFromString
self.proto_message.ParseFromString(string)
File
"/Users/jarvis/.virtualenvs/ak-sci-env/lib/python2.7/site-packages/google/protobuf/message.py",
line 182, in ParseFromString
self.MergeFromString(serialized)
File
"/Users/jarvis/.virtualenvs/ak-sci-env/lib/python2.7/site-packages/google/protobuf/internal/python_message.py",
line 800, in MergeFromString
raise message_mod.DecodeError('Truncated message.')
google.protobuf.message.DecodeError: Truncated message.
If I compare the len() of the two strings (pre-writing/reading v.
post-writing/reading) I tend to get two add'l chars on the reading, so I
guess I'm confused by how this is getting truncated? Is python running into
a newline char and interpreting it literally somehow?
Any help would be great...I've been banging my head against this all day :)
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at http://groups.google.com/group/protobuf?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
