Hi, I am not a direct user of protobuf, I was training my model with
tensorflow using Python, while attempting to save my checkpoint model, this
error related to protobuf occurred:
[libprotobuf ERROR google/protobuf/wire_format_lite.cc:577] String field
'tensorflow.TensorShapeProto.Dim.name' contains invalid UTF-8 data when
parsing a protocol buffer. Use the 'bytes' type if you intend to send raw
bytes.
Traceback (most recent call last):
File "train.py", line 77, in <module>
train(model)
File "train.py", line 24, in train
model.optimize()
File "/user-data/HyperBox-main/script/model/box_model.py", line 329, in
optimize
self.save_model(itr)
File "/user-data/HyperBox-main/script/model/box_model.py", line 140, in
save_model
self.saver.save(self.sess, filename)
File
"/opt/conda/lib/python3.8/site-packages/tensorflow/python/training/saver.py",
line 1208, in save
self.export_meta_graph(
File
"/opt/conda/lib/python3.8/site-packages/tensorflow/python/training/saver.py",
line 1254, in export_meta_graph
graph_def=ops.get_default_graph().as_graph_def(add_shapes=True),
File
"/opt/conda/lib/python3.8/site-packages/tensorflow/python/framework/ops.py",
line 3345, in as_graph_def
result, _ = self._as_graph_def(from_version, add_shapes)
File
"/opt/conda/lib/python3.8/site-packages/tensorflow/python/framework/ops.py",
line 3262, in _as_graph_def
graph.ParseFromString(compat.as_bytes(data))
google.protobuf.message.DecodeError: Error parsing message
from the information above, we can see that:
1) some so-called 'invalid UTF-8 data' was sent to protobuf;
2) Use the 'bytes' type if you intend to send raw bytes.
the function at the second to last line compat.as_bytes(data) is defined as:
def as_bytes(bytes_or_text, encoding='utf-8'):
if isinstance(bytes_or_text, bytearray):
return bytes(bytes_or_text)
elif isinstance(bytes_or_text, _six.text_type):
##_six.text_type = unicode
return bytes_or_text.encode(encoding)
elif isinstance(bytes_or_text, bytes):
return bytes_or_text
else:
raise TypeError('Expected binary or unicode string, got %r' %
(bytes_or_text,))
we can see that this function makes sure that what will be passed into
graph.ParseFromString(which later passed to protobuf) is definitely utf-8
data or bytes, however, the protobuf keeps claiming the data passed to it
is not valid utf-8 data. Taking one step back, even if the data was not
valid utf-8, it is definitely bytes, which it should take in normally.
And I printed some information before the execution of
graph.ParseFromString(compat.as_bytes(data)):
pprint(type(data))
graph.ParseFromString(compat.as_bytes(data))
and here's the result:
<class 'bytes'>
the data has already been bytes even before it is passed into
compat.as_bytes!
I am sorry for sounding emotional, it's just that I've been stuck with this
problem for too long and it really gets me frustrated.
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/protobuf/95731d6d-f624-4953-bbfa-ea8478374c9an%40googlegroups.com.
