[protobuf] Been working on this problem for a fortnight, really need help
Hi, I am not a direct user of protobuf, I was training my model with
tensorflow using Python, while attempting to save my checkpoint model, this
error related to protobuf occurred:
[libprotobuf ERROR google/protobuf/wire_format_lite.cc:577] String field
'tensorflow.TensorShapeProto.Dim.name' contains invalid UTF-8 data when
parsing a protocol buffer. Use the 'bytes' type if you intend to send raw
bytes.
Traceback (most recent call last):
File "train.py", line 77, in
train(model)
File "train.py", line 24, in train
model.optimize()
File "/user-data/HyperBox-main/script/model/box_model.py", line 329, in
optimize
self.save_model(itr)
File "/user-data/HyperBox-main/script/model/box_model.py", line 140, in
save_model
self.saver.save(self.sess, filename)
File
"/opt/conda/lib/python3.8/site-packages/tensorflow/python/training/saver.py",
line 1208, in save
self.export_meta_graph(
File
"/opt/conda/lib/python3.8/site-packages/tensorflow/python/training/saver.py",
line 1254, in export_meta_graph
graph_def=ops.get_default_graph().as_graph_def(add_shapes=True),
File
"/opt/conda/lib/python3.8/site-packages/tensorflow/python/framework/ops.py",
line 3345, in as_graph_def
result, _ = self._as_graph_def(from_version, add_shapes)
File
"/opt/conda/lib/python3.8/site-packages/tensorflow/python/framework/ops.py",
line 3262, in _as_graph_def
graph.ParseFromString(compat.as_bytes(data))
google.protobuf.message.DecodeError: Error parsing message
from the information above, we can see that:
1) some so-called 'invalid UTF-8 data' was sent to protobuf;
2) Use the 'bytes' type if you intend to send raw bytes.
the function at the second to last line compat.as_bytes(data) is defined as:
def as_bytes(bytes_or_text, encoding='utf-8'):
if isinstance(bytes_or_text, bytearray):
return bytes(bytes_or_text)
elif isinstance(bytes_or_text, _six.text_type):
##_six.text_type = unicode
return bytes_or_text.encode(encoding)
elif isinstance(bytes_or_text, bytes):
return bytes_or_text
else:
raise TypeError('Expected binary or unicode string, got %r' %
(bytes_or_text,))
we can see that this function makes sure that what will be passed into
graph.ParseFromString(which later passed to protobuf) is definitely utf-8
data or bytes, however, the protobuf keeps claiming the data passed to it
is not valid utf-8 data. Taking one step back, even if the data was not
valid utf-8, it is definitely bytes, which it should take in normally.
And I printed some information before the execution of
graph.ParseFromString(compat.as_bytes(data)):
pprint(type(data))
graph.ParseFromString(compat.as_bytes(data))
and here's the result:
the data has already been bytes even before it is passed into
compat.as_bytes!
I am sorry for sounding emotional, it's just that I've been stuck with this
problem for too long and it really gets me frustrated.
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to protobuf+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/protobuf/9786c408-3487-435a-891a-cd1c644c4fcen%40googlegroups.com.
[protobuf] Been working on this problem for a fortnight, really need help
Hi, I am not a direct user of protobuf, I was training my model with
tensorflow using Python, while attempting to save my checkpoint model, this
error related to protobuf occurred:
[libprotobuf ERROR google/protobuf/wire_format_lite.cc:577] String field
'tensorflow.TensorShapeProto.Dim.name' contains invalid UTF-8 data when
parsing a protocol buffer. Use the 'bytes' type if you intend to send raw
bytes.
Traceback (most recent call last):
File "train.py", line 77, in
train(model)
File "train.py", line 24, in train
model.optimize()
File "/user-data/HyperBox-main/script/model/box_model.py", line 329, in
optimize
self.save_model(itr)
File "/user-data/HyperBox-main/script/model/box_model.py", line 140, in
save_model
self.saver.save(self.sess, filename)
File
"/opt/conda/lib/python3.8/site-packages/tensorflow/python/training/saver.py",
line 1208, in save
self.export_meta_graph(
File
"/opt/conda/lib/python3.8/site-packages/tensorflow/python/training/saver.py",
line 1254, in export_meta_graph
graph_def=ops.get_default_graph().as_graph_def(add_shapes=True),
File
"/opt/conda/lib/python3.8/site-packages/tensorflow/python/framework/ops.py",
line 3345, in as_graph_def
result, _ = self._as_graph_def(from_version, add_shapes)
File
"/opt/conda/lib/python3.8/site-packages/tensorflow/python/framework/ops.py",
line 3262, in _as_graph_def
graph.ParseFromString(compat.as_bytes(data))
google.protobuf.message.DecodeError: Error parsing message
from the information above, we can see that:
1) some so-called 'invalid UTF-8 data' was sent to protobuf;
2) Use the 'bytes' type if you intend to send raw bytes.
the function at the second to last line compat.as_bytes(data) is defined as:
def as_bytes(bytes_or_text, encoding='utf-8'):
if isinstance(bytes_or_text, bytearray):
return bytes(bytes_or_text)
elif isinstance(bytes_or_text, _six.text_type):
##_six.text_type = unicode
return bytes_or_text.encode(encoding)
elif isinstance(bytes_or_text, bytes):
return bytes_or_text
else:
raise TypeError('Expected binary or unicode string, got %r' %
(bytes_or_text,))
we can see that this function makes sure that what will be passed into
graph.ParseFromString(which later passed to protobuf) is definitely utf-8
data or bytes, however, the protobuf keeps claiming the data passed to it
is not valid utf-8 data. Taking one step back, even if the data was not
valid utf-8, it is definitely bytes, which it should take in normally.
And I printed some information before the execution of
graph.ParseFromString(compat.as_bytes(data)):
pprint(type(data))
graph.ParseFromString(compat.as_bytes(data))
and here's the result:
the data has already been bytes even before it is passed into
compat.as_bytes!
I am sorry for sounding emotional, it's just that I've been stuck with this
problem for too long and it really gets me frustrated.
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to protobuf+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/protobuf/95731d6d-f624-4953-bbfa-ea8478374c9an%40googlegroups.com.
