Hello, zjk, have you finally solved this problem? I encountered exactly the
same issue recently while training a model with TensorFlow 1.15.
On Tuesday, May 3, 2022 at 1:42:19 AM UTC+8 Deanna Garcia wrote:
> I think you should file this bug with tensorflow since you are only using
> protobufs through them.
>
> On Saturday, April 23, 2022 at 10:49:41 AM UTC-7 zjk wrote:
>
>> Hi, I am not a direct user of protobuf, I was training my model with
>> tensorflow using Python, while attempting to save my checkpoint model, this
>> error related to protobuf occurred:
>>
>> [libprotobuf ERROR google/protobuf/wire_format_lite.cc:577] String field '
>> tensorflow.TensorShapeProto.Dim.name' contains invalid UTF-8 data when
>> parsing a protocol buffer. Use the 'bytes' type if you intend to send raw
>> bytes.
>> Traceback (most recent call last):
>> File "train.py", line 77, in <module>
>> train(model)
>> File "train.py", line 24, in train
>> model.optimize()
>> File "/user-data/HyperBox-main/script/model/box_model.py", line 329, in
>> optimize
>> self.save_model(itr)
>> File "/user-data/HyperBox-main/script/model/box_model.py", line 140, in
>> save_model
>> self.saver.save(self.sess, filename)
>> File
>> "/opt/conda/lib/python3.8/site-packages/tensorflow/python/training/saver.py",
>>
>> line 1208, in save
>> self.export_meta_graph(
>> File
>> "/opt/conda/lib/python3.8/site-packages/tensorflow/python/training/saver.py",
>>
>> line 1254, in export_meta_graph
>> graph_def=ops.get_default_graph().as_graph_def(add_shapes=True),
>> File
>> "/opt/conda/lib/python3.8/site-packages/tensorflow/python/framework/ops.py",
>> line 3345, in as_graph_def
>> result, _ = self._as_graph_def(from_version, add_shapes)
>> File
>> "/opt/conda/lib/python3.8/site-packages/tensorflow/python/framework/ops.py",
>> line 3262, in _as_graph_def
>> graph.ParseFromString(compat.as_bytes(data))
>> google.protobuf.message.DecodeError: Error parsing message
>>
>> from the information above, we can see that:
>> 1) some so-called 'invalid UTF-8 data' was sent to protobuf;
>> 2) Use the 'bytes' type if you intend to send raw bytes.
>>
>> the function at the second to last line compat.as_bytes(data) is defined
>> as:
>> def as_bytes(bytes_or_text, encoding='utf-8'):
>> if isinstance(bytes_or_text, bytearray):
>> return bytes(bytes_or_text)
>> elif isinstance(bytes_or_text, _six.text_type):
>> ##_six.text_type = unicode
>> return bytes_or_text.encode(encoding)
>> elif isinstance(bytes_or_text, bytes):
>> return bytes_or_text
>> else:
>> raise TypeError('Expected binary or unicode string, got %r' %
>> (bytes_or_text,))
>>
>> we can see that this function makes sure that what will be passed into
>> graph.ParseFromString(which later passed to protobuf) is definitely utf-8
>> data or bytes, however, the protobuf keeps claiming the data passed to it
>> is not valid utf-8 data. Taking one step back, even if the data was not
>> valid utf-8, it is definitely bytes, which it should take in normally.
>>
>> And I printed some information before the execution of
>> graph.ParseFromString(compat.as_bytes(data)):
>>
>> pprint(type(data))
>> graph.ParseFromString(compat.as_bytes(data))
>>
>> and here's the result:
>> <class 'bytes'>
>> the data has already been bytes even before it is passed into
>> compat.as_bytes!
>>
>> I am sorry for sounding emotional, it's just that I've been stuck with
>> this problem for too long and it really gets me frustrated.
>>
>>
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/protobuf/77532789-5439-4a1b-bf21-321d4e9ac398n%40googlegroups.com.
