Status: New
Owner: liuj...@google.com
Labels: Type-Defect Priority-Medium
New issue 454 by johannes...@gmail.com: Python API parallel read access is
not thread safe when repeated fields are empty
http://code.google.com/p/protobuf/issues/detail?id=454
We have found out that the current implementation of the python message API
is not completely thread-safe for read-only access. Attached is a test case
that reproduces the error reliably.
What steps will reproduce the problem?
1. Extract the attached test program
2. Generate the protocol classes using 'protoc --python_out=. test.proto'
3. Run the test program prototest.py
What is the expected output? What do you see instead?
Expected: Program should run endlessly without errors
What happens after short time is the following error:
Traceback (most recent call last):
File "prototest.py", line 46, in <module>
str(protoObject)
File "/usr/lib/python2.7/site-packages/google/protobuf/internal/python_message.py",
line 660, in __str__
return text_format.MessageToString(self)
File "/usr/lib/python2.7/site-packages/google/protobuf/text_format.py",
line 58, in MessageToString
PrintMessage(message, out, as_utf8=as_utf8, as_one_line=as_one_line)
File "/usr/lib/python2.7/site-packages/google/protobuf/text_format.py",
line 67, in PrintMessage
for field, value in message.ListFields():
File "/usr/lib/python2.7/site-packages/google/protobuf/internal/python_message.py",
line 554, in ListFields
all_fields = [item for item in self._fields.iteritems() if
_IsPresent(item)]
RuntimeError: dictionary changed size during iteration
What version of the product are you using? On what operating system?
python 2.7, protobuf 2.4.1, Linux 64 bit
Please provide any additional information below.
What happens in detail is the following:
The main thread (the one generating the new instances of Test in the test
case) issues a call to the __str__ method of the generated message class.
This method internally iterates over a dictionary of present fields in the
message (google/protobuf/internal/python_message.py", line 554, in
ListFields). This dictionary changes during the iteration if another thread
tries to access a repeated field, which is empty in the respective message
instance. The current implementation of the access routine changes the
field dictionary using the python method setdefault() (the respective call
can be found in _AddPropertiesForRepeatedField, python_message.py at line
398). What is necessary to reproduce this bug is therefore a potentially
long running iteration in ListFields, which we have created in the test
case using many fields with long values to print. Additionally, someone
needs to (read! not write) access a repeated field without contents.
A simple solution to this could be to create a copy of the dictionary in
ListFields for the iteration, e.g. using items() instead of iteritems().
Attachments:
pythontest.tar.gz 648 bytes
--
You received this message because you are subscribed to the Google Groups "Protocol
Buffers" group.
To post to this group, send email to protobuf@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
