Mark Liu created BEAM-5788:
------------------------------
Summary: wordcount_fnapi_it failed on TestDataflowRunner because
of JSON string decoding error
Key: BEAM-5788
URL: https://issues.apache.org/jira/browse/BEAM-5788
Project: Beam
Issue Type: Sub-task
Components: test-failures
Reporter: Mark Liu
Assignee: Valentyn Tymofieiev
Similar to BEAM-5785, wordcount_fnapi_it failed on Python 3 when running with
TestDataflowRunner. Got TypeError: the JSON object must be str, not 'bytes'.
This error cause infinite retry before job could submitted to service.
More details about my env and test:
Python version: 3.5.3
Test: apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_fnapi_it
Command:
{code}
python setup.py nosetests \
--tests
apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_fnapi_it \
--nocapture \
--nologcapture \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=<my_project> \
--staging_location=<my_staging> \
--temp_location=<my_temp> \
--output=<my_output> \
--sdk_location=.../beam/sdks/python/dist/apache-beam-2.9.0.dev0.tar.gz \
--num_workers=1"
{code}
Stacktrace:
{code}
WARNING:root:Retry with exponential backoff: waiting for 7.661876827680761
seconds before retrying exists because we caught exception: TypeError: the JSON
object must be str, not 'bytes'
Traceback for above exception (most recent call last):
File ".../beam/sdks/python/apache_beam/utils/retry.py", line 184, in wrapper
return fun(*args, **kwargs)
File ".../beam/sdks/python/apache_beam/io/gcp/gcsio.py", line 375, in exists
self.client.objects.Get(request) # metadata
File
".../beam/sdks/python/apache_beam/io/gcp/internal/clients/storage/storage_v1_client.py",
line 955, in Get
download=download)
File
".../tmp/virtualenvs/py3-env/lib/python3.5/site-packages/apitools/base/py/base_api.py",
line 722, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File
".../tmp/virtualenvs/py3-env/lib/python3.5/site-packages/apitools/base/py/base_api.py",
line 728, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File
".../tmp/virtualenvs/py3-env/lib/python3.5/site-packages/apitools/base/py/base_api.py",
line 611, in __ProcessHttpResponse
response_type, http_response.content)
File
".../tmp/virtualenvs/py3-env/lib/python3.5/site-packages/apitools/base/py/base_api.py",
line 442, in DeserializeMessage
message = encoding.JsonToMessage(response_type, data)
File
".../tmp/virtualenvs/py3-env/lib/python3.5/site-packages/apitools/base/py/encoding.py",
line 104, in JsonToMessage
return _ProtoJsonApiTools.Get().decode_message(message_type, message)
File
".../tmp/virtualenvs/py3-env/lib/python3.5/site-packages/apitools/base/py/encoding.py",
line 290, in decode_message
message_type, result)
File
".../tmp/virtualenvs/py3-env/lib/python3.5/site-packages/apitools/base/protorpclite/protojson.py",
line 210, in decode_message
dictionary = json.loads(encoded_message)
File "/usr/lib/python3.5/json/__init__.py", line 312, in loads
s.__class__.__name__))
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
