[ https://issues.apache.org/jira/browse/BEAM-5788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Liu resolved BEAM-5788. ---------------------------- Resolution: Fixed Fix Version/s: 2.10.0 > wordcount_fnapi_it failed on TestDataflowRunner because of JSON string > decoding error > ------------------------------------------------------------------------------------- > > Key: BEAM-5788 > URL: https://issues.apache.org/jira/browse/BEAM-5788 > Project: Beam > Issue Type: Sub-task > Components: test-failures > Reporter: Mark Liu > Assignee: Mark Liu > Priority: Major > Fix For: 2.10.0 > > Time Spent: 4h 50m > Remaining Estimate: 0h > > Similar to BEAM-5785, wordcount_fnapi_it failed on Python 3 when running with > TestDataflowRunner. Got TypeError: the JSON object must be str, not 'bytes'. > This error cause infinite retry before job could submitted to service. > More details about my env and test: > Python version: 3.5.3 > Test: > apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_fnapi_it > Command: > {code} > python setup.py nosetests \ > --tests > apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_fnapi_it \ > --nocapture \ > --nologcapture \ > --test-pipeline-options=" \ > --runner=TestDataflowRunner \ > --project=<my_project> \ > --staging_location=<my_staging> \ > --temp_location=<my_temp> \ > --output=<my_output> \ > > > --sdk_location=.../beam/sdks/python/dist/apache-beam-2.9.0.dev0.tar.gz \ > --num_workers=1" > {code} > Stacktrace: > {code} > WARNING:root:Retry with exponential backoff: waiting for 7.661876827680761 > seconds before retrying exists because we caught exception: TypeError: the > JSON object must be str, not 'bytes' > Traceback for above exception (most recent call last): > File ".../beam/sdks/python/apache_beam/utils/retry.py", line 184, in wrapper > return fun(*args, **kwargs) > File ".../beam/sdks/python/apache_beam/io/gcp/gcsio.py", line 375, in exists > self.client.objects.Get(request) # metadata > File > ".../beam/sdks/python/apache_beam/io/gcp/internal/clients/storage/storage_v1_client.py", > line 955, in Get > download=download) > File > ".../tmp/virtualenvs/py3-env/lib/python3.5/site-packages/apitools/base/py/base_api.py", > line 722, in _RunMethod > return self.ProcessHttpResponse(method_config, http_response, request) > File > ".../tmp/virtualenvs/py3-env/lib/python3.5/site-packages/apitools/base/py/base_api.py", > line 728, in ProcessHttpResponse > self.__ProcessHttpResponse(method_config, http_response, request)) > File > ".../tmp/virtualenvs/py3-env/lib/python3.5/site-packages/apitools/base/py/base_api.py", > line 611, in __ProcessHttpResponse > response_type, http_response.content) > File > ".../tmp/virtualenvs/py3-env/lib/python3.5/site-packages/apitools/base/py/base_api.py", > line 442, in DeserializeMessage > message = encoding.JsonToMessage(response_type, data) > File > ".../tmp/virtualenvs/py3-env/lib/python3.5/site-packages/apitools/base/py/encoding.py", > line 104, in JsonToMessage > return _ProtoJsonApiTools.Get().decode_message(message_type, message) > File > ".../tmp/virtualenvs/py3-env/lib/python3.5/site-packages/apitools/base/py/encoding.py", > line 290, in decode_message > message_type, result) > File > ".../tmp/virtualenvs/py3-env/lib/python3.5/site-packages/apitools/base/protorpclite/protojson.py", > line 210, in decode_message > dictionary = json.loads(encoded_message) > File "/usr/lib/python3.5/json/__init__.py", line 312, in loads > s.__class__.__name__)) > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)