[
https://issues.apache.org/jira/browse/BEAM-6532?focusedWorklogId=194177&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-194177
]
ASF GitHub Bot logged work on BEAM-6532:
----------------------------------------
Author: ASF GitHub Bot
Created on: 04/Feb/19 18:45
Start Date: 04/Feb/19 18:45
Worklog Time Spent: 10m
Work Description: tvalentyn commented on pull request #7721: [BEAM-6532]
Regenerate BQ client to match apitools==0.5.26 and fix BQ io to work on Python 3
URL: https://github.com/apache/beam/pull/7721
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/)
| --- | --- | --- | --- | --- | ---
Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
| --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
</br> [](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/)
| --- | --- | ---
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 194177)
Time Spent: 10m
Remaining Estimate: 0h
> BigQuery IO does not work in Python 3
> -------------------------------------
>
> Key: BEAM-6532
> URL: https://issues.apache.org/jira/browse/BEAM-6532
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
> Reporter: Valentyn Tymofieiev
> Assignee: Valentyn Tymofieiev
> Priority: Blocker
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Suspecting an issue in apitools library, similar to one [~markflyhigh]
> discovered for GCS in BEAM-6154.
> {noformat}
> WARNING:root:Retry with exponential backoff: waiting for 7.8972419555895
> seconds before retrying get_table_location because we caught exception:
> TypeError: the JSON object must be str, not 'b
> ytes'
>
>
> Traceback for above exception (most recent call last):
>
>
> File
> "/home/valentyn/projects/beam/beam/beam/sdks/python/apache_beam/utils/retry.py",
> line 184, in wrapper
>
> return fun(*args, **kwargs)
>
>
> File
> "/home/valentyn/projects/beam/beam/beam/sdks/python/apache_beam/io/gcp/bigquery_tools.py",
> line 421, in get_table_location
>
> table = self._get_table(project_id, dataset_id, table_id)
>
>
> File
> "/home/valentyn/projects/beam/beam/beam/sdks/python/apache_beam/utils/retry.py",
> line 197, in wrapper
>
> raise_with_traceback(exn, exn_traceback)
>
>
> File
> "/home/valentyn/tmp/venv/py3env/lib/python3.5/site-packages/future/utils/__init__.py",
> line 421, in raise_with_traceback
>
> raise exc.with_traceback(traceback)
>
>
> File
> "/home/valentyn/projects/beam/beam/beam/sdks/python/apache_beam/utils/retry.py",
> line 184, in wrapper
>
> return fun(*args, **kwargs)
>
>
> File
> "/home/valentyn/projects/beam/beam/beam/sdks/python/apache_beam/io/gcp/bigquery_tools.py",
> line 333, in _get_table
>
> response = self.client.tables.Get(request)
>
>
> File
> "/home/valentyn/projects/beam/beam/beam/sdks/python/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py",
> line 579, in Get
> config, request, global_params=global_params)
>
>
> File
> "/home/valentyn/tmp/venv/py3env/lib/python3.5/site-packages/apitools/base/py/base_api.py",
> line 731, in _RunMethod
>
> return self.ProcessHttpResponse(method_config, http_response, request)
>
>
> File
> "/home/valentyn/tmp/venv/py3env/lib/python3.5/site-packages/apitools/base/py/base_api.py",
> line 737, in ProcessHttpResponse
>
> self.__ProcessHttpResponse(method_config, http_response, request))
>
>
> File
> "/home/valentyn/tmp/venv/py3env/lib/python3.5/site-packages/apitools/base/py/base_api.py",
> line 620, in __ProcessHttpResponse
>
> return self.__client.DeserializeMessage(response_type, content)
>
>
> File
> "/home/valentyn/tmp/venv/py3env/lib/python3.5/site-packages/apitools/base/py/base_api.py",
> line 446, in DeserializeMessage
>
> message = encoding.JsonToMessage(response_type, data)
>
>
> File
> "/home/valentyn/tmp/venv/py3env/lib/python3.5/site-packages/apitools/base/py/encoding_helper.py",
> line 123, in JsonToMessage
>
> return _ProtoJsonApiTools.Get().decode_message(message_type, message)
>
>
> File
> "/home/valentyn/tmp/venv/py3env/lib/python3.5/site-packages/apitools/base/py/encoding_helper.py",
> line 309, in decode_message
>
> message_type, result)
>
>
> File
> "/home/valentyn/tmp/venv/py3env/lib/python3.5/site-packages/apitools/base/protorpclite/protojson.py",
> line 212, in decode_message
>
> dictionary = json.loads(encoded_message)
>
>
> File "/usr/lib/python3.5/json/__init__.py", line 312, in loads
>
>
> s.__class__.__name__))
>
>
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
