Repository: beam Updated Branches: refs/heads/master 06809b982 -> a2ff162c9
Updates WriteToBigQuery PTransform to get project id from GoogleCloudOptions when using DirectRunner. WriteToBigQuery PTransform behaves differently for DirectRunner and DataflowRunner when it comes to determining the project that the output table belongs to. If a project is not specified, DataflowRunner defauls to GoogleCloudOptions.project while DirectRunner does not. This PR fixes this inconsistency by defaulting to GoogleCloudOptions.project for DirectRunner as well. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/ae810034 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/ae810034 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/ae810034 Branch: refs/heads/master Commit: ae8100348f43e50d27a500d4196fa08137ad81b8 Parents: 06809b9 Author: chamik...@google.com <chamik...@google.com> Authored: Fri Sep 22 17:59:50 2017 -0700 Committer: chamik...@google.com <chamik...@google.com> Committed: Mon Sep 25 11:10:22 2017 -0700 ---------------------------------------------------------------------- sdks/python/apache_beam/io/gcp/bigquery.py | 3 +++ 1 file changed, 3 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/beam/blob/ae810034/sdks/python/apache_beam/io/gcp/bigquery.py ---------------------------------------------------------------------- diff --git a/sdks/python/apache_beam/io/gcp/bigquery.py b/sdks/python/apache_beam/io/gcp/bigquery.py index 40611ac..ee79ae5 100644 --- a/sdks/python/apache_beam/io/gcp/bigquery.py +++ b/sdks/python/apache_beam/io/gcp/bigquery.py @@ -1412,6 +1412,9 @@ bigquery_v2_messages.TableSchema): raise TypeError('Unexpected schema argument: %s.' % schema) def expand(self, pcoll): + if self.table_reference.projectId is None: + self.table_reference.projectId = pcoll.pipeline.options.view_as( + GoogleCloudOptions).project bigquery_write_fn = BigQueryWriteFn( table_id=self.table_reference.tableId, dataset_id=self.table_reference.datasetId,
