KyungmoKu opened a new pull request, #41470: URL: https://github.com/apache/beam/pull/41470
Closes apache/beam#22531. Closes apache/beam#20297, together with [#11700](https://github.com/apache/beam/pull/11700), which added job labels to `ReadFromBigQuery`. Python `ReadFromBigQuery` accepts BigQuery [job labels](https://cloud.google.com/bigquery/docs/adding-labels#adding_labels_to_jobs), but `WriteToBigQuery` does not. This adds the same `bigquery_job_labels` option for load, copy, and schema-update jobs created by `WriteToBigQuery`. ```python WriteToBigQuery( table="project:dataset.table", method=WriteToBigQuery.Method.FILE_LOADS, bigquery_job_labels={"environment": "production"}, ) ``` `WriteToBigQuery` supports `FILE_LOADS`, `STREAMING_INSERTS`, and `STORAGE_WRITE_API`. Only `FILE_LOADS` creates BigQuery jobs, so BigQuery does not support job labels for the other two [write methods](https://cloud.google.com/dataflow/docs/guides/write-to-bigquery#choose_a_write_method). The docstring explains this restriction, and passing nonempty `bigquery_job_labels` to either unsupported method raises `ValueError`. ## Validation From `sdks/python`, after the [Python development setup](https://github.com/apache/beam/blob/master/contributor-docs/code-change-guide.md#console-setup-1): ```sh pytest -q apache_beam/io/gcp/bigquery_test.py apache_beam/io/gcp/bigquery_file_loads_test.py -k "bigquery_job_labels or to_from_runner_api" ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
