KyungmoKu opened a new pull request, #41470:
URL: https://github.com/apache/beam/pull/41470

   Closes apache/beam#22531.
   Closes apache/beam#20297, together with 
[#11700](https://github.com/apache/beam/pull/11700), which added job labels to 
`ReadFromBigQuery`.
   
   Python `ReadFromBigQuery` accepts BigQuery [job 
labels](https://cloud.google.com/bigquery/docs/adding-labels#adding_labels_to_jobs),
 but `WriteToBigQuery` does not.
   This adds the same `bigquery_job_labels` option for load, copy, and 
schema-update jobs created by `WriteToBigQuery`.
   
   ```python
   WriteToBigQuery(
       table="project:dataset.table",
       method=WriteToBigQuery.Method.FILE_LOADS,
       bigquery_job_labels={"environment": "production"},
   )
   ```
   
   `WriteToBigQuery` supports `FILE_LOADS`, `STREAMING_INSERTS`, and 
`STORAGE_WRITE_API`.
   Only `FILE_LOADS` creates BigQuery jobs, so BigQuery does not support job 
labels for the other two [write 
methods](https://cloud.google.com/dataflow/docs/guides/write-to-bigquery#choose_a_write_method).
   The docstring explains this restriction, and passing nonempty 
`bigquery_job_labels` to either unsupported method raises `ValueError`.
   
   ## Validation
   
   From `sdks/python`, after the [Python development 
setup](https://github.com/apache/beam/blob/master/contributor-docs/code-change-guide.md#console-setup-1):
   
   ```sh
   pytest -q apache_beam/io/gcp/bigquery_test.py 
apache_beam/io/gcp/bigquery_file_loads_test.py -k "bigquery_job_labels or 
to_from_runner_api"
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to