damccorm commented on code in PR #39160:
URL: https://github.com/apache/beam/pull/39160#discussion_r3537340898
##########
sdks/python/apache_beam/io/gcp/bigquery.py:
##########
@@ -2937,6 +2937,13 @@ class ReadFromBigQuery(PTransform):
PCollection with a schema and yielding Beam Rows via the option
`BEAM_ROW`. For more information on schemas, see
https://beam.apache.org/documentation/programming-guide/#what-is-a-schema)
+ query_output_schema: Required when output_type is 'BEAM_ROW' and a query
+ is specified. A BigQuery schema describing the query result columns,
+ since the schema cannot be auto-derived from an existing table when
+ using a query. Accepts the same formats as WriteToBigQuery's schema
+ parameter: a dict like
+ ``{'fields': [{'name': 'col', 'type': 'STRING', 'mode': 'NULLABLE'}]}``,
+ a JSON string, or a TableSchema object.
Review Comment:
We should also probably test the happy path without mocks.
##########
sdks/python/apache_beam/io/gcp/bigquery.py:
##########
@@ -2951,11 +2958,13 @@ def __init__(
use_native_datetime=False,
output_type=None,
timeout=None,
+ query_output_schema=None,
Review Comment:
If query is a kwarg, this should probably also be a kwarg
##########
sdks/python/apache_beam/io/gcp/bigquery.py:
##########
@@ -2937,6 +2937,13 @@ class ReadFromBigQuery(PTransform):
PCollection with a schema and yielding Beam Rows via the option
`BEAM_ROW`. For more information on schemas, see
https://beam.apache.org/documentation/programming-guide/#what-is-a-schema)
+ query_output_schema: Required when output_type is 'BEAM_ROW' and a query
+ is specified. A BigQuery schema describing the query result columns,
+ since the schema cannot be auto-derived from an existing table when
+ using a query. Accepts the same formats as WriteToBigQuery's schema
+ parameter: a dict like
+ ``{'fields': [{'name': 'col', 'type': 'STRING', 'mode': 'NULLABLE'}]}``,
+ a JSON string, or a TableSchema object.
Review Comment:
What happens when the schema doesn't match the actual data returned? I think
we should add a test for this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
