jrmccluskey commented on code in PR #39236: URL: https://github.com/apache/beam/pull/39236#discussion_r3759338936
########## sdks/python/apache_beam/io/gcp/bigquery.py: ########## @@ -197,6 +197,43 @@ def compute_table_name(row): a tuple of PCollectionViews to be passed to the schema callable (much like the `table_side_inputs` parameter). +Dynamic Schemas with Storage Write API +-------------------------------------- +When writing to dynamic destinations with `method=STORAGE_WRITE_API`, a union schema +containing all fields across destination tables is required at the PCollection level Review Comment: Each destination table only implements its own schema, it does not have to implement the full union schema. The union schema is (unfortunately) necessary in this setup because the storage write API calls are cross-language and a coder has to be defined to send elements to the Java SDK that actually does the writes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
