carlpayne opened a new issue, #24063:
URL: https://github.com/apache/beam/issues/24063

   ### What would you like to happen?
   
   When writing data with BigQueryIO using 
`BigQueryIO.Write.Method.STORAGE_API_AT_LEAST_ONCE` or 
`BigQueryIO.Write.Method.STORAGE_WRITE_API`, it would be very useful to be able 
to specify one or more `SchemaUpdateOption` values via 
`.withSchemaUpdateOptions`. 
   
   Currently this only works for `BigQueryIO.Write.Method.FILE_LOADS`, which 
means that for other insert methods new fields are either ignored (if 
`ignoreUnknownValues` is set), or fail to insert. In our use case, we are 
streaming Avro records from Kafka to BigQuery, with the schema constantly 
changing (mostly new fields). Having BigQueryIO manage this would be a huge 
benefit.
   
   ### Issue Priority
   
   Priority: 3
   
   ### Issue Component
   
   Component: io-java-gcp


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to