Hello! Once again I ask for the wisdom of the Beam community. :) I want to know if it's possible to dynamically update BQ schemas with BigQueryIO.Write.Method.STORAGE_WRITE_API, and if so, how.
I know that we can do this with FILE_LOADS enum and .withSchemaUpdateOptions. With these settings, If a record is written with 2 fields in BQ, and immediately after, a new record arrives with 3 fields and 3 fields schema, the new field will be added for the new record, and a null value with be written for the previous record. Is this possible with STORAGE_WRITE_API? Attending this link, I understand it should, right? Thanks! Link: https://cloud.google.com/bigquery/docs/write-api#:~:text=Schema%20update%20detection.%20If%20the%20underlying%20table%20schema%20changes%20while%20the%20client%20is%20streaming%2C%20then%20the%20Storage%20Write%20API%20notifies%20the%20client.%20The%20client%20can%20decide%20whether%20to%20reconnect%20using%20the%20updated%20schema%2C%20or%20continue%20to%20write%20to%20the%20existing%20connection . *Juan Calvo Ferrándiz* Data Engineer
