@calonso sorry I've been traveling internationally a lot recently (on vacation 
now) and haven't stayed on top of this.

I have some concerns about adding this capability given that there will be a 
new BigQuery connector that doesn't use these inserts at all, and code that 
relies on this functionality simply will not function with the new connector. 
However as long as the capability is guarded behind Experimental (meaning that 
the functionality will change, and no compatibility guarantees are made), I 
think it's ok to add this.

My biggest concern with the current PR is changing the types of PCollections 
(e.g. from String -> BigQueryWriteResult). Several runners (Dataflow, Flink) 
support updating streaming pipelines. However if one of these PCollections has 
changed types, the update must fail. Is there any way to structure this without 
changing intermediate types?


[ Full content available at: https://github.com/apache/beam/pull/6055 ]
This message was relayed via gitbox.apache.org for devnull@infra.apache.org

Reply via email to