@calonso sorry I've been traveling internationally a lot recently (on vacation now) and haven't stayed on top of this.
I have some concerns about adding this capability given that there will be a new BigQuery connector that doesn't use these inserts at all, and code that relies on this functionality simply will not function with the new connector. However as long as the capability is guarded behind Experimental (meaning that the functionality will change, and no compatibility guarantees are made), I think it's ok to add this. My biggest concern with the current PR is changing the types of PCollections (e.g. from String -> BigQueryWriteResult). Several runners (Dataflow, Flink) support updating streaming pipelines. However if one of these PCollections has changed types, the update must fail. Is there any way to structure this without changing intermediate types? [ Full content available at: https://github.com/apache/beam/pull/6055 ] This message was relayed via gitbox.apache.org for devnull@infra.apache.org