We currently have a lot of jobs that write to BQ using avro (using withAvroFormatFunction), and would like to start experimenting with the streaming write API.
I see two options for how to do this: - Implement support for avro -> protobuf like TableRow and beam Row do - Add something like withProtoFormatFunction to go from T -> DynamicMessage The second option seems more efficient since it'd avoid an intermediate hop through avro (or TableRow or beam Row), but option 1 would mean we can use our code as-is with the new streaming write API. Are there any plans for implementing anything like this already in the works? As a related aside, it's becoming INCREDIBLY complicated to figure out which combination of settings is supported for each write method, I usually need to read through the code to figure it out. Has there been any thought given to improving the UX here? Maybe a builder pattern where each write method has its own sub-builder? Something like: BigQueryIO.write().withFileLoads().withAvroFormatFunction() ?
