ruodingt opened a new issue, #23746: URL: https://github.com/apache/beam/issues/23746
### What would you like to happen? Hi, Beam community, I very much like the feature WriteToBigquery offering to dynamically deciding the `table`: https://github.com/apache/beam/blob/6adecd438790d8c1b5182043db16232b68ff7a98/sdks/python/apache_beam/io/gcp/bigquery.py#L1788 I am thinking about a potential improvement opportunity. Currently `WriteToBigquery` supports either dict or TableRow, and it assumes that table reference can be inferred from the content of dict or TableRow to be written into BQ. We probably should remove that assumption. This is case I am expecting WriteToBigquery to support: Given a PCollection of `K, V` pair of <str, dict>, I hope `K` can be the used to infer table name yet get `V` written into the dictionary. (or even do some further processing on V) I can see there is a few way to do that. The simplest way I can think of is to allow `table` arg to be an instance of `DoFn` or `AppendDestinationsFn`. https://github.com/apache/beam/blob/b5a0f4895ccf7bd2f8b03217d866e8d0c67f6f52/sdks/python/apache_beam/io/gcp/bigquery_tools.py#L1488 If the sdk allow us to overwrite `AppendDestinationsFn` and pass it into `WriteToBigquery` we will have more freedom to customise the class. I am happy to contribute and make this feature come true. But would be keen to get communities' feedback first:) ### Issue Priority Priority: 2 ### Issue Component Component: io-py-gcp -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
