> I don't think BigQuery offers a way to automatically create datasets when
writing.
This exactly, and it also makes sense from the standpoint of the BQ
permissions model. In BigQuery, table creation requires that the service
account have privileges at the dataset level, but dataset creation
The functionality does not come from BigQueryIO itself but it just exposes
existing BigQuery feature CreateDisposition [1]. I don't think BigQuery
offers a way to automatically create datasets when writing.
Is it possible to create such Datasets from a ParDo in your pipeline that
precedes
Hey folks, why isn't there any capability of creating datasets automatically
just like tables in BigQueryIO? Actually, we at our company have dynamic
dataset architecture which means as the packet arrives, we need to create new
datasets and tables on the go. Since BigQueryIO already have
