The functionality does not come from BigQueryIO itself but it just exposes existing BigQuery feature CreateDisposition [1]. I don't think BigQuery offers a way to automatically create datasets when writing. Is it possible to create such Datasets from a ParDo in your pipeline that precedes BigQueryIO write transform ?
Thanks, Cham [1] https://cloud.google.com/bigquery/docs/reference/auditlogs/rest/Shared.Types/CreateDisposition On Wed, Dec 2, 2020 at 3:20 PM Vasu Gupta <[email protected]> wrote: > Hey folks, why isn't there any capability of creating datasets > automatically just like tables in BigQueryIO? Actually, we at our company > have dynamic dataset architecture which means as the packet arrives, we > need to create new datasets and tables on the go. Since BigQueryIO already > have functionality of creating tables automatically so we were thinking > that why not a similar functionality for dataset can be implemented in > BigQueryIO. >
