We don't want to parse BigQuery SQL in Beam, as having our own parser would take a lot of effort to maintain. Instead, the preferred way to discover tables in a query is to run it in dry-run mode and look at the list of referenced tables, as mentioned in the bug: https://issues.apache.org/jira/browse/BEAM-1909?focusedCommentId=16020138&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16020138
Dry run option: https://github.com/apache/beam/blob/e278e57077d8ab6459a9833eccd2ef20dc2faeae/sdks/python/apache_beam/io/gcp/bigquery.py#L806 REST API reference for referencedTables: https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs#statistics.query.referencedTables [ Full content available at: https://github.com/apache/beam/pull/5435 ] This message was relayed via gitbox.apache.org for [email protected]
