Akshar Dave created AIRFLOW-3114:
------------------------------------
Summary: Add feature to create External BigQuery Table for
querying Cloud Bigtable data
Key: AIRFLOW-3114
URL: https://issues.apache.org/jira/browse/AIRFLOW-3114
Project: Apache Airflow
Issue Type: Improvement
Components: contrib, gcp, hooks
Affects Versions: 1.10.0, 2.0.0
Reporter: Akshar Dave
Fix For: 2.0.0, 1.10.1
Currently using the BigQuery operator and the bq hook we can create an external
BQ table but there is no way to create a permanent external table on Cloud
Bigtable data.
* For
[externalDataConfiguration.sourceUris|https://cloud.google.com/bigquery/docs/reference/v2/tables#externalDataConfiguration.sourceUris],
you can specify exactly one [Cloud Bigtable
URI|https://cloud.google.com/bigquery/external-data-bigtable#bigtable-uri], and
it must be a fully specified, valid HTTPS URL.
* Specify the data format properties by setting the
[externalDataConfiguration.sourceFormat|https://cloud.google.com/bigquery/docs/reference/v2/tables#externalDataConfiguration.sourceFormat]
property. For Google Cloud Bigtable, specify {{"BIGTABLE"}}.
Below are the links to Doc, Python Api and the JSON Rest API that can be
helpful:
[https://cloud.google.com/bigquery/external-data-bigtable]
[https://cloud.google.com/bigquery/docs/reference/rest/v2/tables#externalDataConfiguration.sourceFormat]
[https://cloud.google.com/bigquery/docs/reference/rest/v2/tables#externalDataConfiguration.sourceUris]
[https://github.com/GoogleCloudPlatform/google-cloud-python/blob/38080c8920d20b01f06c1eaef79eb527525de552/bigquery/google/cloud/bigquery/external_config.py]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
