mik-laj commented on a change in pull request #6227: [AIRFLOW-XXX] Example for
BigQuery to BigQuery operator
URL: https://github.com/apache/airflow/pull/6227#discussion_r330139083
##########
File path: airflow/example_dags/example_bigquery_to_bigquery.py
##########
@@ -0,0 +1,34 @@
+import airflow
+from airflow.models import DAG
+from airflow.operators import bash_operator
+from airflow.contrib.operators.bigquery_to_bigquery import
BigQueryToBigQueryOperator
+
+args = {
+ 'owner': 'Airflow',
+ 'start_date': airflow.utils.dates.days_ago(2),
+}
+
+dag = DAG(
+ dag_id='example_bigquery_to_bigquery',
+ default_args=args,
+ schedule_interval="@daily",
+)
+
+# [START howto_operator_bq_to_bq]
+copy_table = BigQueryToBigQueryOperator(
+ task_id='bq_to_bq_example',
+ source_project_dataset_tables='bigquery-public-data.baseball.schedules',
+ destination_project_dataset_table='bigquery-public-data.test.schedules',
+ write_disposition='WRITE_EMPTY',
+ create_disposition='CREATE_IF_NEEDED',
+ bigquery_conn_id='bigquery_default',
+ encryption_configuration = {
+ "kmsKeyName":
"projects/testp/locations/us/keyRings/test-kr/cryptoKeys/test-key"
+ },
+ dag=dag)
+# [END howto_operator_bq_to_bq]
+
+delete_test_dataset = bash_operator.BashOperator(
+ task_id='delete_airflow_test_dataset',
+ bash_command='bq rm -f -t bigquery-public-data.test.schedules',
Review comment:
Is there no suitable operator that can replace this command? This is not a
good solution, because this command fail if you do not have ADC configured.
Airflow also supports connection configuration via service accounts from a file
and a dictionary
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
