bugraoz93 commented on issue #26884: URL: https://github.com/apache/airflow/issues/26884#issuecomment-1327961674
Hey @bhankit1410, I think this feature has already been supported. I have tested on `BigQueryToBigQueryOperator`. I created a simple DAG. You can see the DAG as follows. I generated sample 1000 records with multiple types of column types such as integer, UUID and timestamp. I have inserted data into a table. I have tested with every three Write Dispositions. Every case I have tested has been working for me so far. https://gist.github.com/bugraoz93/d3ee6d2d03d1881de4614d1e7c3b8234 > import datetime > > from airflow import DAG > from airflow.providers.google.cloud.transfers.bigquery_to_bigquery import BigQueryToBigQueryOperator > > with DAG( > dag_id="test_dag", > max_active_runs=1, > start_date=datetime.datetime(2022, 11, 24), > schedule_interval="@once", > catchup=False, > concurrency=1, > ) as dag: > project = "test_project" > dataset = "bugraoz93_test" > source_table = "test_table$20221125" > destination_table = "test_table_dest$20221124" > > copy_table_to_fact = BigQueryToBigQueryOperator( > task_id="copy_test", > gcp_conn_id="gcp_conn", > source_project_dataset_tables="{project}.{dataset}.{table}".format( > project=project, dataset=dataset, table=source_table), > destination_project_dataset_table="{project}.{dataset}.{table}".format( > project=project, dataset=dataset, table=destination_table > ), > write_disposition="WRITE_APPEND", > create_disposition="CREATE_IF_NEEDED", > dag=dag, > ) > I have also checked the code. I have tested the individual methods to ensure that they can process the `$` sign within the table name without any exceptions. There is no part within the code that prevents this feature to work. Could you please expand your case a little bit? Which Apache Airflow version are you using to achieve it? Which provider version are you using for google (apache-airflow-providers-google)? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
