Monica Parpi created AIRFLOW-3541: ------------------------------------- Summary: Include --useAvroLogicalTypes setting for the Airflow operator GoogleCloudStorageToBigQueryOperator Key: AIRFLOW-3541 URL: https://issues.apache.org/jira/browse/AIRFLOW-3541 Project: Apache Airflow Issue Type: New Feature Components: operators Environment: Composer Reporter: Monica Parpi
Getting an error [1] when loading data from GCS into BigQuery using the GoogleCloudStorageToBigQueryOperator for a Avro schema containing this column: { "name": "date_requested", "type": "int", "logicalType": "date", } This column contains the number of days since the epoch as indicated here [2]. It is possible to load this data into a DATE column by using the "--useAvroLogicalTypes" parameter when initiating a load using the CLI [3] It would be very useful if this setting could be enabled via the Airflow operator that performs the equivalent data load [4]. Thank you! [ [1] https://issuetracker.google.com/35905894|https://issuetracker.google.com/35905894] [[2] http://avro.apache.org/docs/current/spec.html#Date|http://avro.apache.org/docs/current/spec.html#Date] [3] [https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs#configuration.load.useAvroLogicalTypes] [4] [https://airflow.apache.org/_modules/airflow/contrib/operators/gcs_to_bq.html#GoogleCloudStorageToBigQueryOperator] -- This message was sent by Atlassian JIRA (v7.6.3#76005)