[
https://issues.apache.org/jira/browse/AIRFLOW-3541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ash Berlin-Taylor updated AIRFLOW-3541:
---------------------------------------
Fix Version/s: (was: 2.0.0)
> Include --useAvroLogicalTypes setting for the Airflow operator
> GoogleCloudStorageToBigQueryOperator
> ---------------------------------------------------------------------------------------------------
>
> Key: AIRFLOW-3541
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3541
> Project: Apache Airflow
> Issue Type: New Feature
> Components: operators
> Environment: Composer
> Reporter: Monica Parpi
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.10.3
>
>
> Getting an error [1] when loading data from GCS into BigQuery using the
> GoogleCloudStorageToBigQueryOperator for a Avro schema containing this column:
> {
> "name": "date_requested",
> "type": "int",
> "logicalType": "date",
> }
> This column contains the number of days since the epoch as indicated here [2].
> It is possible to load this data into a DATE column by using the
> "--useAvroLogicalTypes" parameter when initiating a load using the CLI [3]
> It would be very useful if this setting could be enabled via the Airflow
> operator that performs the equivalent data load [4].
> Thank you!
> [ [1]
> https://issuetracker.google.com/35905894|https://issuetracker.google.com/35905894]
>
> [[2]
> http://avro.apache.org/docs/current/spec.html#Date|http://avro.apache.org/docs/current/spec.html#Date]
> [3]
> [https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs#configuration.load.useAvroLogicalTypes]
> [4]
> [https://airflow.apache.org/_modules/airflow/contrib/operators/gcs_to_bq.html#GoogleCloudStorageToBigQueryOperator]
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
