[
https://issues.apache.org/jira/browse/AIRFLOW-7117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17080064#comment-17080064
]
Hongyi Wang commented on AIRFLOW-7117:
--------------------------------------
Issue solved with [GitHub Pull Request
#8049|https://github.com/apache/airflow/pull/8049]
> Honor self.schema in sql_to_gcs as schema to upload
> ---------------------------------------------------
>
> Key: AIRFLOW-7117
> URL: https://issues.apache.org/jira/browse/AIRFLOW-7117
> Project: Apache Airflow
> Issue Type: Improvement
> Components: operators
> Affects Versions: 1.10.5, 1.10.6, 1.10.7, 1.10.8, 1.10.9
> Reporter: Hongyi Wang
> Assignee: Hongyi Wang
> Priority: Minor
>
> In
> [sql_to_gcs|https://github.com/apache/airflow/blob/master/airflow/providers/google/cloud/operators/sql_to_gcs.py],
> there is a
> [self.schema|https://github.com/apache/airflow/blob/master/airflow/providers/google/cloud/operators/sql_to_gcs.py#L61-L65],
> which was introduced in
> [pr-3091|https://github.com/apache/airflow/pull/3091/files] to allow user to
> provide their own schema while loading from sql to Google BigQuery.
> This is very useful when user want custom type mapping. For example,
> [mysql.date is default to map
> bigquery.TIMESTAMP|https://github.com/apache/airflow/blob/master/airflow/providers/google/cloud/operators/mysql_to_gcs.py#L49],
> but sometimes, people want it to be bigquery.DATE.
> However, during
> [mysql_to_gcs|https://github.com/apache/airflow/blob/master/airflow/providers/google/cloud/operators/mysql_to_gcs.py]
> re-organization, we lost that ability. (Only [use self.schema for
> pre-processing|https://github.com/apache/airflow/blob/master/airflow/providers/google/cloud/operators/sql_to_gcs.py#L229-L232],
> while data type is no longer configurable).
> As a conclusion, we need to honor self.schema (if provided) to override
> default schema, like we did before.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
