[ https://issues.apache.org/jira/browse/AIRFLOW-7117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17074258#comment-17074258 ]
ASF subversion and git services commented on AIRFLOW-7117: ---------------------------------------------------------- Commit 7ef75d2393f30d155de550e6d1ee8c055e2abfee in airflow's branch refs/heads/master from Howie Wang [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=7ef75d2 ] [AIRFLOW-7117] Honor self.schema in sql_to_gcs as schema to upload (#8049) > Honor self.schema in sql_to_gcs as schema to upload > --------------------------------------------------- > > Key: AIRFLOW-7117 > URL: https://issues.apache.org/jira/browse/AIRFLOW-7117 > Project: Apache Airflow > Issue Type: Improvement > Components: operators > Affects Versions: 1.10.5, 1.10.6, 1.10.7, 1.10.8, 1.10.9 > Reporter: Hongyi Wang > Assignee: Hongyi Wang > Priority: Minor > > In > [sql_to_gcs|https://github.com/apache/airflow/blob/master/airflow/providers/google/cloud/operators/sql_to_gcs.py], > there is a > [self.schema|https://github.com/apache/airflow/blob/master/airflow/providers/google/cloud/operators/sql_to_gcs.py#L61-L65], > which was introduced in > [pr-3091|https://github.com/apache/airflow/pull/3091/files] to allow user to > provide their own schema while loading from sql to Google BigQuery. > This is very useful when user want custom type mapping. For example, > [mysql.date is default to map > bigquery.TIMESTAMP|https://github.com/apache/airflow/blob/master/airflow/providers/google/cloud/operators/mysql_to_gcs.py#L49], > but sometimes, people want it to be bigquery.DATE. > However, during > [mysql_to_gcs|https://github.com/apache/airflow/blob/master/airflow/providers/google/cloud/operators/mysql_to_gcs.py] > re-organization, we lost that ability. (Only [use self.schema for > pre-processing|https://github.com/apache/airflow/blob/master/airflow/providers/google/cloud/operators/sql_to_gcs.py#L229-L232], > while data type is no longer configurable). > As a conclusion, we need to honor self.schema (if provided) to override > default schema, like we did before. -- This message was sent by Atlassian Jira (v8.3.4#803005)