[
https://issues.apache.org/jira/browse/AIRFLOW-5334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17058511#comment-17058511
]
Benedetto Vitale edited comment on AIRFLOW-5334 at 3/13/20, 8:34 AM:
---------------------------------------------------------------------
I do not and it is fine but because of that check I am forced to specify ether
autodetect or schema fields and autodetect might change the defined schema of
the existing table
was (Author: benvit92):
I do not and it is fine but because of that check I am forced to specify ether
autodetect or schema fields and autodetect might change the define schema of
the existing table
> After Airflow upgrade to 1.10.3 in the gcs to bq operator it is mandatory to
> specify either autodetect or schema paramaters
> ---------------------------------------------------------------------------------------------------------------------------
>
> Key: AIRFLOW-5334
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5334
> Project: Apache Airflow
> Issue Type: Bug
> Components: gcp
> Affects Versions: 1.10.3
> Reporter: Benedetto Vitale
> Priority: Major
>
> In PR [https://github.com/apache/airflow/pull/3880] the following check was
> introduced when adding support for autodetect feature in the GCS To BQ
> operator:
> {code:java}
> if schema_fields is None and not autodetect:
> raise ValueError( 'You must either pass a schema or autodetect=True.')
> {code}
> But this behaviour is supported at the moment by google apis themselves, so
> it indeed added a functionality this PR but also removed one implicitly, was
> this expected ?
> In our case we have big query schemas already defined with Google Deployment
> Manager and we do not need to pass either of those fields, is this a
> behaviour you guys consider a bug or is there something wrong with defining
> the schema outside Airflow itself according to you ? At the moment the 1.10.3
> upgrade breaks many of our workflows because of that check (a similar check
> was added also to the Big Query hook)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
