Guillermo Rodríguez Cano updated AIRFLOW-1874:
    Comment: was deleted

(was: Hello,

while checking the operators mentioned in this Jira it turns that they don't 
support yet the `use_legacy_sql` parameter that is supported downstream (in the 
`BigQueryHook` to be precise) as per the mentioned pull request.
The changes are minimal now that there is support but that still leaves this 
Jira pending.)

> Support standard SQL in Check, ValueCheck and IntervalCheck BigQuery operators
> ------------------------------------------------------------------------------
>                 Key: AIRFLOW-1874
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1874
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: contrib, gcp, operators
>            Reporter: Guillermo Rodríguez Cano
>            Assignee: Guillermo Rodríguez Cano
>            Priority: Major
>             Fix For: 2.0.0
> BigQueryCheckOperator, BigQueryValueCheckOperator and 
> BigQueryIntervalCheckOperator do not support disabling use of default legacy 
> SQL in BigQuery.
> This is a major blocker to support correct migration to standard SQL when 
> queries are complicated. For example, a query that can be queried in legacy 
> SQL may be blocked from any subsequent view done in standard SQL that this 
> view uses as the queries are bound to either standard or legacy SQL but not a 
> mix.
> These operators inherit from base ones of the same name (without the BigQuery 
> prefix) from Airflow which may make the process more complicated as the flag 
> to use standard SQL should be enabled because the underlying BigQueryHook has 
> the corresponding parameter, use_legacy_sql, set to True, when running a 
> query. But it is not possible to pass parameters all the way to it via the 
> aforementioned operators.
> The workaround of including #standardSQL and a new line before the query 
> doesn't work either as there is mismatch. BigQuery reports the following in 
> fact: "Query text specifies use_legacy_sql:false, while API options 
> specify:true"
> A workaround for queries on views using standard SQL is to persist the result 
> of the query in a temporary table, then run the check operation and 
> thereafter delete the temporary table. 

This message was sent by Atlassian JIRA

Reply via email to