[
https://issues.apache.org/jira/browse/AIRFLOW-2772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612438#comment-16612438
]
Iuliia Volkova commented on AIRFLOW-2772:
-----------------------------------------
[~kaxilnaik] , [~fenglu] , please write your opinion/decision about this task.
I can pick up it if needed.
Asserts already was removed, now it is the raise of ValueError:
https://github.com/apache/incubator-airflow/blob/master/airflow/contrib/hooks/bigquery_hook.py#L1666
And I see mails about this problem
(https://groups.google.com/forum/#!topic/cloud-composer-discuss/r7jPZnCKLnQ).
Maybe remove such validation and error raise? It seems like it's wrong in this
case.
> BigQuery hook does not allow specifying both the partition field name and
> table name at the same time
> -----------------------------------------------------------------------------------------------------
>
> Key: AIRFLOW-2772
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2772
> Project: Apache Airflow
> Issue Type: Bug
> Components: hooks
> Reporter: Berislav Lopac
> Priority: Minor
>
> When creating a load job for a single partition in a BigQuery's partitioned
> table, it is possible to specify either the table name with the partition
> (e.g. {{dataset_name.table_name$partition_id}}), or the field used for the
> partition (e.g. {{time_partitioning=\{"field": "field_name"\}}}) -- but not
> both.
> This is the code that raises the exception, at the very end of
> {{contrib/hooks/bigquery_hook.py}}:
> {code}
> assert not time_partitioning_in.get('field'), (
> "Cannot specify field partition and partition name "
> "(dataset.table$partition) at the same time"
> )
> {code}
> My first problem is using {{assert}} for flow control, but more importantly
> it is not clear what is the rationale for this check and the error if both
> are defined? The code works well if we provide just the partition field
> specification, but passing only the partition table name results in the
> following BQ error:
> {code}Incompatible table partitioning specification. Expects partitioning
> specification interval(type:day,field:local_event_start_date), but input
> partitioning specification is interval(type:day){code}
> which implies that sending both should be perfectly fine.
> Can anyone provide any insight?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
