[
https://issues.apache.org/jira/browse/AIRFLOW-2679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bolke de Bruin updated AIRFLOW-2679:
------------------------------------
Fix Version/s: (was: 1.10)
(was: 1.10.0)
> GoogleCloudStorageToBigQueryOperator to support UPSERT
> ------------------------------------------------------
>
> Key: AIRFLOW-2679
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2679
> Project: Apache Airflow
> Issue Type: Improvement
> Reporter: jack
> Priority: Major
>
> Currently the
> {color:#222222}GoogleCloudStorageToBigQueryOp{color}{color:#222222}erator
> support incremental load using
> *{color:#404040}max_id_key{color}*{color:#404040} {color}.{color}
>
> {color:#222222}However many systems actually needs "UPSERT" in terms of - if
> row exists update it, if not insert/copy it.{color}
> {color:#222222}Currently the operator assumes that we only need to insert new
> data, it can't handle update of data. Most of the time data is not static it
> changes with time. Yesterday order status was NEW today it's Processing,
> tomorrow it's SENT in a month it will be REFUNDED etc... {color}
>
> {color:#222222} {color}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
