[ https://issues.apache.org/jira/browse/AIRFLOW-2679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bolke de Bruin updated AIRFLOW-2679: ------------------------------------ Fix Version/s: (was: 1.10) (was: 1.10.0) > GoogleCloudStorageToBigQueryOperator to support UPSERT > ------------------------------------------------------ > > Key: AIRFLOW-2679 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2679 > Project: Apache Airflow > Issue Type: Improvement > Reporter: jack > Priority: Major > > Currently the > {color:#222222}GoogleCloudStorageToBigQueryOp{color}{color:#222222}erator > support incremental load using > *{color:#404040}max_id_key{color}*{color:#404040} {color}.{color} > > {color:#222222}However many systems actually needs "UPSERT" in terms of - if > row exists update it, if not insert/copy it.{color} > {color:#222222}Currently the operator assumes that we only need to insert new > data, it can't handle update of data. Most of the time data is not static it > changes with time. Yesterday order status was NEW today it's Processing, > tomorrow it's SENT in a month it will be REFUNDED etc... {color} > > {color:#222222} {color} -- This message was sent by Atlassian JIRA (v7.6.3#76005)