Gustavo de Morais created FLINK-41534:
-----------------------------------------

             Summary: Accept key-only deletes in the upsert-kafka sink
                 Key: FLINK-41534
                 URL: https://issues.apache.org/jira/browse/FLINK-41534
             Project: Flink
          Issue Type: Improvement
          Components: Connectors / Kafka
            Reporter: Gustavo de Morais
            Assignee: Gustavo de Morais


The upsert-kafka sink declares that it requires full deletes, although it only 
writes a tombstone with the key. Once the upsert-kafka source declares key-only 
deletes, a pipeline from an upsert-kafka source into an upsert-kafka sink with 
the same primary key needs a {{ChangelogNormalize}} again to turn key-only 
deletes into full deletes, which adds state the sink does not need.

We should declare {{keyOnlyDeletes}} on the sink changelog mode where only the 
key is needed for a delete: upsert mode with a single topic and the default 
partitioner. Multiple topics (the {{topic}} metadata column) and custom 
partitioners read value columns of a delete, so they keep requiring full 
deletes.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to