Gustavo de Morais created FLINK-41533:
-----------------------------------------

             Summary: Upsert-kafka source declares full deletes but emits 
key-only deletes
                 Key: FLINK-41533
                 URL: https://issues.apache.org/jira/browse/FLINK-41533
             Project: Flink
          Issue Type: Bug
          Components: Connectors / Kafka
            Reporter: Gustavo de Morais
            Assignee: Gustavo de Morais


The upsert-kafka source declares its changelog mode as {{UPDATE_AFTER, DELETE}} 
without {{keyOnlyDeletes(true)}}, so the planner treats its deletes as full 
deletes. At runtime a Kafka tombstone becomes a DELETE row with only the key 
fields set and all value columns NULL.

Since FLINK-37475 the planner drops the {{ChangelogNormalize}} when the source 
already provides the delete kind a consumer requires. For upsert-kafka this 
means consumers that require full deletes receive key-only deletes with NULL 
values. For example, a row-semantic {{TO_CHANGELOG}} with an {{op_mapping}} 
that maps DELETE but not UPDATE_BEFORE emits DELETE rows with NULL values 
instead of the last row.

We should declare {{keyOnlyDeletes(true)}} on the upsert-kafka source, so the 
planner keeps the {{ChangelogNormalize}} where full deletes are required.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to