[ 
https://issues.apache.org/jira/browse/FLINK-41533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated FLINK-41533:
-----------------------------------
    Labels: pull-request-available  (was: )

> Upsert-kafka source declares full deletes but emits key-only deletes
> --------------------------------------------------------------------
>
>                 Key: FLINK-41533
>                 URL: https://issues.apache.org/jira/browse/FLINK-41533
>             Project: Flink
>          Issue Type: Bug
>          Components: Connectors / Kafka
>            Reporter: Gustavo de Morais
>            Assignee: Gustavo de Morais
>            Priority: Major
>              Labels: pull-request-available
>
> The upsert-kafka source declares its changelog mode as {{UPDATE_AFTER, 
> DELETE}} without {{keyOnlyDeletes(true)}}, so the planner treats its deletes 
> as full deletes. At runtime a Kafka tombstone becomes a DELETE row with only 
> the key fields set and all value columns NULL.
> Since FLINK-37475 the planner drops the {{ChangelogNormalize}} when the 
> source already provides the delete kind a consumer requires. For upsert-kafka 
> this means consumers that require full deletes receive key-only deletes with 
> NULL values. For example, a row-semantic {{TO_CHANGELOG}} with an 
> {{op_mapping}} that maps DELETE but not UPDATE_BEFORE emits DELETE rows with 
> NULL values instead of the last row.
> We should declare {{keyOnlyDeletes(true)}} on the upsert-kafka source, so the 
> planner keeps the {{ChangelogNormalize}} where full deletes are required.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to