[
https://issues.apache.org/jira/browse/FLINK-41533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-41533:
-----------------------------------
Labels: pull-request-available (was: )
> Upsert-kafka source declares full deletes but emits key-only deletes
> --------------------------------------------------------------------
>
> Key: FLINK-41533
> URL: https://issues.apache.org/jira/browse/FLINK-41533
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka
> Reporter: Gustavo de Morais
> Assignee: Gustavo de Morais
> Priority: Major
> Labels: pull-request-available
>
> The upsert-kafka source declares its changelog mode as {{UPDATE_AFTER,
> DELETE}} without {{keyOnlyDeletes(true)}}, so the planner treats its deletes
> as full deletes. At runtime a Kafka tombstone becomes a DELETE row with only
> the key fields set and all value columns NULL.
> Since FLINK-37475 the planner drops the {{ChangelogNormalize}} when the
> source already provides the delete kind a consumer requires. For upsert-kafka
> this means consumers that require full deletes receive key-only deletes with
> NULL values. For example, a row-semantic {{TO_CHANGELOG}} with an
> {{op_mapping}} that maps DELETE but not UPDATE_BEFORE emits DELETE rows with
> NULL values instead of the last row.
> We should declare {{keyOnlyDeletes(true)}} on the upsert-kafka source, so the
> planner keeps the {{ChangelogNormalize}} where full deletes are required.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)