[
https://issues.apache.org/jira/browse/FLINK-40874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gustavo de Morais reassigned FLINK-40874:
-----------------------------------------
Assignee: (was: Rion Williams)
> Mini-batch streaming join evaluates non-equi conditions on key-only deletes
> of folded pairs
> -------------------------------------------------------------------------------------------
>
> Key: FLINK-40874
> URL: https://issues.apache.org/jira/browse/FLINK-40874
> Project: Flink
> Issue Type: Sub-task
> Components: Table SQL / API
> Affects Versions: 2.3.0, 2.2.1, 2.1.3
> Reporter: Gustavo de Morais
> Priority: Major
>
> In mini-batch mode, a key-only DELETE followed by an accumulation of the same
> unique key within one bundle is processed as a suppressed pair. Both calls of
> the pair use the partial delete row: once to retract the old matches, and
> once to decide whether a null padding change cancels out. With a non-equi
> join condition on non-key columns, this produces wrong matches and wrong null
> padding.
> Example, mini-batch on: {{{}L LEFT JOIN R ON L.k = R.k AND R.v <>
> 'unknown'{}}}, where R is an upsert input with key-only deletes.
>
> {code:java}
> bundle 1: +I L(k=1), +I R(id=1, k=1, v=AIR) -> +I[L, AIR]
> bundle 2: -D R(id=1) (key only), +I R(id=1, k=1, v=unknown) -> nothing
> expected: -D[L, AIR], +I[L, null] {code}
> Expected: -D[L, AIR], +I[L, null]. Actual: the null padding change is
> suppressed based on the partial row.Proposed fix: look up the stored record
> once per pair, and use it for both the retraction and the pair's null padding
> check.Related: FLINK-40858 (the same fix for the non-mini-batch path).
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
