[
https://issues.apache.org/jira/browse/FLINK-40831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gustavo de Morais updated FLINK-40831:
--------------------------------------
Affects Version/s: 2.1.3
1.20.5
2.2.1
2.3.0
> Mini-batch outer join loses the null-padded row when an input record is
> updated without UPDATE_BEFORE
> -----------------------------------------------------------------------------------------------------
>
> Key: FLINK-40831
> URL: https://issues.apache.org/jira/browse/FLINK-40831
> Project: Flink
> Issue Type: Sub-task
> Components: Table SQL / API
> Affects Versions: 2.3.0, 2.2.1, 1.20.5, 2.1.3
> Reporter: Gustavo de Morais
> Assignee: Gustavo de Morais
> Priority: Major
>
> {{MiniBatchStreamingJoinOperator}} shares the per-record logic of
> {{{}StreamingJoinOperator{}}}, so it has the same problem. When a join input
> has no UPDATE_BEFORE, an UPDATE_AFTER can replace a stored record with the
> same unique key, and the operator counts it as a new match on the outer side.
> After the record is deleted, the count doesn't reach 0, and the null-padded
> row is never emitted again.
> Mini-batch adds one more case. Within a bundle, a {{{}-U{}}}/{{{}+U{}}} pair
> on the same unique key is folded into a suppressed pair: the {{-U}}
> decrements the count but keeps the record in state. The paired {{+U}} must
> then always count again. Otherwise an outer row with two matches loses one,
> and deleting the other match emits a null-padded row while a match still
> exists.
> Example, with order 1 matching two shipments, {{S1}} and {{{}S2{}}}:
> # One bundle: {{{}-U S1{}}}, {{+U S1'}} → the count must stay 2.
> # {{-D S2}} → the count goes to 1. Only {{-D[order 1, S2]}} is expected,
> with no {{{}+I[order 1, null]{}}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
