Gustavo de Morais created FLINK-40809:
-----------------------------------------
Summary: Regular outer join loses the null-padded row when an
input record is updated without UPDATE_BEFORE
Key: FLINK-40809
URL: https://issues.apache.org/jira/browse/FLINK-40809
Project: Flink
Issue Type: Sub-task
Components: Table SQL / API
Affects Versions: 1.20.5, 2.2.1, 2.3.0, 2.4.0
Reporter: Gustavo de Morais
Assignee: Gustavo de Morais
When the planner does not require UPDATE_BEFORE on a join input, the input's
unique key must contain the join key and the sink key must match the query's
upsert key. {{StreamingJoinOperator}} can then receive an UPDATE_AFTER, or a
repeated INSERT, that replaces a stored record with the same unique key. This
is common with upsert sources such as Kafka upsert topics.
The operator counts the replacement as a new association on the outer side.
When the record is later deleted, the counter goes from 2 to 1 instead of 1 to
0, so the null-padded row is never emitted again. The row silently disappears
from the sink.
Example:
{code:java}
-- customers: upsert source, PRIMARY KEY (id); sink PRIMARY KEY (order_id)
INSERT INTO sink
SELECT o.order_id, o.customer_id, c.name
FROM orders o LEFT JOIN customers c ON o.customer_id = c.id; {code}
# {{+I customers(7, 'Ann')}} → {{+I[o1, 7, Ann]}}
# {{+U customers(7, 'Anna')}} → {{{}+I[o1, 7, Anna]{}}}, and {{o1}} now counts
2 matches instead of 1
# {{-D customers(7)}} → {{{}-D[o1, 7, Anna]{}}}, but {{+I[o1, 7, null]}} is
missing
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
