MartijnVisser opened a new pull request, #29204:
URL: https://github.com/apache/flink/pull/29204
## What is the purpose of the change
Failing tests only, no fix: this draft PR shows the defect described in
FLINK-40681. When the planner does not require UPDATE_BEFORE on the inputs of
an outer join, an accumulate message that replaces a row on a unique-key inner
side is counted as an additional association of the outer row. The later DELETE
of that inner row then never re-emits the null-padded outer row, and the row
disappears from an upsert sink. The same arithmetic is in `JoinHelper` (async
state) and `StreamingSemiAntiJoinOperator`; the mini-batch join additionally
folds an UPDATE_AFTER followed by a DELETE within one bundle into nothing.
The nested query of FLINK-23740 is included as a differential test against a
batch oracle: the reported FULL OUTER JOIN variant is correct on master, its
LEFT OUTER JOIN control variant is not.
## Brief change log
- `StreamingJoinOperatorTest`: LEFT, RIGHT and FULL outer join cases where
the inner side sends a replacing UPDATE_AFTER (or a duplicate INSERT) and then
a DELETE; run with sync and async state
- `StreamingMiniBatchJoinOperatorTest`: the same across bundles and within
one bundle
- `StreamingSemiAntiJoinOperatorTest`: the anti join variant
- `OuterJoinUpsertInputITCase`: `A LEFT JOIN D` and the nested LEFT JOIN
query of FLINK-23740 into an upsert sink, asserting the raw changelog; a
control with a larger sink key that keeps UPDATE_BEFORE stays green
- `JoinTest`: plans pinning that the joins run in `[I,UA,D]` for these
queries
- `NestedOuterJoinChangelogStressITCase`: seeded differential test of the
FLINK-23740 query in both variants against a batch oracle
## Verifying this change
This change added tests. All of them except the control and the FULL OUTER
JOIN variant fail on master by design; CI is expected to be red until a fix
lands.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? not applicable
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (please specify the tool below)
Generated-by: Claude Code (Claude Fable 5.1)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
