weiqingy opened a new pull request, #28877: URL: https://github.com/apache/flink/pull/28877
Part of the FLIP-497 implementation stack under umbrella [FLINK-36953](https://issues.apache.org/jira/browse/FLINK-36953). Landing order: | Step | Sub-task | Scope | | --- | --- | --- | | PR-1a | [FLINK-40167](https://issues.apache.org/jira/browse/FLINK-40167) | EARLY_FIRE hint surface + option validation (#28353, merged) | | PR-1b | [FLINK-40168](https://issues.apache.org/jira/browse/FLINK-40168) | Thread the hint into the interval join (#28796, merged) | | PR-2 | [FLINK-40169](https://issues.apache.org/jira/browse/FLINK-40169) | `target` option (#28827, in review) | | **PR-3 (this PR)** | [FLINK-40170](https://issues.apache.org/jira/browse/FLINK-40170) | Update-producing changelog mode + insert-only guard | | PR-4 | [FLINK-40171](https://issues.apache.org/jira/browse/FLINK-40171) | Runtime early-fire emit + retraction | | PR-5 | [FLINK-40172](https://issues.apache.org/jira/browse/FLINK-40172) | Processing-time early fire on an event-time join | | PR-6 | [FLINK-40173](https://issues.apache.org/jira/browse/FLINK-40173) | State restore coverage | | PR-7 | [FLINK-40174](https://issues.apache.org/jira/browse/FLINK-40174) | User-facing documentation | Opened as a draft because it is stacked on #28827, which is still in review. Until that merges, the commit list and diff here also carry PR-2's commit. Once #28827 merges I will rebase onto master, leaving only this PR's change, and take it out of draft. ## What is the purpose of the change With the `EARLY_FIRE` hint, an outer interval join emits a speculative null-padded row after the delay and corrects it when a match later arrives, so its result is no longer insert-only. This makes the planner infer that update-producing changelog mode, and reject an insert-only downstream with an error that names the hint instead of the generic "doesn't support consuming update changes" message. Inner joins and negative-window joins only ever emit inserts, so the hint stays inert for them and they remain append-only. Runtime behavior is unchanged. The operator still ignores the hint; emission lands in PR-4. ## Brief change log - `FlinkChangelogModeInferenceProgram` gets its own `ModifyKindSet` arm for `StreamPhysicalIntervalJoin`: children still consume insert-only, but the node provides `INSERT` and, when the hint makes it update-producing, `UPDATE`. It keeps its place in the `UpdateKind` and `DeleteKind` arms. - `StreamPhysicalIntervalJoin.produceEarlyFireUpdates` gates that on three conditions: the hint is set, the join is outer, and the window span is non-negative. - A tailored `TableException` when such a join feeds an insert-only downstream. - The compiled-plan round-trip test's sink now declares `sink-insert-only=false`. Its early-fire outer join produces updates under the new inference, so the previously insert-only sink no longer accepts that pipeline. This is the one existing behavior this PR changes. ## Verifying this change This change added tests and can be verified as follows: - `EarlyFireJoinHintTest`: an early-fire outer join is inferred as `changelogMode=[I,UA]`; an insert-only downstream fails with the tailored error; a negative-window join and an inner join both stay `[I]` while still carrying `earlyFireDelay` on the join node, so the hint is attached but inert rather than absent. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no - The serializers: no - The runtime per-record code paths (performance sensitive): no - Anything that affects deployment or recovery: no - The S3 file system connector: no ## Documentation - Does this pull request introduce a new feature? no (planner inference for the FLIP-497 hint) - If yes, how is the feature documented? not applicable --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes (please specify the tool below) Generated-by: Claude Code (Anthropic) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
