fhueske opened a new pull request, #29081:
URL: https://github.com/apache/flink/pull/29081
## What is the purpose of the change
LATERAL SNAPSHOT detected the build-side watermark by looking for a row-time
indicator field in the projected TABLE argument. When the watermark sits on a
column that the column-expansion strategy hides from SELECT * (e.g. a virtual
metadata column under EXCLUDE_DEFAULT_VIRTUAL_METADATA_COLUMNS), TABLE t
expands to a projection that drops the row-time attribute, so the rule found
none and wrongly rejected the query with "LATERAL SNAPSHOT requires a watermark
on the build-side input".
Recover the row-time attribute from below the hiding projection and
re-append it so it reaches the operator: from a WatermarkAssigner, or, when the
watermark was pushed into the source, by re-stamping the scan's row-time column
(resolved by name from the watermark spec) as a row-time indicator. The
top-level Calc projects the re-appended column away, so the output is unchanged.
## Brief change log
* unhide rowtime attributes on the input relation of a SNAPSHOT function call
* add rowtime attribute to projection of Calc nodes in the subtree of the
function input argument
* fail if no rowtime attribute can be found (like today)
* add tests to ensure hidden rowtime attributes become available
* plan-level tests + semantic test
## Verifying this change
Run new and existing tests for LateralSnapshotJoin
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? n/a
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (please specify the tool below)
Generated-by: Claude Opus 4.8 (1M context)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
