wwj6591812 opened a new pull request, #9142:
URL: https://github.com/apache/paimon/pull/9142
### Purpose
Batch pipelines sometimes need to process only the newest committed
snapshot, but currently must first query its ID and construct
`incremental-between` boundaries.
This adds `scan.mode = 'latest-delta'` for batch sources. At planning time
it selects the literal latest snapshot `N` and reuses the existing DELTA
scanner for `(N - 1, N]`.
It deliberately does not search backwards for an APPEND snapshot. Therefore,
if the latest snapshot is COMPACT or OVERWRITE, the query returns no records
instead of treating rewritten files as new logical data. Streaming use is
rejected, and boundary or time-travel companion options are rejected rather
than ignored.
This is a convenience mode, not a persistent consumer cursor. Workflows that
must consume multiple snapshots reliably should continue to use explicit
`incremental-between` boundaries or tags.
```sql
SELECT *
FROM t /*+ OPTIONS('scan.mode' = 'latest-delta') */;
```
### Tests
- Added core coverage for latest APPEND, latest COMPACT, empty table,
streaming rejection, level-0 preservation, incompatible options, and
postpone-merge rejection.
- Added a Flink SQL integration test proving two appends return only the
latest snapshot.
- Added Flink catalog option coverage for batch and streaming modes.
- Regenerated configuration docs with the official docs profile.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
