Stephen0421 opened a new pull request, #10420:
URL: https://github.com/apache/paimon/pull/10420
### Purpose
Add the Spark procedure `sys.create_tag_from_watermark`, matching the
existing Flink procedure.
The procedure finds the earliest retained snapshot whose watermark is
greater than or equal to the given value, including snapshots that have already
expired but are still kept by a tag, then creates a tag on that snapshot.
Snapshots with a null watermark are skipped. If none matches, it throws
`SnapshotNotExistException`.
This only reads watermarks already stored on snapshots (for example from
Flink or Spark Structured Streaming writes). It does not generate a watermark.
```sql
CALL sys.create_tag_from_watermark(
`table` => 'default.T',
`tag` => 'my_tag',
`watermark` => 1724404318750,
time_retained => '1 d'
);
```
Output columns: `tagName`, `snapshot`, `commit_time`, `watermark`.
### Test
- [x] `CreateTagFromWatermarkProcedureTest`: tag the first snapshot at or
after the watermark, and fall back to an expired snapshot that is still kept by
a tag
- [x] A watermark later than every snapshot throws
`SnapshotNotExistException`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
