yihua opened a new pull request, #983:
URL: https://github.com/apache/hudi-rs/pull/983
## Description
Time travel never read clean metadata, so a read at an instant older than
the latest clean's earliest commit to retain skipped every file group the clean
emptied and returned the remaining rows without error. Snapshot planning now
refuses an explicit `as_of_timestamp` earlier than that bound (or, when the
clean records none under `KEEP_LATEST_FILE_VERSIONS`, earlier than the first
active commit), with the same messages as the clean half of Java's
`TimelineUtils.validateTimestampAsOf`. This adds `Action::Clean` and a small
decoder for `earliestCommitToRetain`. The check costs one timeline listing plus
a read of the latest clean file per explicit as-of read, and does not apply to
latest-snapshot or incremental reads. The other half of `validateTimestampAsOf`
(rejecting an as-of at or after the first incomplete commit) is not ported here.
Closes #913
## How are the changes test-covered
- [ ] N/A
- [x] Automated tests (unit and/or integration tests)
- [ ] Manual tests
- [ ] Details are described below
New Spark-written COW fixtures (table versions 6 and 9 with
`KEEP_LATEST_COMMITS`, version 9 with `KEEP_LATEST_FILE_VERSIONS`, with gold
data) back integration tests that assert the same refusals and results as Hudi
1.1.1's Spark reader through `read`, `read_stream` and `get_file_slices`; on
main they return partial rows instead. Unit tests cover the clean metadata
decoding.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]