fhan688 opened a new pull request, #19226:
URL: https://github.com/apache/hudi/pull/19226
### Describe the issue this Pull Request addresses
`rollback_to_instant` currently rolls back only the named instant. If the
target instant is not the latest completed instant, later
completed instants can remain on the active timeline, so the table is not
actually rolled back to the requested instant.
MOR rollback planning also has a correctness gap around compaction.
Rolling back a later inflight deltacommit after a completed
compaction must not delete files produced by the compaction. Conversely,
rolling back the compaction itself must still clean up the compaction output.
### Summary and Changelog
This PR fixes rollback-to-instant semantics and MOR rollback planning
around compaction instants.
Changes:
- Update Spark SQL `rollback_to_instant` to rollback the target instant
and all later commit/compaction instants in reverse timeline
order.
- Handle requested/inflight compaction rollback explicitly in the
procedure path.
- Generate rollback requests for MOR compaction instants even when the
compaction instant is requested.
- Use completed commit metadata when available to reduce rollback
partition listing scope.
- For MOR compaction rollback, keep compaction output when rolling back a
later inflight deltacommit, and clean it when the compaction
itself is rolled back.
- Add regression coverage for multi-instant rollback and MOR compaction
rollback cleanup.
No code was copied.
Validation:
- `mvn ... test-compile`
- `TestCallProcedure`: 9 tests passed
-
`TestHoodieSparkMergeOnReadTableRollback#testRollbackCompactionCleansFilesAfterHigherInflightDeltaRollback`:
1 test passed
### Impact
No new public API, config, or storage format change.
Behavior change:
- `rollback_to_instant` now rolls back the requested instant and all later
instants, matching rollback-to-instant semantics.
- MOR rollback planning avoids deleting committed compaction output when
rolling back a later inflight deltacommit.
### Risk Level
medium
Rollback and MOR compaction cleanup are correctness-sensitive paths. The
changes are scoped to rollback planning/execution and covered
by targeted regression tests for both Spark SQL procedure behavior and MOR
compaction file cleanup.
### Documentation Update
none
This is a correctness fix for existing rollback behavior and does not add
a new feature, config, or public API.
### Contributor's checklist
- [x] Read through [contributor's
guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Enough context is provided in the sections above
- [x] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
