nada-attia opened a new pull request, #18142:
URL: https://github.com/apache/hudi/pull/18142
### Describe the issue this Pull Request addresses
This PR adds a new API `getAllLogFilesWithMaxCommit` to `LogReaderUtils`
that retrieves log files created by commits with instant timestamps less than
or equal to a specified max commit time. This is useful for MDT (Metadata
Table) consistency checks and log file validation scenarios.
### Summary and Changelog
**Summary:** Adds a new utility method to fetch log files filtered by commit
time, along with the commit times associated with each block in those log files.
**Changelog:**
- Added `getAllLogFilesWithMaxCommit` method to `LogReaderUtils` class
- The method:
- Gets the filtered timeline based on commits completed before or on the
max instant
- Gets all file slices in given partitions based on max commit instant
- Gets all log files from each file slice
- For each log file, returns the list of commit instant times for blocks
created on or before the max commit instant time
- Added unit tests in `TestLogReaderUtils.java`
### Impact
**Public API Changes:**
- Added new public static method
`getAllLogFilesWithMaxCommit(HoodieTableMetaClient,
AbstractTableFileSystemView, List<String>, String, HoodieEngineContext)` to
`LogReaderUtils`
**User-Facing Changes:**
Users can now programmatically retrieve log files filtered by a maximum
commit time, which is useful for:
- MDT consistency validation
- Log file auditing and debugging
- Custom tooling for log file analysis
### Risk Level
**Low**
The changes are purely additive:
- New utility method added to existing class
- No modification to existing functionality
- Well-tested with unit tests
- Uses existing, stable Hudi APIs internally
### Documentation Update
None - This is an internal utility method. The Javadoc on the method
provides sufficient documentation for developers.
### Contributor's checklist
- [x] Read through [contributor's
guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Enough context is provided in the sections above
- [x] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
