nanjeshramesh opened a new pull request, #20353: URL: https://github.com/apache/hudi/pull/20353
### Describe the issue this Pull Request addresses Closes #20325 Scheduling log compaction throws when a Merge-on-Read file slice contains CDC log files and the planner takes the block metadata scanning path. `HoodieLogBlockMetadataScanner` builds its reader context without a schema, and a CDC data block cannot be constructed without one (`Option.of(readerSchema)` in `HoodieCDCDataBlock`). If a CDC block does get past that, `BaseHoodieLogRecordReader.scanInternal` rejects the block type. ### Summary and Changelog `HoodieLogBlockMetadataScanner` now drops CDC log files from the log files it scans. This matches `HoodieMergedLogRecordScanner` and `HoodieUnMergedLogRecordScanner`, which already skip `.cdc` log files because they only carry supplemental change data and are never merged into the base file. Files are matched on the CDC log file suffix, so it also covers tables where CDC was disabled after CDC logs were written. Added `TestHoodieLogBlockMetadataScanner`, which writes a regular data log file and a CDC log file for the same file group and scans both. It fails on master and passes with this change. ### Impact Log compaction scheduling no longer fails on file slices that have CDC log files. Which file slices are chosen for log compaction is otherwise unchanged. The log file count shortcut in `HoodieLogCompactionPlanGenerator` is not touched. ### Risk Level low. The scanner only inspects block metadata, and CDC blocks were never part of what it reports. ### Documentation Update none ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
