nsivabalan commented on issue #14437: URL: https://github.com/apache/hudi/issues/14437#issuecomment-4795158848
This issue appears to be resolved on current `master`. The class was renamed to `BaseHoodieLogRecordReader`, and `scanInternal` now uses a two-pass scan that explicitly supports out-of-order rollback blocks in multi-writer scenarios. The Javadoc at `hudi-common/src/main/java/org/apache/hudi/common/table/log/BaseHoodieLogRecordReader.java` lines 224-248 calls out the exact scenario from this issue: > Scanning blocks is easy to do in single writer mode, where the rollback block is right after the effected data blocks. With multi-writer mode the blocks can be out of sync. An example scenario. `B1, B2, B3, B4, R1(B3), B5`. In this case, rollback block R1 is invalidating the B3 which is not the previous block. In pass 1 (lines 272-354), each rollback command block is resolved via the `TARGET_INSTANT_TIME` header and the targeted instant's blocks are removed from `instantToBlocksMap` regardless of position (lines 334-345). Pass 2 (lines 370-411) then iterates instants in reverse order to correctly position compacted blocks. The two-pass logic was introduced in commit `86a1efbff1` (HUDI-3900, PR #5958). Closing as resolved seems appropriate — please reopen if you can reproduce on a recent release. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
