prashantwason commented on PR #9545:
URL: https://github.com/apache/hudi/pull/9545#issuecomment-1702598436
Another idea:
Since the log blocks are completely created in memory before writing then
out to disk, we could invent a new flag for the log block header - final_block
When a log block has final_block=true then it means that that this is the
final block for the commit else this is followed by more blocks.
e.g. Commit 0 which adds a single log block.
log file = [log_block_c0_1(final_block=true)]
Commit 1 which adds 2 log blocks.
log file = [log_block_c0_1(final_block=true),
log_block_c1_1(final_block=false), log_block_c0_2(final_block=true)]
The logic to detect the completed log blocks may be simpler in this case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
