n3nash commented on a change in pull request #1332: [HUDI -409] Match header
and footer block length to improve corrupted block detection
URL: https://github.com/apache/incubator-hudi/pull/1332#discussion_r379557708
##########
File path:
hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieLogFileReader.java
##########
@@ -362,13 +370,18 @@ public HoodieLogBlock prev() throws IOException {
// blocksize should read everything about a block including the length as
well
try {
inputStream.seek(reverseLogFilePosition - blockSize);
+ // get the block size from head and match it with the block size from
tail
Review comment:
This code is pretty complicated to understand. I see that you removed
hasNext() and added some code around this + handling of corrupt blocks here.
Can this be simplified with hasNext(), checkCorruptBlock() method level
abstractions ? We need more logs as well to explain such checks..
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
