manojpec commented on a change in pull request #3977:
URL: https://github.com/apache/hudi/pull/3977#discussion_r749886365
##########
File path:
hudi-common/src/main/java/org/apache/hudi/common/table/log/AbstractHoodieLogRecordReader.java
##########
@@ -374,21 +374,23 @@ private void
processQueuedBlocksForInstant(Deque<HoodieLogBlock> logBlocks, int
LOG.info("Number of remaining logblocks to merge " + logBlocks.size());
// poll the element at the bottom of the stack since that's the order it
was inserted
HoodieLogBlock lastBlock = logBlocks.pollLast();
- switch (lastBlock.getBlockType()) {
- case AVRO_DATA_BLOCK:
- processDataBlock((HoodieAvroDataBlock) lastBlock, keys);
- break;
- case HFILE_DATA_BLOCK:
- processDataBlock((HoodieHFileDataBlock) lastBlock, keys);
- break;
- case DELETE_BLOCK:
- Arrays.stream(((HoodieDeleteBlock)
lastBlock).getKeysToDelete()).forEach(this::processNextDeletedKey);
- break;
- case CORRUPT_BLOCK:
- LOG.warn("Found a corrupt block which was not rolled back");
- break;
- default:
- break;
+ if (lastBlock != null) {
Review comment:
During EMR/S3 testing, I encountered NPE in this block. Looking more
into it, ArrayDeque is not thread safe (though my test wasn't doing any multi
writer tests) and `isEmpty()` check is not fool proof. That is, it can say it
is not empty and still have poll() returning null. Haven't got a chance to dig
how this case is getting exposed in my test case, but for now closing the gap
and avoiding the crash by doing an additional check.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
