manojpec commented on a change in pull request #3977:
URL: https://github.com/apache/hudi/pull/3977#discussion_r751036800
##########
File path:
hudi-common/src/main/java/org/apache/hudi/common/table/log/AbstractHoodieLogRecordReader.java
##########
@@ -374,21 +374,23 @@ private void
processQueuedBlocksForInstant(Deque<HoodieLogBlock> logBlocks, int
LOG.info("Number of remaining logblocks to merge " + logBlocks.size());
// poll the element at the bottom of the stack since that's the order it
was inserted
HoodieLogBlock lastBlock = logBlocks.pollLast();
- switch (lastBlock.getBlockType()) {
- case AVRO_DATA_BLOCK:
- processDataBlock((HoodieAvroDataBlock) lastBlock, keys);
- break;
- case HFILE_DATA_BLOCK:
- processDataBlock((HoodieHFileDataBlock) lastBlock, keys);
- break;
- case DELETE_BLOCK:
- Arrays.stream(((HoodieDeleteBlock)
lastBlock).getKeysToDelete()).forEach(this::processNextDeletedKey);
- break;
- case CORRUPT_BLOCK:
- LOG.warn("Found a corrupt block which was not rolled back");
- break;
- default:
- break;
+ if (lastBlock != null) {
Review comment:
Spent some more time on this, but couldn't recreate the problem. The one
difference now is, the ExternalSpillableMap throwing ArithmeticException is
fixed now which wasn't the case when I tested on EMR. So, revering this
lastBlock null check for now, until i am able to recreate the problem and root
cause it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
