[
https://issues.apache.org/jira/browse/HDFS-16138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17409130#comment-17409130
]
Renukaprasad C commented on HDFS-16138:
---------------------------------------
Thanks [~hemanthboyina], Exception being thrown in
org.apache.hadoop.util.ExitUtil#terminate(int, java.lang.String) via BP
Processing thread -
org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.BlockReportProcessingThread#run
Below code, which create new exception bu consuming the actual exception.
public static void terminate(int status, String msg) throws ExitException {
terminate(new ExitException(status, msg));
}
I couldnt extend UT as error is from private thread. Simulation would required
lot of mocking. If you still insist, we shall look into it further.
Also, other comments addressed and pushed the changes. Please review the
changes. Thank you.
> BlockReportProcessingThread exit doesnt print the acutal stack
> --------------------------------------------------------------
>
> Key: HDFS-16138
> URL: https://issues.apache.org/jira/browse/HDFS-16138
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Renukaprasad C
> Assignee: Renukaprasad C
> Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> BlockReportProcessingThread thread may gets exited with multiple reasons, but
> the current logging prints only the exception message with different stack
> which is difficult to debug the issue.
>
> Existing logging:
> 2021-07-20 10:20:23,104 [Block report processor] INFO util.ExitUtil
> (ExitUtil.java:terminate(210)) - Exiting with status 1: Block report
> processor encountered fatal exception: java.lang.AssertionError
> 2021-07-20 10:20:23,104 [Block report processor] ERROR util.ExitUtil
> (ExitUtil.java:terminate(213)) - Terminate called
> 1: Block report processor encountered fatal exception:
> java.lang.AssertionError
> at
> org.apache.hadoop.util.ExitUtil.terminate(ExitUtil.java:304)
> at
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$BlockReportProcessingThread.run(BlockManager.java:5315)
> Exception in thread "Block report processor" 1: Block report processor
> encountered fatal exception: java.lang.AssertionError
> at
> org.apache.hadoop.util.ExitUtil.terminate(ExitUtil.java:304)
> at
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$BlockReportProcessingThread.run(BlockManager.java:5315)
>
> Actual issue found at:
> 2021-07-20 10:20:23,101 [Block report processor] ERROR
> blockmanagement.BlockManager (BlockManager.java:run(5314)) -
> java.lang.AssertionError
> java.lang.AssertionError
> at
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.addStoredBlock(BlockManager.java:3480)
> at
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.processAndHandleReportedBlock(BlockManager.java:4280)
> at
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.addBlock(BlockManager.java:4202)
> at
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.processIncrementalBlockReport(BlockManager.java:4338)
> at
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.processIncrementalBlockReport(BlockManager.java:4305)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.processIncrementalBlockReport(FSNamesystem.java:4853)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer$2.run(NameNodeRpcServer.java:1657)
> at
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$BlockReportProcessingThread.processQueue(BlockManager.java:5334)
> at
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$BlockReportProcessingThread.run(BlockManager.java:5312)
>
> This issue found while working on FGL branch. But, same issue can happen in
> Trunk also in any error scenario.
>
> [~hemanthboyina] [~hexiaoqiao]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
