[
https://issues.apache.org/jira/browse/HDFS-17097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744429#comment-17744429
]
Xiaoqiao He commented on HDFS-17097:
------------------------------------
[~FuzzingTeam] Thanks for your report. We received many JIRAs reported from you
where have the same format. And some of them are very interesting. Would you
mind to confirm if this is one robot or human here (Please let's me if
something wrong) to have a better prospects for cooperation. Thanks again.
> Out of Memory when set dfs.namenode.blockreport.queue.size to a large number
> ----------------------------------------------------------------------------
>
> Key: HDFS-17097
> URL: https://issues.apache.org/jira/browse/HDFS-17097
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: ConfX
> Priority: Critical
> Attachments: reproduce.sh
>
>
> h2. What happened:
> When setting {{dfs.namenode.blockreport.queue.size}} to a large number,
> {{BlockManager}} throws an out-of-memory exception due to inappropriate
> checking and handling.
> *There is no checking or error-handling logic for this parameter at all.*
> h2. StackTrace:
>
> {noformat}
> java.lang.OutOfMemoryError: Java heap space
> at
> java.base/java.util.concurrent.ArrayBlockingQueue.<init>(ArrayBlockingQueue.java:270)
> at
> java.base/java.util.concurrent.ArrayBlockingQueue.<init>(ArrayBlockingQueue.java:254)
> at
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$BlockReportProcessingThread.<init>(BlockManager.java:5243)
> at
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.<init>(BlockManager.java:613)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:869)
> at
> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.initialize(SecondaryNameNode.java:248)
> at
> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.<init>(SecondaryNameNode.java:194)
> at
> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.<init>(SecondaryNameNode.java:181){noformat}
> h2. Reproduce:
> (1) Set {{dfs.namenode.blockreport.queue.size}} to a large value, e.g.,
> 1807890864
> (2) Run a simple test that exercises this parameter, e.g.
> {{org.apache.hadoop.hdfs.server.namenode.TestNameNodeHttpServerXFrame#testSecondaryNameNodeXFrame}}
> For an easy reproduction, run the reproduce.sh in the attachment.
> We are happy to provide a patch if this issue is confirmed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
