[
https://issues.apache.org/jira/browse/HDFS-17097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17746189#comment-17746189
]
Xiaoqiao He commented on HDFS-17097:
------------------------------------
[~FuzzingTeam] Great, I try to search JIRA from your report, both of them focus
COMMON and HDFS modules, some are not critical issues but both are worth to
improvement IMO. Not sure if use automatic test system tools here, would you
mind to cover other modules, such as YARN?
P.S. I try reach out to you via email, but no mail address left anywhere, Let's
build email connection if you don't mind ( hexiaoqiao#apche.org, replace # to @
).
> Out of Memory when set dfs.namenode.blockreport.queue.size to a large number
> ----------------------------------------------------------------------------
>
> Key: HDFS-17097
> URL: https://issues.apache.org/jira/browse/HDFS-17097
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: ConfX
> Priority: Critical
> Attachments: reproduce.sh
>
>
> h2. What happened:
> When setting {{dfs.namenode.blockreport.queue.size}} to a large number,
> {{BlockManager}} throws an out-of-memory exception due to inappropriate
> checking and handling.
> *There is no checking or error-handling logic for this parameter at all.*
> h2. StackTrace:
>
> {noformat}
> java.lang.OutOfMemoryError: Java heap space
> at
> java.base/java.util.concurrent.ArrayBlockingQueue.<init>(ArrayBlockingQueue.java:270)
> at
> java.base/java.util.concurrent.ArrayBlockingQueue.<init>(ArrayBlockingQueue.java:254)
> at
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$BlockReportProcessingThread.<init>(BlockManager.java:5243)
> at
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.<init>(BlockManager.java:613)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:869)
> at
> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.initialize(SecondaryNameNode.java:248)
> at
> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.<init>(SecondaryNameNode.java:194)
> at
> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.<init>(SecondaryNameNode.java:181){noformat}
> h2. Reproduce:
> (1) Set {{dfs.namenode.blockreport.queue.size}} to a large value, e.g.,
> 1807890864
> (2) Run a simple test that exercises this parameter, e.g.
> {{org.apache.hadoop.hdfs.server.namenode.TestNameNodeHttpServerXFrame#testSecondaryNameNodeXFrame}}
> For an easy reproduction, run the reproduce.sh in the attachment.
> We are happy to provide a patch if this issue is confirmed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
