ConfX created HDFS-17097:
----------------------------
Summary: Out of Memory when set
dfs.namenode.blockreport.queue.size to a large number
Key: HDFS-17097
URL: https://issues.apache.org/jira/browse/HDFS-17097
Project: Hadoop HDFS
Issue Type: Bug
Reporter: ConfX
Attachments: reproduce.sh
h2. What happened:
When setting {{dfs.namenode.blockreport.queue.size}} to a large number,
{{BlockManager}} throws an out-of-memory exception due to inappropriate
checking and handling.
*There is no checking or error-handling logic for this parameter at all.*
h2. StackTrace:
{noformat}
java.lang.OutOfMemoryError: Java heap space
at
java.base/java.util.concurrent.ArrayBlockingQueue.<init>(ArrayBlockingQueue.java:270)
at
java.base/java.util.concurrent.ArrayBlockingQueue.<init>(ArrayBlockingQueue.java:254)
at
org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$BlockReportProcessingThread.<init>(BlockManager.java:5243)
at
org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.<init>(BlockManager.java:613)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:869)
at
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.initialize(SecondaryNameNode.java:248)
at
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.<init>(SecondaryNameNode.java:194)
at
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.<init>(SecondaryNameNode.java:181){noformat}
h2. Reproduce:
(1) Set {{dfs.namenode.blockreport.queue.size}} to a large value, e.g.,
1807890864
(2) Run a simple test that exercises this parameter, e.g.
{{org.apache.hadoop.hdfs.server.namenode.TestNameNodeHttpServerXFrame#testSecondaryNameNodeXFrame}}
For an easy reproduction, run the reproduce.sh in the attachment.{{{}{}}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
