[
https://issues.apache.org/jira/browse/HDFS-17097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17746010#comment-17746010
]
ConfX edited comment on HDFS-17097 at 7/23/23 8:25 PM:
-------------------------------------------------------
Thank you for reaching out. We're humans. :D
We use a template to make it easier for both (1) you to read reports and (2) us
to prepare reports.
We are very happy to help, just let us know how.
[~hexiaoqiao] It would be great if you could point out the interesting cases
that you mentioned such that we can start writing patches for those cases as
soon as possible.
was (Author: JIRAUSER296392):
Thank you for reaching out. We're humans. :D
We use a template to make it easier for both (1) you to read reports and (2) us
to prepare reports.
We are very happy to help, just let us know how.
> Out of Memory when set dfs.namenode.blockreport.queue.size to a large number
> ----------------------------------------------------------------------------
>
> Key: HDFS-17097
> URL: https://issues.apache.org/jira/browse/HDFS-17097
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: ConfX
> Priority: Critical
> Attachments: reproduce.sh
>
>
> h2. What happened:
> When setting {{dfs.namenode.blockreport.queue.size}} to a large number,
> {{BlockManager}} throws an out-of-memory exception due to inappropriate
> checking and handling.
> *There is no checking or error-handling logic for this parameter at all.*
> h2. StackTrace:
>
> {noformat}
> java.lang.OutOfMemoryError: Java heap space
> at
> java.base/java.util.concurrent.ArrayBlockingQueue.<init>(ArrayBlockingQueue.java:270)
> at
> java.base/java.util.concurrent.ArrayBlockingQueue.<init>(ArrayBlockingQueue.java:254)
> at
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$BlockReportProcessingThread.<init>(BlockManager.java:5243)
> at
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.<init>(BlockManager.java:613)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:869)
> at
> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.initialize(SecondaryNameNode.java:248)
> at
> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.<init>(SecondaryNameNode.java:194)
> at
> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.<init>(SecondaryNameNode.java:181){noformat}
> h2. Reproduce:
> (1) Set {{dfs.namenode.blockreport.queue.size}} to a large value, e.g.,
> 1807890864
> (2) Run a simple test that exercises this parameter, e.g.
> {{org.apache.hadoop.hdfs.server.namenode.TestNameNodeHttpServerXFrame#testSecondaryNameNodeXFrame}}
> For an easy reproduction, run the reproduce.sh in the attachment.
> We are happy to provide a patch if this issue is confirmed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
