[
https://issues.apache.org/jira/browse/HDFS-1348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12900777#action_12900777
]
Hairong Kuang commented on HDFS-1348:
-------------------------------------
Read more code to understand how check works. Currently check iterating up to a
default of 5 datanodes to see if they are done with decommission or not. It is
easy to release the global fsnamesytem lock in between checking two nodes.
When checking one datanode, it transverses all its blocks to see if they are
replicated during which the global lock is held. If a datanode has like 100K
blocks, this is going to be a non-trivial amount of time. Any idea for breaking
this into smaller chunks without sacrificing accuracy?
> DecommissionManager holds fsnamesystem lock during the whole process of
> checking if decomissioning DataNodes are finished or not
> --------------------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-1348
> URL: https://issues.apache.org/jira/browse/HDFS-1348
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: name-node
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.22.0
>
>
> NameNode normally is busy all the time. Its log is full of activities every
> second. But once for a while, NameNode seems to pause for more than 10
> seconds without doing anything, leaving a blank in its log even though no
> garbage collection is happening.
> One culprit is DecommionManager. Its monitor holds the fsynamesystem lock
> during the whole process of checking if decomissioning DataNodes are finished
> or not, during which it checks every block of up to a default of 5 datanodes.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
