[
https://issues.apache.org/jira/browse/HDFS-4075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13481587#comment-13481587
]
Ravi Prakash commented on HDFS-4075:
------------------------------------
Hi Kihwal!
The log message is
bq. LOG.info("Invalidated" + numOverReplicated + " over-replicated blocks on " +
bq. srcNode + " during recommissioning");
which might mislead me to believe that the block invalidated was on srcNode,
when it could be any one of the 4 nodes. Maybe something to the effect
"Recommissioning of srcNode led to numOverReplicated over-replicated blocks to
be invalidated"?
Can you please also explain the change in DatanodeManager.java in this patch?
node.isAlive will be updated only when the node heartbeats in. So when will
blockManager.processOverReplicatedBlocksOnReCommission(node);
be called?
> Reduce recommissioning overhead
> -------------------------------
>
> Key: HDFS-4075
> URL: https://issues.apache.org/jira/browse/HDFS-4075
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 0.23.4, 2.0.2-alpha
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Critical
> Attachments: hdfs-4075.patch
>
>
> When datanodes are recommissioned,
> {BlockManager#processOverReplicatedBlocksOnReCommission()} is called for each
> rejoined node and excess blocks are added to the invalidate list. The problem
> is this is done while the namesystem write lock is held.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
