Fengnan Li created HDFS-15634:
---------------------------------
Summary: Invalidate block on decommissioning DataNode after
replication
Key: HDFS-15634
URL: https://issues.apache.org/jira/browse/HDFS-15634
Project: Hadoop HDFS
Issue Type: Improvement
Components: hdfs
Reporter: Fengnan Li
Assignee: Fengnan Li
Right now when a DataNode starts decommission, Namenode will mark it as
decommissioning and its blocks will be replicated over to different DataNodes,
then marked as decommissioned. These blocks are not touched since they are not
counted as live replicas.
Proposal: Invalidate these blocks once they are replicated and there are enough
live replicas in the cluster.
Reason: A recent shutdown of decommissioned datanodes to finished the flow
caused Namenode latency spike since namenode needs to remove all of the blocks
from its memory and this step requires holding write lock. If we have gradually
invalidated these blocks the deletion will be much easier and faster.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
