BELUGA BEHR created HDFS-13157:
----------------------------------
Summary: Do Not Remove Blocks Sequentially During Dec omission
Key: HDFS-13157
URL: https://issues.apache.org/jira/browse/HDFS-13157
Project: Hadoop HDFS
Issue Type: Improvement
Components: datanode, namenode
Affects Versions: 3.0.0
Reporter: BELUGA BEHR
>From what I understand of [DataNode
>decommissioning|https://github.com/apache/hadoop/blob/42a1c98597e6dba2e371510a6b2b6b1fb94e4090/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeAdminManager.java]
> it appears that all the blocks are scheduled for removal _in order._. I'm
>not 100% sure what the ordering is exactly, but I think it's loops through
>each data volume and schedules each block to be replicated elsewhere. The net
>affect is that during a decommission, all of the DataNode transfer threads
>slam on a single volume until it is cleaned out. At which point, they all
>slam on the next volume, etc.
Please randomize the block list so that there is a more even distribution
across all volumes when decommissioning a node.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
