caozhiqiang created HDFS-16613:
----------------------------------
Summary: EC: Improve performance of decommissioning dn with many
ec blocks
Key: HDFS-16613
URL: https://issues.apache.org/jira/browse/HDFS-16613
Project: Hadoop HDFS
Issue Type: Improvement
Components: ec, erasure-coding, namenode
Affects Versions: 3.4.0
Reporter: caozhiqiang
Assignee: caozhiqiang
In a hdfs cluster with a lot of EC blocks, decommission a dn is very slow. The
reason is unlike replication blocks can be replicated from any dn which has the
same block replication, the ec block have to be replicated from the
decommissioning dn. The configurations dfs.namenode.replication.max-streams and
dfs.namenode.replication.max-streams-hard-limit will limit the replication
speed, but increase these configurations will create risk to the whole
cluster's network. So it should add a new configuration to limit the
decommissioning dn, distinguished from the cluster wide max-streams limit.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
