[
https://issues.apache.org/jira/browse/HDFS-14133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liaoyuxiangqin updated HDFS-14133:
----------------------------------
Description:
When i testing blocks reconstruction triger by data disk offline, i found
that the Replicated blocks reconstruction speed limit by
{color:#FF0000}dfs.namenode.replication.work.multiplier.per.iteration{color}
in namenode compute work, and for a given node the maximum number of outgoing
replication streams at one time limit by used
{color:#FF0000}dfs.namenode.replication.max-streams{color} and
{color:#FF0000}dfs.namenode.replication.max-streams-hard-limit{color}, but for
Erasure Coded blocks only limit total blocks to reconstruct one time by
{color:#FF0000}dfs.namenode.replication.work.multiplier.per.iteration{color}
but not limit maximum of outgoing replication streams per node, so that maybe
cause node load imbalance, i think also ec need add node limit.
> Add limit of replication streams on a given node at one time for erasurecode
> blocks resconstruction
> ----------------------------------------------------------------------------------------------------
>
> Key: HDFS-14133
> URL: https://issues.apache.org/jira/browse/HDFS-14133
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: erasure-coding
> Affects Versions: 3.3.0
> Reporter: liaoyuxiangqin
> Priority: Major
>
> When i testing blocks reconstruction triger by data disk offline, i found
> that the Replicated blocks reconstruction speed limit by
> {color:#FF0000}dfs.namenode.replication.work.multiplier.per.iteration{color}
> in namenode compute work, and for a given node the maximum number of outgoing
> replication streams at one time limit by used
> {color:#FF0000}dfs.namenode.replication.max-streams{color} and
> {color:#FF0000}dfs.namenode.replication.max-streams-hard-limit{color}, but
> for Erasure Coded blocks only limit total blocks to reconstruct one time by
> {color:#FF0000}dfs.namenode.replication.work.multiplier.per.iteration{color}
> but not limit maximum of outgoing replication streams per node, so that maybe
> cause node load imbalance, i think also ec need add node limit.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
