[
https://issues.apache.org/jira/browse/HDFS-2537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13145621#comment-13145621
]
Aaron T. Myers commented on HDFS-2537:
--------------------------------------
I agree with all of this, Nathan. I don't have time to work on this right now,
but will be happy to help with reviews if someone else does have time to devote
to this issue.
> re-replicating under replicated blocks should be more dynamic
> -------------------------------------------------------------
>
> Key: HDFS-2537
> URL: https://issues.apache.org/jira/browse/HDFS-2537
> Project: Hadoop HDFS
> Issue Type: Improvement
> Affects Versions: 0.20.205.0, 0.23.0
> Reporter: Nathan Roberts
>
> When a node fails or is decommissioned, a large number of blocks become
> under-replicated. Since re-replication work is distributed, the hope would be
> that all blocks could be restored to their desired replication factor in very
> short order. This doesn't happen though because the load the cluster is
> willing to devote to this activity is mostly static (controlled by
> configuration variables). Since it's mostly static, the rate has to be set
> conservatively to avoid overloading the cluster with replication work.
> This problem is especially noticeable when you have lots of small blocks. It
> can take many hours to re-replicate the blocks that were on a node while the
> cluster is mostly idle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
