[ https://issues.apache.org/jira/browse/HDFS-12482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lei (Eddy) Xu updated HDFS-12482: --------------------------------- Attachment: HDFS-12482.01.patch Thanks for the reviews, [~andrew.wang] Updated the patch to add documents. Empirically, an value between {{(0, 1.0]}} seems can achieve similar recovery speed and network / cpu overhead between replicated blocks and ec block recovery _on my testing cluster_. But it highly depends on HW. I will set this value to {{0.5}} initially in this patch. > Provide a configuration to adjust the weight of EC recovery tasks to adjust > the speed of recovery > ------------------------------------------------------------------------------------------------- > > Key: HDFS-12482 > URL: https://issues.apache.org/jira/browse/HDFS-12482 > Project: Hadoop HDFS > Issue Type: Improvement > Components: erasure-coding > Affects Versions: 3.0.0-alpha4 > Reporter: Lei (Eddy) Xu > Assignee: Lei (Eddy) Xu > Priority: Minor > Labels: hdfs-ec-3.0-nice-to-have > Attachments: HDFS-12482.00.patch, HDFS-12482.01.patch > > > The relative speed of EC recovery comparing to 3x replica recovery is a > function of (EC codec, number of sources, NIC speed, and CPU speed, and etc). > Currently the EC recovery has a fixed {{xmitsInProgress}} of {{max(# of > sources, # of targets)}} comparing to {{1}} for 3x replica recovery, and NN > uses {{xmitsInProgress}} to decide how much recovery tasks to schedule to the > DataNode this we can add a coefficient for user to tune the weight of EC > recovery tasks. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org