I did have these settings on all the hdfs-site.xml nodes: <property> <!-- 100Mbit/s --> <name>dfs.balance.bandwidthPerSec</name> <value>131072000</value> </property> <property> <name>dfs.max-repl-streams</name> <value>50</value> </property>
It is still taking over 1 day or longer for 1TB of under replicated blocks to replicate. Thanks! Jonathan -----Original Message----- From: Joey Echeverria [mailto:[email protected]] Sent: Friday, August 12, 2011 9:14 AM To: [email protected] Subject: Re: Speed up node under replicated block during decomission You can configure the undocumented variable dfs.max-repl-streams to increase the number of replications a data-node is allowed to handle at one time. The default value is 2. [1] -Joey [1] https://issues.apache.org/jira/browse/HADOOP-2606?focusedCommentId=12578700&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-12578700 On Fri, Aug 12, 2011 at 12:09 PM, Charles Wimmer <[email protected]> wrote: > The balancer bandwidth setting does not affect decommissioning nodes. > Decommisssioning nodes replicate as fast as the cluster is capable. > > The replication pace has many variables. > Number nodes that are participating in the replication. > The amount of network bandwidth each has. > The amount of other HDFS activity at the time. > Total blocks being replicated. > Total data being replicated. > Many others. > > > On 8/12/11 8:58 AM, "[email protected]" > <[email protected]> wrote: > > Hi All, > > I'm trying to decommission data node from my cluster. I put the data node in > the /usr/lib/hadoop/conf/dfs.hosts.exclude list and restarted the name nodes. > The under-replicated blocks are starting to replicate, but it's going down > in a very slow pace. For 1 TB of data it takes over 1 day to complete. We > change the settings as below and try to increase the replication rate. > > Added this to hdfs-site.xml on all the nodes on the cluster and restarted the > data nodes and name node processes. > <property> > <!-- 100Mbit/s --> > <name>dfs.balance.bandwidthPerSec</name> > <value>131072000</value> > </property> > > Speed didn't seem to pick up. Do you know what may be happening? > > Thanks! > Jonathan > > This message is for the designated recipient only and may contain privileged, > proprietary, or otherwise private information. If you have received it in > error, please notify the sender immediately and delete the original. Any > other use of the email by you is prohibited. > > -- Joseph Echeverria Cloudera, Inc. 443.305.9434
