Answering my own question here. Make sure your dfs.balance.bandwidthPerSec is the same on all of your datanodes, or else, you will observe the "funkiness".
--jake On 12/21/09 3:41 PM, Jacob Olsen wrote: > Answering my own questions here: > > But, after a few stops/starts and WARNS: > > > On 12/21/09 2:48 PM, Jacob Olsen wrote: >> And indeed, I verify only ~1G of data has been moved. Any ideas? >> >> --jake >> >> On 12/21/09 2:18 PM, Jacob Olsen wrote: >>> When I run the balancer, it says it's moving 10G, but after those 10G are >>> moved, it's reported that only ~ 1G is moved. >>> >>> What could be causing this discrepancy? >>> >>> Thanks so much in advance for your help. >>> >>> --jake >>> >>> 2009-12-21 21:53:25,967 INFO >>> org.apache.hadoop.hdfs.server.balancer.Balancer: 1 over utilized nodes: >>> 172.18.141.95:50010 >>> 2009-12-21 21:53:25,967 INFO >>> org.apache.hadoop.hdfs.server.balancer.Balancer: 0 under utilized nodes: >>> 2009-12-21 21:53:25,967 INFO >>> org.apache.hadoop.hdfs.server.balancer.Balancer: Need to move 292.74 GB >>> bytes to make the cluster balanced. >>> 2009-12-21 21:53:25,968 INFO >>> org.apache.hadoop.hdfs.server.balancer.Balancer: Decided to move 10 GB >>> bytes from 172.18.141.95:50010 to 172.18.141.98:50010 >>> 2009-12-21 21:53:25,968 INFO >>> org.apache.hadoop.hdfs.server.balancer.Balancer: Will move 10 GBbytes in >>> this iteration >>> >>> ..... moving blocks ..... >>> >>> 2009-12-21 22:15:56,249 INFO >>> org.apache.hadoop.hdfs.server.balancer.Balancer: 1 over utilized nodes: >>> 172.18.141.95:50010 >>> 2009-12-21 22:15:56,249 INFO >>> org.apache.hadoop.hdfs.server.balancer.Balancer: 0 under utilized nodes: >>> 2009-12-21 22:15:56,249 INFO >>> org.apache.hadoop.hdfs.server.balancer.Balancer: Need to move 291.4 GB >>> bytes to make the cluster balanced. >>> 2009-12-21 22:15:56,249 INFO >>> org.apache.hadoop.hdfs.server.balancer.Balancer: Decided to move 10 GB >>> bytes from 172.18.141.95:50010 to 172.18.141.96:50010 >>> 2009-12-21 22:15:56,249 INFO >>> org.apache.hadoop.hdfs.server.balancer.Balancer: Will move 10 GBbytes in >>> this iteration >>> >>
