PROTECTED]
Sent: Monday, December 31, 2007 1:27 PM
To: hadoop-dev@lucene.apache.org; Bryan Duxbury
Subject: Re: Question about HDFS allocations
AFAIK, hdfs doesn't have any notion of balancing data, nor can it do much to
avoid running disks full. What you describe would certainly be a useful
fe
There is also a script added but its not in a release yet its in trunk
start-balancer.sh
its in the bin folder
this is from the source code
* To start:
* bin/start-balancer.sh [-threshold ]
* Example: bin/ start-balancer.sh
* start the balancer with a default threshold of 10%
* bin/ start-balanc
maybe try adding these to your config and set them for your setup
dfs.datanode.du.reserved
0
Reserved space in bytes per volume. Always leave this much
space free for non dfs use.
dfs.datanode.du.pct
0.98f
When calculating remaining space, only use this percentage of
the real a
AFAIK, hdfs doesn't have any notion of balancing data, nor can it do much to
avoid running disks full. What you describe would certainly be a useful
feature.
There is a crude way to force the DFS to rebalance: if a machine gets too full,
you can remove it from the dfs cluster. The namenode wi
