Todd Lipcon wrote:
Hi Mayuran,
Do you do all of your uploads of data into your Hadoop cluster from node001
and node002?
If so, keep in mind that one of your replicas will always be written on
localhost in the case that it is part of the cluster.
You should consider running the rebalancer to even up your space usage.
-Todd
Actually yes I have been doing this. I'll try rebalancer, thanks for
your help.
M
On Tue, Aug 11, 2009 at 11:09 AM, Mayuran Yogarajah <
[email protected]> wrote:
I have a 6 node cluster running Hadoop 0.18.3. I'm trying to figure out
how the data was spread out like this:
node001 94.15%
node002 94.16%
node003 48.22%
node004 47.85%
node005 48.12%
node006 43.18%
Node 001 (NN) and node 002( secondary NN) both got full, while the other
data nodes had more space left. I had assumed that Hadoop would distribute
more blocks to nodes 3-6 since they had much more space, but it ended up
filling up nodes1 and 2. Is this expected?
thanks,
M
