Re: NN + secondary got full, even though data nodes had plenty of space

[Mayuran Yogarajah]

Amandeep Khurana wrote:

Is your NN doubling up as a DN? If its not, I wonder how the NN is full...

Yes both NN and secondary are doubling as DN, sorry I should have mentioned
this earlier.

M

On 8/11/09, Mayuran Yogarajah <mayuran.yogara...@casalemedia.com> wrote:

I have a 6 node cluster running Hadoop 0.18.3.  I'm trying to figure out
how the data was spread out like this:

node001         94.15%
node002         94.16%
node003         48.22%
node004         47.85%
node005         48.12%
node006         43.18%

Node 001 (NN) and node 002( secondary NN) both got full, while the other
data nodes had more space left.  I had assumed that Hadoop would distribute
more blocks to nodes 3-6 since they had much more space, but it ended up
filling up nodes1 and 2.  Is this expected?

thanks,
M

--


Amandeep Khurana
Computer Science Graduate Student
University of California, Santa Cruz