Hello,
Amandeep Khurana wrote:
So you are running 16 map tasks per node? Plus 2 reducers?
Thats correct.
I think that's high. With 6gb RAM, you should be looking at around 2
map tasks plus 1 reducer...
I have 9 nodes with quad core + 8gb RAM and I run 2M+1R on each node..
I thought the number of maps should be set to 1/2 - 2 * number of cpus,
thats why
we set it so high. Right now I've set:
mapred.tasktracker.map.tasks.maximum = 16
mapred.tasktracker.reduce.tasks.maximum = 16
So the max mappers/reducers is 96/96.
How much heap size have you given your hadoop instance?
Also, is there a lot of processing going on in the mappers and reducers?
Yes these are pretty intensive jobs.
thanks,
M
