Hi all,
I am using a HTable as input to my map jobs and my reducer outputs to
another Htable. There are 10 regions of my input HTable. And I have set
conf.set("mapred.tasktracker.map.tasks.maximum", "2");
conf.set("mapred.tasktracker.map.tasks.maximum", "2");
c.setNumReduceTasks(26);
My cluster contains 15 nodes(out of which 2 are maters). When I run my job,
only 2 map tasks run at a time and the remaining 8 are shown as pending. 24
reduce tasks(out of 26) also get started initially and remaing 2 are shown
as pending. I am confused why only 2 tasks are running at a time, though
there are a total of 26 slots for map tasks.
However, this does not happen when I run jobs in which I take files as
inputs(i.e. only simple MapReduce jobs and not involving HBase at all). Only
when a Htable is taken as input very few map tasks run concurrently than
expected.
Can anyone suggest why this is happening?
Thanks,
Akhil
--
View this message in context:
http://www.nabble.com/Why-only-few-map-tasks-are-running-at-a-time-inspite-of-plenty-of-scope-for-remaining--tp24636333p24636333.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.
