CapacityScheduler has a feature called 'High RAM Jobs' where-in you
can specify, for a given job, that a single map/reduce task needs more
than 1 slot. Thus you could consume all the map/reduce slots on a
given TT for a single task of your job. This should suffice.
Arun
On Jun 30, 2010, at 5:09 AM, Pierre ANCELOT wrote:
Sure, but not the number of tasks running concurrently on a node at
the same
time.
On Wed, Jun 30, 2010 at 1:57 PM, Ted Yu <[email protected]> wrote:
The number of map tasks is determined by InputSplit.
On Wednesday, June 30, 2010, Pierre ANCELOT <[email protected]>
wrote:
Hi,
Okay, so, if I set the 20 by default, I could maybe limit the
number of
concurrent maps per node instead?
job.setNumReduceTasks exists but I see no equivalent for maps,
though I
think there was a setNumMapTasks before...
Was it removed? Why?
Any idea about how to acheive this?
Thank you.
On Wed, Jun 30, 2010 at 12:08 PM, Amareshwari Sri Ramadasu <
[email protected]> wrote:
Hi Pierre,
"mapred.tasktracker.map.tasks.maximum" is a cluster level
configuration,
cannot be set per job. It is loaded only while bringing up the
TaskTracker.
Thanks
Amareshwari
On 6/30/10 3:05 PM, "Pierre ANCELOT" <[email protected]> wrote:
Hi everyone :)
There's something I'm probably doing wrong but I can't seem to
figure
out
what.
I have two hadoop programs running one after the other.
This is done because they don't have the same needs in term of
processor
in
memory, so by separating them I optimize each task better.
Fact is, I need for the first job on every node
mapred.tasktracker.map.tasks.maximum set to 12.
For the second task, I need it to be set to 20.
so by default I set it to 12 and in the second job's code, I set
this:
Configuration hadoopConfiguration = new Configuration();
hadoopConfiguration.setInt("mapred.tasktracker.map.tasks.maximum",
20);
But when running the job, instead of having the 20 tasks on each
node as
expected, I have 12....
Any idea please?
Thank you.
Pierre.
--
http://www.neko-consulting.com
Ego sum quis ego servo
"Je suis ce que je protège"
"I am what I protect"
--
http://www.neko-consulting.com
Ego sum quis ego servo
"Je suis ce que je protège"
"I am what I protect"
--
http://www.neko-consulting.com
Ego sum quis ego servo
"Je suis ce que je protège"
"I am what I protect"
