On Tue, 18 Mar 2008 19:53:04 -0500, Ted Dunning <[EMAIL PROTECTED]> wrote:
I think the original request was to limit the sum of maps and reduces
rather than limiting the two parameters independently.
Ted, yes this is exactly what I'm looking for. I just found an issue that
seems to state that the old deprecated property is there, but it is not
documented:
https://issues.apache.org/jira/browse/HADOOP-2300
I tried using the max tasks in combination with setting the new values,
but that didn't seem to work. =( My machine labelled as "LIMITED MACHINE"
had 2 maps and 1 reduce running at the same time.
The scenario I have is that I want to run multiple concurrent jobs through
my cluster and have the CPU usage for that node be bound. Should I file a
new issue?
This was all with Hadoop 0.16.0
LIMITED MACHINE:
<property>
<name>mapred.tasktracker.tasks.maximum</name>
<value>2</value>
<description>The maximum number of total tasks that will be run
simultaneously by a task tracker.
</description>
</property>
<property>
<name>mapred.tasktracker.map.tasks.maximum</name>
<value>1</value>
<description>The maximum number of map tasks that will be run
simultaneously by a task tracker.
</description>
</property>
<property>
<name>mapred.tasktracker.reduce.tasks.maximum</name>
<value>1</value>
<description>The maximum number of reduce tasks that will be run
simultaneously by a task tracker.
</description>
</property>
OTHER CLUSTER MACHINES:
<property>
<name>mapred.tasktracker.tasks.maximum</name>
<value>8</value>
<description>The maximum number of total tasks that will be run
simultaneously by a task tracker.
</description>
</property>
<property>
<name>mapred.tasktracker.map.tasks.maximum</name>
<value>4</value>
<description>The maximum number of map tasks that will be run
simultaneously by a task tracker.
</description>
</property>
<property>
<name>mapred.tasktracker.reduce.tasks.maximum</name>
<value>4</value>
<description>The maximum number of reduce tasks that will be run
simultaneously by a task tracker.
</description>
</property>
On 3/18/08 5:26 PM, "Arun C Murthy" <[EMAIL PROTECTED]> wrote:
The map/reduce tasks are not threads, they are run in separate JVMs
which are forked by the tasktracker.
Arun, yes, I did mean tasks, not threads.
--
Jimmy
