[
https://issues.apache.org/jira/browse/MAPREDUCE-1379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12800627#action_12800627
]
Bochun Bai commented on MAPREDUCE-1379:
---------------------------------------
Arun, answering your first question "why do need this new config".
Limit separately is minimal 2 tasks running concurrently, 1 mapper and 1
reducer. I have seen some questions in the maillist asked how to limit them
both. I think he must face the same problem I did.
Some of my servers are shared with other service. I use map=1 reduce=1 total=1.
And these 2 situations made me this patch.
Situation A:
Most jobs have thousand of map tasks and only a few reducer.
Situation B:
But some of the tasks set the number of reduce tasks large.
Solve A: For 8-core CPU, m=7,r=1 seted
Solve B: For 8-core CPU, m=8,r=8,t=9 seted. to avoid high loadavg.
I have to config both of them, so a total config is needed.
> Limit both numMapTasks and numReduceTasks
> -----------------------------------------
>
> Key: MAPREDUCE-1379
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1379
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: tasktracker
> Affects Versions: 0.22.0
> Reporter: Bochun Bai
> Priority: Trivial
> Attachments: limit-both-numMapTasks-and-numReduceTasks.patch
>
>
> In some environment, the number of concurrent running process is very
> sensitive.
> mapreduce.tasktracker.map.tasks.maximum
> and
> mapreduce.tasktracker.reduce.tasks.maximum
> limit tasks running on each tasktracker separately.
> This patch limits them together, using
> mapreduce.tasktracker.total.tasks.maximum
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
