[
https://issues.apache.org/jira/browse/MAPREDUCE-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12780178#action_12780178
]
Matei Zaharia commented on MAPREDUCE-1198:
------------------------------------------
I've looked at the patch in more detail. It mostly looks good, but I have a
question: When no task is found in the loop over Schedulables (for (Schedulable
sched: scheds)), why do you still increase the mapAssigningCount or
reduceAssigningCount after the loop? My guess is that you're increasing it so
that it reaches its cap and we stop trying to assign that type of task.
However, it would be clearer if you just set mapRejected or reduceRejected to
true when you don't find a task of that type. You could add a boolean variable
outside the loop called foundTask, initialize it to false, and set it to true
when you do find a task (before the break). Then, after the loop, if findTask
== false, set mapRejected or reduceRejected to true.
Also, a couple of smaller comments:
* Instead of the names mapAssigningCount and reduceAssigningCount, you can use
mapsAssigned and reducesAssigned.
* In the inner loop, the launchedMap.add(job); statement that is inside an if
(taskType == TaskType.MAP) could be moved below to the MAP case of the "update
running task counts" statement.
* "to prevent the type from starving" -> "to prevent that task type from
starving"
* Capitalize the this in "// this is the only exit of the while (true) loop"
> Alternatively schedule different types of tasks in fair share scheduler
> -----------------------------------------------------------------------
>
> Key: MAPREDUCE-1198
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1198
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: contrib/fair-share
> Affects Versions: 0.22.0
> Reporter: Scott Chen
> Assignee: Scott Chen
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-1198-v1.patch, MAPREDUCE-1198-v2.patch,
> MAPREDUCE-1198-v3.patch
>
>
> Matei has mentioned in MAPREDUCE-961 that the current scheduler will first
> try to launch map tasks until canLaunthTask() returns false then look for
> reduce tasks. This might starve reduce task. He also mention that
> alternatively schedule different types of tasks can solve this problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
