[
https://issues.apache.org/jira/browse/MAPREDUCE-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12780281#action_12780281
]
Matei Zaharia commented on MAPREDUCE-1198:
------------------------------------------
A couple of comments:
* I think initializing the HashSet outside the while loop and clearing it on
every for iteration is unnecessary optimization. I'd suggest initializing it in
the while loop instead (i.e. have it be a variable right above the for
(Schedulable sched: scheds) loop). Also, if you do this, you don't need to
clear the set in each iteration of the for loop, and you can do the
visitedForMap.addAll() after the for loop. It's okay if a job is added to the
HashSet twice - after all, it is a Set. The only thing we want to avoid is
having the same set used for maps and reduces.
* It would be better to call the mapAssigned and reduceAssigned variables
mapsAssigned and reducesAssigned instead, to make it clear that they're counts.
* In my opinion setting mapRejected / reduceRejected = true and then unsetting
it is more confusing than having an extra foundTask flag that is set to true
when we find a task. I'd prefer the latter, because I want the code to be as
easy as possible for a newcomer to understand. However, I don't have strong
feelings about this.
> Alternatively schedule different types of tasks in fair share scheduler
> -----------------------------------------------------------------------
>
> Key: MAPREDUCE-1198
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1198
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: contrib/fair-share
> Affects Versions: 0.22.0
> Reporter: Scott Chen
> Assignee: Scott Chen
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-1198-v1.patch, MAPREDUCE-1198-v2.patch,
> MAPREDUCE-1198-v3.patch, MAPREDUCE-1198-v4.patch
>
>
> Matei has mentioned in MAPREDUCE-961 that the current scheduler will first
> try to launch map tasks until canLaunthTask() returns false then look for
> reduce tasks. This might starve reduce task. He also mention that
> alternatively schedule different types of tasks can solve this problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
