[
https://issues.apache.org/jira/browse/MAPREDUCE-517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun C Murthy updated MAPREDUCE-517:
------------------------------------
Attachment: MAPREDUCE-517_yhadoop20.patch
Updated patch for y20.
Highlights:
# CS assigns multiple tasks per heartbeat, at most 1 off-switch task per
heartbeat.
# I've incorporated HADOOP-538 also to ensure jobs at the head of the queue do
not aggressively grab tasks hurting locality for others.
## The implementation tracks 'number of scheduling opportunities' missed by a
job and gets jobs to use that to prevent starvation.
## I've also added 'pace' to the back-off by getting ensuring jobs do not
back-off as aggressively as they make progress.
## The patch also gets small jobs to back off less vis-a-vis larger jobs by
ensuring the backoff considers the #maps in the jobs.
## The patch also ensures jobs with no locality e.g. sleep-job/randomwriter do
not care about backoff since it doesn't make sense at all.
> The capacity-scheduler should assign multiple tasks per heartbeat
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-517
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-517
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Arun C Murthy
> Assignee: Arun C Murthy
> Priority: Critical
> Attachments: HADOOP-5090-20090504.txt, HADOOP-5090-20090506.txt,
> HADOOP-5090-20090604.txt, MAPREDUCE-517_yhaddop20.patch,
> MAPREDUCE-517_yhadoop20.patch
>
>
> HADOOP-3136 changed the default o.a.h.mapred.JobQueueTaskScheduler to assign
> multiple tasks per TaskTracker heartbeat, the capacity-scheduler should do
> the same.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
