In the pools with minimum slots, new job will always receive slots even if the
minimum slots limit has been fulfilled
---------------------------------------------------------------------------------------------------------------------
Key: MAPREDUCE-1361
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1361
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: contrib/fair-share
Affects Versions: 0.20.1
Reporter: Scott Chen
Assignee: Scott Chen
Fix For: 0.20.1
In 0.20, the fair scheduler compares all the jobs based on their running tasks,
minimum slots and deficit. If the number of running tasks is less than the
number of minimum slots, it will be scheduled first.
Consider a pool with minimum slot of 1000 but already have 5000 running tasks.
If we launch another job on this pool, this new job will receive minimum slots
based on its weight. This new job may have higher weight if NewJobWeightBooster
is used.
So this new job will still get extra slots even if the pool's running tasks are
way more than the minimum slots.
The latest version does not have this problem because it first compares pool
then compares jobs in the pool.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
