[
https://issues.apache.org/jira/browse/MAPREDUCE-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12860873#action_12860873
]
Hemanth Yamijala commented on MAPREDUCE-1723:
---------------------------------------------
Having pre-emption (assuming we are able to get it right, which I have no doubt
is going to be incredibly hard), will solve a lot of other use cases that
people have given feedback about the capacity scheduler in the past. And it
will solve the basic problem faced due to the particular setup described by
Subramaniam in the description. That's why I brought it up.
Just to be clear, I am not proposing we solve this problem using pre-emption.
Only listing it as an option to consider. If we all shudder and reject it for
its complexity, it's completely fine.
> Capacity Scheduler should allow configuration of Map & Reduce task slots
> independently per queue
> ------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1723
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1723
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: contrib/capacity-sched
> Affects Versions: 0.20.1
> Environment: all
> Reporter: Subramaniam Krishnan
> Fix For: 0.20.3
>
>
> The Capacity Scheduler allows configuration of percentage of task slots per
> queue. We have a scenario in which our biggest queue (50% quota) has Jobs
> with mainly Map tasks & we need to enforce strict capacity limits per queue
> due to SLA requirements. So other smaller queues which require Reduce tasks
> gets starved even though the Reduce slots are idle. The Grid can be more
> efficiently utilized if Capacity Scheduler allows configuration of Map &
> Reduce task slots capacity independently per queue.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
