[
https://issues.apache.org/jira/browse/YARN-596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948574#comment-13948574
]
Sandy Ryza commented on YARN-596:
---------------------------------
Thanks for working on this, Wei.
{code}
+ // Collect running containers from over-scheduled queues
+ Map<RMContainer, FSSchedulerApp> apps =
+ new HashMap<RMContainer, FSSchedulerApp>();
+ Map<RMContainer, FSLeafQueue> queues =
+ new HashMap<RMContainer, FSLeafQueue>();
+ for (FSLeafQueue sched : scheduler.getQueueManager().getLeafQueues()) {
+ if (Resources.greaterThan(RESOURCE_CALCULATOR, clusterCapacity,
+ sched.getResourceUsage(), sched.getFairShare())) {
+ for (AppSchedulable as : sched.getRunnableAppSchedulables()) {
+ for (RMContainer c : as.getApp().getLiveContainers()) {
+ apps.put(c, as.getApp());
+ queues.put(c, sched);
+ }
+ }
+ }
+ }
{code}
Do we still need this anymore? If we're not collecting the running containers
here, it seems a little weird to build up the container->app/queue maps here.
We should be able to look up the application attempt and corresponding queue
with data that the RMContainer has, using Fair Scheduler data structures. Or
return them with preemptContainer.
Can we rename FSPreemption to FairSchedulerPreemptor or just Preemptor?
What's the reasoning for cloneQueueApps and not just traversing the existing
queues?
> In fair scheduler, intra-application container priorities affect
> inter-application preemption decisions
> -------------------------------------------------------------------------------------------------------
>
> Key: YARN-596
> URL: https://issues.apache.org/jira/browse/YARN-596
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler
> Affects Versions: 2.0.3-alpha
> Reporter: Sandy Ryza
> Assignee: Sandy Ryza
> Attachments: YARN-596.patch, YARN-596.patch, YARN-596.patch
>
>
> In the fair scheduler, containers are chosen for preemption in the following
> way:
> All containers for all apps that are in queues that are over their fair share
> are put in a list.
> The list is sorted in order of the priority that the container was requested
> in.
> This means that an application can shield itself from preemption by
> requesting it's containers at higher priorities, which doesn't really make
> sense.
> Also, an application that is not over its fair share, but that is in a queue
> that is over it's fair share is just as likely to have containers preempted
> as an application that is over its fair share.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
