[
https://issues.apache.org/jira/browse/YARN-596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei Yan updated YARN-596:
-------------------------
Attachment: YARN-596.patch
Thanks, [~sandyr].
Attached a new patch:
1. remove the app map and queue map.
2. Rename FSPreemption to FairSchedulerPreemptor.
For the cloneQueueApps() function: we reuse the comparator/policy (used by the
assignContainer()) to sort queues/apps, and always select the one that is most
over its fairshare. The comparator needs to call getResourceUsage(), which is
calculated as sum_of_live_containers. However, for preemption, the
resourceUsage should be (sum_of_live_containers -
sum_of_toBePreempted_containers). Instead of defining a new comparator, I
snapshot the resourceusage to a TempScheduler structure, which overrides the
getResourceUsage() function.
> In fair scheduler, intra-application container priorities affect
> inter-application preemption decisions
> -------------------------------------------------------------------------------------------------------
>
> Key: YARN-596
> URL: https://issues.apache.org/jira/browse/YARN-596
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler
> Affects Versions: 2.0.3-alpha
> Reporter: Sandy Ryza
> Assignee: Sandy Ryza
> Attachments: YARN-596.patch, YARN-596.patch, YARN-596.patch,
> YARN-596.patch
>
>
> In the fair scheduler, containers are chosen for preemption in the following
> way:
> All containers for all apps that are in queues that are over their fair share
> are put in a list.
> The list is sorted in order of the priority that the container was requested
> in.
> This means that an application can shield itself from preemption by
> requesting it's containers at higher priorities, which doesn't really make
> sense.
> Also, an application that is not over its fair share, but that is in a queue
> that is over it's fair share is just as likely to have containers preempted
> as an application that is over its fair share.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
