[
https://issues.apache.org/jira/browse/YARN-4661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15186519#comment-15186519
]
Karthik Kambatla commented on YARN-4661:
----------------------------------------
Interesting thought. In the context of YARN-4752, I am attempting comprehensive
changes to preemption. As discussed in YARN-2154, we think it might best to
just preempt any container if the corresponding application is over its
fairshare. The primary goal here is to ensure the preempted container(s) are
all on one node and the resultant resources are actually enough to allocate the
container we are preempting these for.
In that context, I believe it would quite expensive to also consider a
preemption policy.
Do you agree? If yes, may be, we should close this as "Won't fix"?
> Per-queue preemption policy in FairScheduler
> --------------------------------------------
>
> Key: YARN-4661
> URL: https://issues.apache.org/jira/browse/YARN-4661
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: fairscheduler
> Affects Versions: 2.6.0
> Reporter: He Tianyi
> Priority: Minor
>
> When {{FairScheduler}} needs to preempt container, it tries to find a
> container by hierachically sorting and selecting {{AppSchedulable}} with most
> 'over fairshare' (in {{FairSharePolicy}}), and pick its latest launched
> container.
> In some case, strategy above become non-optimal, one may want to kill latest
> container (not {{AppSchedulable}}) launched in the queue for better trade-off
> between fairness and efficiency. Since most app with over fairshare tend to
> be started longer ago than other apps, perhaps even its latent launch
> container is running quite some time.
> Maybe besides {{policy}}, we make it possible to also specify a
> {{preemptionPolicy}} only for selecting container to preempt, without
> changing scheduling policy.
> For example:
> {quote}
> <queue name="queueA">
> <preemptionPolicy>fifo</preemptionPolicy>
> <policy>fair</policy>
> </queue>
> {quote}
> Any suggestions or comments?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
