[
https://issues.apache.org/jira/browse/YARN-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14708215#comment-14708215
]
Karthik Kambatla commented on YARN-2154:
----------------------------------------
bq. The previous ordering is better since if you happen to choose something
just above its fairShare, after preemption it may go below and cause additional
preemption, causing excessive thrashing of resources.
Has been a while since I saw the patch, but would think that it honors the
guarantees FairScheduler provides:
# Resources are preempted from a Schedulable, only if it is above its fairshare.
# Preempting resources from a Schedulable doesn't take it under its fairshare.
In other words, it shouldn't lead to additional preemption. It is possible that
its fairshare goes up right after a preemption and triggers additional
preemption, but I guess we can't foresee it.
Ordering shouldn't affect correctness of 1 or 2. Ordering does reduce the
impact of preemption in case the fairshare increases because another queue goes
inactive.
[~asuresh] - could you confirm the patch conforms to the above guarantees? Do
we have unittests that check these invariants? Once we identify all the apps we
could preempt from, can we order them for efficiency purposes? What would be
the cost of such ordering?
> FairScheduler: Improve preemption to preempt only those containers that would
> satisfy the incoming request
> ----------------------------------------------------------------------------------------------------------
>
> Key: YARN-2154
> URL: https://issues.apache.org/jira/browse/YARN-2154
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: fairscheduler
> Affects Versions: 2.4.0
> Reporter: Karthik Kambatla
> Assignee: Arun Suresh
> Priority: Critical
> Attachments: YARN-2154.1.patch
>
>
> Today, FairScheduler uses a spray-gun approach to preemption. Instead, it
> should only preempt resources that would satisfy the incoming request.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
