[
https://issues.apache.org/jira/browse/YARN-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15127182#comment-15127182
]
Eric Payne commented on YARN-4108:
----------------------------------
Thanks [~leftnoteasy] for your work on the new patch, {{YARN-4108.2.patch}}.
{quote}
Currently, if both queueA and queueB are over their guaranteed capacity,
preemption will still occur if queueA is more over capacity than queueB. I
think it is probably important to preserve this behavior (YARN-2592).
{quote}
I'm still concerned that this patch will change the above-mentioned behavior.
In {{LeafQueue#setPreemptionAllowed}}, I see the following:
{code}
limits.setIsAllowPreemption(usedCapacity < guaranteedCapacity);
{code}
In the current preemption paradigm, the first queue queue can cause containers
to be preempted from second queue even if the first queue is using more than
its guaranteed capacity, if the second queue is more over capacity than the
first. The above code seems to prevent that.
> CapacityScheduler: Improve preemption to preempt only those containers that
> would satisfy the incoming request
> --------------------------------------------------------------------------------------------------------------
>
> Key: YARN-4108
> URL: https://issues.apache.org/jira/browse/YARN-4108
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
> Reporter: Wangda Tan
> Assignee: Wangda Tan
> Attachments: YARN-4108-design-doc-V3.pdf,
> YARN-4108-design-doc-v1.pdf, YARN-4108-design-doc-v2.pdf, YARN-4108.1.patch,
> YARN-4108.2.patch, YARN-4108.poc.1.patch, YARN-4108.poc.2-WIP.patch,
> YARN-4108.poc.3-WIP.patch, YARN-4108.poc.4-WIP.patch
>
>
> This is sibling JIRA for YARN-2154. We should make sure container preemption
> is more effective.
> *Requirements:*:
> 1) Can handle case of user-limit preemption
> 2) Can handle case of resource placement requirements, such as: hard-locality
> (I only want to use rack-1) / node-constraints (YARN-3409) / black-list (I
> don't want to use rack1 and host\[1-3\])
> 3) Can handle preemption within a queue: cross user preemption (YARN-2113),
> cross applicaiton preemption (such as priority-based (YARN-1963) /
> fairness-based (YARN-3319)).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
