[
https://issues.apache.org/jira/browse/YARN-2009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15609597#comment-15609597
]
Wangda Tan commented on YARN-2009:
----------------------------------
[~eepayne], [~sunilg].
I'm not sure if we need to consider free resource for a queue.
There're two scenarios we need to cover
1) One queue's used < configured, and cluster is full
2) One queue's used < configured, and cluster has enough free resource
For 1), we should do something for the queue to make sure it can balance, if
inter-queue preemption happens at the same time, less intra-queue preemption
happens
For 2), it is possible that we mark some resource to preemption candidate, but
before these resource preempted, the queue will get more resource and
previously added candidates will be cancelled.
And also, the whole point of intra-queue preemption is to make sure queue can
balance itself. We have configs for intra-queue preemption
minimum-threshold/max-allowable-limit to make sure we will
1) Preempt queue's resource when it is not beyond a threshold
2) Do not preempt too much resource
In addition, inter-queue preemption happens first before intra-queue preemption.
With all the above safeguards, existing logics looks safe since we won't kill
containers immediately after they get selected. I'm a little concern that
considering the (configured - used) resource of queue could cause sometimes
intra-queue preemption cannot be triggered.
> Priority support for preemption in ProportionalCapacityPreemptionPolicy
> -----------------------------------------------------------------------
>
> Key: YARN-2009
> URL: https://issues.apache.org/jira/browse/YARN-2009
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacityscheduler
> Reporter: Devaraj K
> Assignee: Sunil G
> Attachments: YARN-2009.0001.patch, YARN-2009.0002.patch,
> YARN-2009.0003.patch, YARN-2009.0004.patch, YARN-2009.0005.patch,
> YARN-2009.0006.patch, YARN-2009.0007.patch, YARN-2009.0008.patch,
> YARN-2009.0009.patch, YARN-2009.0010.patch, YARN-2009.0011.patch,
> YARN-2009.0012.patch, YARN-2009.0013.patch, YARN-2009.0014.patch,
> YARN-2009.0015.patch
>
>
> While preempting containers based on the queue ideal assignment, we may need
> to consider preempting the low priority application containers first.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
