[
https://issues.apache.org/jira/browse/YARN-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15188878#comment-15188878
]
Jian He commented on YARN-4108:
-------------------------------
For the new logic in PCPP, every round it’s looping all previously calculated
killable containers and check whether it’s finished, if not, deducting the
previously killable containers resource and then calculating the new extra set
of containers to kill. I think in a simpler way, we can re-generate the whole
new set of containers(as it already does) every round . Because in other
scenarios, previously marked killable containers will may become invalid too,
then we need to repeat similar logic. That is:
- PCPP does not need to track the killableConainers.
- once PCPP calculated the containers to kill, replace that synchronously in
PreemptionManager.
- when looping the queues, as done in the patch, getKillableResource and
getKillableContainers accordingly from PreemptionManager.
> CapacityScheduler: Improve preemption to preempt only those containers that
> would satisfy the incoming request
> --------------------------------------------------------------------------------------------------------------
>
> Key: YARN-4108
> URL: https://issues.apache.org/jira/browse/YARN-4108
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
> Reporter: Wangda Tan
> Assignee: Wangda Tan
> Attachments: YARN-4108-design-doc-V3.pdf,
> YARN-4108-design-doc-v1.pdf, YARN-4108-design-doc-v2.pdf, YARN-4108.1.patch,
> YARN-4108.2.patch, YARN-4108.3.patch, YARN-4108.4.patch, YARN-4108.5.patch,
> YARN-4108.6.patch, YARN-4108.poc.1.patch, YARN-4108.poc.2-WIP.patch,
> YARN-4108.poc.3-WIP.patch, YARN-4108.poc.4-WIP.patch
>
>
> This is sibling JIRA for YARN-2154. We should make sure container preemption
> is more effective.
> *Requirements:*:
> 1) Can handle case of user-limit preemption
> 2) Can handle case of resource placement requirements, such as: hard-locality
> (I only want to use rack-1) / node-constraints (YARN-3409) / black-list (I
> don't want to use rack1 and host\[1-3\])
> 3) Can handle preemption within a queue: cross user preemption (YARN-2113),
> cross applicaiton preemption (such as priority-based (YARN-1963) /
> fairness-based (YARN-3319)).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
