[
https://issues.apache.org/jira/browse/YARN-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15101143#comment-15101143
]
Wangda Tan commented on YARN-4108:
----------------------------------
Hi [~sunilg]
bq. 1. In current PCPP, we first preempt all reserved continers from all
applications in a queue if its overallocated.
I would prefer to following the existing PCPP logic: drop the container
reservation first and let scheduler continue to decide who can allocate/reserve
container on it.
bq. 2. If I understood correctly, "killable containers" will be triggered with
preeempt event only if a proper allocation can happen for target application
(from underserving queue).
For the preempt event sent to AM, my current thinking is, we will send it when
we add it to preempt list (not killable). AM could save state or return
resources. I understand this could lead to unnecessary
add-container-to-preempt-list event send to AM, but I think it's better than
excessive killing containers.
bq. 3. To cancel "killable container", i think PCPP will take the call by
waiting for some interval. So some new configuration is needed for this?
Maybe not, we don't need extra config for cancel killable container
bq. 4. I would like to have some freedom in selecting conatiners (marking) for
preemption. A simple sorting based on submission time or priority seems limited
approach. Could we have some interface here so that we can plugin user specific
comparision cases.
Agree, we can discuss this in a separated JIRA if we agree with the overall
approach (separate container selection and preemption). Will file a new JIRA
when we reach a consensus about the roadmap. [~mding]
> CapacityScheduler: Improve preemption to preempt only those containers that
> would satisfy the incoming request
> --------------------------------------------------------------------------------------------------------------
>
> Key: YARN-4108
> URL: https://issues.apache.org/jira/browse/YARN-4108
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
> Reporter: Wangda Tan
> Assignee: Wangda Tan
> Attachments: YARN-4108-design-doc-V3.pdf,
> YARN-4108-design-doc-v1.pdf, YARN-4108-design-doc-v2.pdf,
> YARN-4108.poc.1.patch, YARN-4108.poc.2-WIP.patch
>
>
> This is sibling JIRA for YARN-2154. We should make sure container preemption
> is more effective.
> *Requirements:*:
> 1) Can handle case of user-limit preemption
> 2) Can handle case of resource placement requirements, such as: hard-locality
> (I only want to use rack-1) / node-constraints (YARN-3409) / black-list (I
> don't want to use rack1 and host\[1-3\])
> 3) Can handle preemption within a queue: cross user preemption (YARN-2113),
> cross applicaiton preemption (such as priority-based (YARN-1963) /
> fairness-based (YARN-3319)).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
