[
https://issues.apache.org/jira/browse/YARN-5830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15784024#comment-15784024
]
Yufei Gu edited comment on YARN-5830 at 12/28/16 11:58 PM:
-----------------------------------------------------------
[~kasha], thanks for the review.
The high-level approach:
1. In the first node, sort the containersToCheck list by putting all non-AM
containers first.
2. We check containers one by one. If we are lucky to found a solution without
any AM container, just return the container list. But if we are not so lucky
and found a solution with AM container, we record this solution in
{{potentialContainers}} and {{break}} here to move to next node because we may
find an solution without AM container in next node.
3. Repeat step 1 and 2 for each node, the only difference is if we've already
got a solution with AM container, we can {{break}} the loop once we meet first
AM container.
4. Return the {{potentialContainers}} if we cannot find any solution without AM
containers.
This approach, I was trying to avoid sort AM and non-AM containers across nodes
since it may be super inefficient if # of node are huge. One of my concern is
that we should check many nodes if there are many nodes with AM containers,
which may be not efficient. We can put a threshold to avoid it.
was (Author: yufeigu):
[~kasha], thanks for the review.
The high-level approach:
1. In first nodes, sort the containersToCheck list by putting all non-AM
containers first.
2. We check containers one by one. If we are lucky to found a solution without
any AM container, just return the container list. But if we are not so lucky
and found a solution with AM container, we record this solution in
{{potentialContainers}} and {{break}} here to move to next node because we may
find an solution without AM container in next node.
3. Repeat step 1 and 2 for each node, the only difference is if we've already
got a solution with AM container, we can {{break}} the loop once we meet first
AM container.
4. Return the {{potentialContainers}} if we cannot find any solution without AM
containers.
One of my concern is that we should check many nodes if there are many nodes
with AM containers, which may be not efficient. We can put a threshold to avoid
it.
> Avoid preempting AM containers
> ------------------------------
>
> Key: YARN-5830
> URL: https://issues.apache.org/jira/browse/YARN-5830
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: fairscheduler
> Reporter: Karthik Kambatla
> Assignee: Yufei Gu
> Attachments: YARN-5830.001.patch
>
>
> While considering containers for preemption, avoid AM containers unless
> absolutely necessary.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
