[
https://issues.apache.org/jira/browse/YARN-5830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15836438#comment-15836438
]
ASF GitHub Bot commented on YARN-5830:
--------------------------------------
Github user kambatla commented on a diff in the pull request:
https://github.com/apache/hadoop/pull/180#discussion_r97617498
--- Diff:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSPreemptionThread.java
---
@@ -119,39 +116,81 @@ public void run() {
continue;
}
- // Figure out list of containers to consider
- List<RMContainer> containersToCheck =
- node.getCopiedListOfRunningContainers();
- containersToCheck.removeAll(node.getContainersForPreemption());
-
- // Initialize potential with unallocated resources
- Resource potential = Resources.clone(node.getUnallocatedResource());
- for (RMContainer container : containersToCheck) {
- FSAppAttempt app =
- scheduler.getSchedulerApp(container.getApplicationAttemptId());
-
- if (app.canContainerBePreempted(container)) {
- // Flag container for preemption
- containers.add(container);
- Resources.addTo(potential, container.getAllocatedResource());
+ int maxAMContainers = bestContainers == null ?
+ Integer.MAX_VALUE : bestContainers.numAMContainers;
+ PreemptableContainers preemptableContainers =
+ identifyContainersToPreemptOnNode(requestCapability, node,
+ maxAMContainers);
+ if (preemptableContainers != null) {
+ if (preemptableContainers.numAMContainers == 0) {
+ return preemptableContainers;
+ } else {
+ bestContainers = preemptableContainers;
}
+ }
+ }
- // Check if we have already identified enough containers
- if (Resources.fitsIn(requestCapability, potential)) {
- // Mark the containers as being considered for preemption on the
node.
- // Make sure the containers are subsequently removed by calling
- // FSSchedulerNode#removeContainerForPreemption.
- node.addContainersForPreemption(containers);
- return containers;
- } else {
- // TODO (YARN-5829): Unreserve the node for the starved app.
+ return bestContainers;
+ }
+
+ /**
+ * Identify containers to preempt on a given node. Try to find a list
with
+ * least AM containers to avoid preempt AM containers. This method
returns a
--- End diff --
s/preempt/preempting
> Avoid preempting AM containers
> ------------------------------
>
> Key: YARN-5830
> URL: https://issues.apache.org/jira/browse/YARN-5830
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: fairscheduler
> Reporter: Karthik Kambatla
> Assignee: Yufei Gu
> Attachments: YARN-5830.001.patch, YARN-5830.002.patch,
> YARN-5830.003.patch, YARN-5830.004.patch, YARN-5830.005.patch,
> YARN-5830.006.patch
>
>
> While considering containers for preemption, avoid AM containers unless
> absolutely necessary.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
