[
https://issues.apache.org/jira/browse/YARN-7839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350496#comment-16350496
]
Panagiotis Garefalakis edited comment on YARN-7839 at 2/2/18 3:13 PM:
----------------------------------------------------------------------
Submitting a simple patch tracking available cluster resources in the
DefaultPlacement algorithm - to support capacity check before placement.
The actual check is part of the attemptPlacementOnNode method which could be
configured with the *ignoreResourceCheck* flag.
In the current patch the check is enabled on placement step and disabled on the
validation step.
A wrapper class *SchedulingRequestWithPlacementAttempt* was also introduced to
keep track of the failed attempts on the rejected SchedulingRequests.
Thoughts? [~asuresh] [~kkaranasos] [~cheersyang]
was (Author: pgaref):
Submitting a simple patch tracking available cluster resources in the
DefaultPlacement algorithm - to support capacity check before placement.
The actual check is part of the attemptPlacementOnNode method which could be
configured with the **ignoreResourceCheck** flag.
In the current patch the check is enabled on placement step and disabled on the
validation step.
A wrapper class SchedulingRequestWithPlacementAttempt was also introduced to
keep track of the failed attempts on the rejected SchedulingRequests.
Thoughts? [~asuresh] [~kkaranasos] [~cheersyang]
> Check node capacity before placing in the Algorithm
> ---------------------------------------------------
>
> Key: YARN-7839
> URL: https://issues.apache.org/jira/browse/YARN-7839
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Arun Suresh
> Assignee: Panagiotis Garefalakis
> Priority: Major
> Attachments: YARN-7839-YARN-6592.001.patch
>
>
> Currently, the Algorithm assigns a node to a request purely based on if the
> constraints are met. It is later in the scheduling phase that the Queue
> capacity and Node capacity are checked. If the request cannot be placed
> because of unavailable Queue/Node capacity, the request is retried by the
> Algorithm.
> For clusters that are running at high utilization, we can reduce the retries
> if we perform the Node capacity check in the Algorithm as well. The Queue
> capacity check and the other user limit checks can still be handled by the
> scheduler (since queues and other limits are tied to the scheduler, and not
> scheduler agnostic)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
