Panagiotis Garefalakis commented on YARN-7839:


Submitting a simple patch tracking available cluster resources in the 
DefaultPlacement algorithm - to support capacity check before placement.

The actual check is part of the attemptPlacementOnNode method which could be 
configured with the **ignoreResourceCheck** flag.

In the current patch the check is enabled on placement step and disabled on the 
validation step.

A wrapper class SchedulingRequestWithPlacementAttempt was also introduced to 
keep track of the failed attempts on the rejected SchedulingRequests.


Thoughts?  [~asuresh] [~kkaranasos] [~cheersyang] 

> Check node capacity before placing in the Algorithm
> ---------------------------------------------------
>                 Key: YARN-7839
>                 URL: https://issues.apache.org/jira/browse/YARN-7839
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Arun Suresh
>            Assignee: Panagiotis Garefalakis
>            Priority: Major
>         Attachments: YARN-7839-YARN-6592.001.patch
> Currently, the Algorithm assigns a node to a request purely based on if the 
> constraints are met. It is later in the scheduling phase that the Queue 
> capacity and Node capacity are checked. If the request cannot be placed 
> because of unavailable Queue/Node capacity, the request is retried by the 
> Algorithm.
> For clusters that are running at high utilization, we can reduce the retries 
> if we perform the Node capacity check in the Algorithm as well. The Queue 
> capacity check and the other user limit checks can still be handled by the 
> scheduler (since queues and other limits are tied to the scheduler, and not 
> scheduler agnostic)

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

Reply via email to