Adam Antal commented on YARN-9930:

Thanks for the POC [~pbacsko]. Conceptually it looks good.

Some comments I can think of:
- During testing (but maybe it'd be helpful for detecting bugs in the future) 
we should build a guarantee into the system to make sure increment and 
decrement runnable apps calls  don't go into negative.
- As always, recovery can be a bit tricky. Let's say we change the 
configuration to limit the number of the apps in a queue to a lower value than 
the number of currently running apps. How CS will handle this case? (Also how 
can we prevent such errors?)
- As far as I understood, max running apps for queue is the sum of all the 
subqueues under it. The {{updateRunnabilityOnAppRemoval()}} and the methods 
called from there seemingly fulfills enforcing this, but I'd recommend creating 
a more complex UT where the max running apps property is configured for 
multiple queue levels under each other. This will probably make a good sanity 
check of your algorithm if there are is a sophisticated queue hierarchy.

> Support max running app logic for CapacityScheduler
> ---------------------------------------------------
>                 Key: YARN-9930
>                 URL: https://issues.apache.org/jira/browse/YARN-9930
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacity scheduler, capacityscheduler
>    Affects Versions: 3.1.0, 3.1.1
>            Reporter: zhoukang
>            Assignee: Peter Bacsko
>            Priority: Major
>         Attachments: YARN-9930-001.patch, YARN-9930-POC01.patch, 
> YARN-9930-POC02.patch, YARN-9930-POC03.patch, YARN-9930-POC04.patch, 
> YARN-9930-POC05.patch
> In FairScheduler, there has limitation for max running which will let 
> application pending.
> But in CapacityScheduler there has no feature like max running app.Only got 
> max app,and jobs will be rejected directly on client.
> This jira i want to implement this semantic for CapacityScheduler.

This message was sent by Atlassian Jira

To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

Reply via email to