[
https://issues.apache.org/jira/browse/YARN-9930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126751#comment-17126751
]
Adam Antal commented on YARN-9930:
----------------------------------
Thanks for the POC [~pbacsko]. Conceptually it looks good.
Some comments I can think of:
- During testing (but maybe it'd be helpful for detecting bugs in the future)
we should build a guarantee into the system to make sure increment and
decrement runnable apps calls don't go into negative.
- As always, recovery can be a bit tricky. Let's say we change the
configuration to limit the number of the apps in a queue to a lower value than
the number of currently running apps. How CS will handle this case? (Also how
can we prevent such errors?)
- As far as I understood, max running apps for queue is the sum of all the
subqueues under it. The {{updateRunnabilityOnAppRemoval()}} and the methods
called from there seemingly fulfills enforcing this, but I'd recommend creating
a more complex UT where the max running apps property is configured for
multiple queue levels under each other. This will probably make a good sanity
check of your algorithm if there are is a sophisticated queue hierarchy.
> Support max running app logic for CapacityScheduler
> ---------------------------------------------------
>
> Key: YARN-9930
> URL: https://issues.apache.org/jira/browse/YARN-9930
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler, capacityscheduler
> Affects Versions: 3.1.0, 3.1.1
> Reporter: zhoukang
> Assignee: Peter Bacsko
> Priority: Major
> Attachments: YARN-9930-001.patch, YARN-9930-POC01.patch,
> YARN-9930-POC02.patch, YARN-9930-POC03.patch, YARN-9930-POC04.patch,
> YARN-9930-POC05.patch
>
>
> In FairScheduler, there has limitation for max running which will let
> application pending.
> But in CapacityScheduler there has no feature like max running app.Only got
> max app,and jobs will be rejected directly on client.
> This jira i want to implement this semantic for CapacityScheduler.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
