shuzirra commented on a change in pull request #3500:
URL: https://github.com/apache/hadoop/pull/3500#discussion_r730821045
##########
File path:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
##########
@@ -2326,6 +2326,57 @@ public void stopQueue() {
}
}
+ void updateMaximumApplications(CapacitySchedulerConfiguration conf) {
+ int maxAppsForQueue = conf.getMaximumApplicationsPerQueue(getQueuePath());
+
+ int maxGlobalApplications = conf.getGlobalMaximumApplicationsPerQueue();
+ int maxSystemApplications = conf.getMaximumSystemApplications();
+ int baseMaxApplications = maxGlobalApplications > 0 ?
+ maxGlobalApplications : maxSystemApplications;
Review comment:
According to the documentation:
Maximum number of applications in the system which can be concurrently
active both running and pending. Limits on each queue are directly proportional
to their queue capacities and user limits. This is a hard limit and any
applications submitted when this limit is reached will be rejected. Default is
10000
This means to me, that maxSystemApp should be an absolute upper limit, if
the user wishes to go above it, they should raise this limit. MaxGlobalApp
should be always <= maxSystem app. But look at this the following way: if the
user has 100 queues with maxGlobalApp 1000, and they saturate 10 of them with
10000 applications, then they won't be able to start any application in the
11th queue, because of the maxSystem is reached. So this limitation is already
in place. As it should be.
See:
https://github.com/apache/hadoop/blob/5337bebcc53a0144311877362787ed9ece0cb118/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java#L1044
So capacity scheduler will enforce the max system application anyway, even
if the leaf queue would allow it, so there is no point in allowing more
application per leaf queue than the system wide max applications.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
