capacity scheduler - maxActiveApplicationsPerUser calculation can be wrong
--------------------------------------------------------------------------
Key: MAPREDUCE-3897
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3897
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.0
Reporter: Thomas Graves
Priority: Critical
The capacity scheduler calculates the maxActiveApplications and the
maxActiveApplicationsPerUser based on the config
yarn.scheduler.capacity.maximum-applications or default 10000.
MaxActiveApplications = max ( ceil ( clusterMemory/minAllocation *
maxAMResource% * absoluteMaxCapacity), 1)
MaxActiveAppsPerUser = max( ceil (maxActiveApplicationsComputedAbove *
(userLimit%/100) * userLimitFactor), 1)
maxActiveApplications is already multiplied by the queue absolute MAXIMUM
capacity, so if max capacity > capacity and if you have user limit factor 1
(which is the default) and only 1 user is running, that user will not be
allowed to use over the queue capacity, so having it relative to MAX capacity
doesn't make sense. That user could easily end up in a deadlock and all its
space used by application masters.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
