[
https://issues.apache.org/jira/browse/YARN-3945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15104199#comment-15104199
]
Naganarasimha G R commented on YARN-3945:
-----------------------------------------
Thanks for the comment [~wangda],
But the particular case which i had raised in the forum was :
{quote}
Came across one scenario where in maxApplications @ cluster level(2 node) was
set to a low
value like 10 and based on capacity configuration for a particular queue it was
coming to
2 as value, but further while calculating maxApplicationsPerUser formula used
is :
maxApplicationsPerUser = (int)(maxApplications * (userLimit / 100.0f) *
userLimitFactor);
{quote}
i had kept userlimit factor as 1 and user limit as 25,
so it was coming as zero. This in my opinion is wrong, i feel better not to
consider *userLimit and userLimitFactor* at all, to reduce the confusion for
the number of applications per user.
Futher even if suppose we consider user limit & userLimitFactor for
*UserAMResource* current approach of calculating it in
{{getUserAMResourceLimitPerPartition}} is different from {{computeUserLimit}}
In *getUserAMResourceLimitPerPartition* :
{code}
return Resources.multiplyAndNormalizeUp(resourceCalculator,
queuePartitionResource,
queueCapacities.getMaxAMResourcePercentage(nodePartition)
* effectiveUserLimit * userLimitFactor, minimumAllocation);
{code}
In *computeUserLimit*
{code}
// Cap final user limit with maxUserLimit
userLimitResource =
Resources.roundUp(
resourceCalculator,
Resources.min(
resourceCalculator, clusterResource,
userLimitResource,
maxUserLimit
),
minimumAllocation);
{code}
IMO it should be min (userLimitResource,maxUserLimit) and not multiple of it ,
thoughts ?
bq. Now numAppsPerUser could be more than numAppsPerQueue (before of
user-limit). Same to user-resource and user-am-resource, it will be helpful to
make sure they're capped by queue's limitation (am-resource, number-am,
queue-max-resource, etc.).
IMO numAppsPerUser can be greater than numAppsPerQueue and user-resource and
user-am-resource greater than the queue's resource or queue's AM resource only
when userLimitFactor is of really greater value, so is it actually required to
be greater than 1, Is it sufficient to restrict this to 1 ?
> maxApplicationsPerUser is wrongly calculated
> --------------------------------------------
>
> Key: YARN-3945
> URL: https://issues.apache.org/jira/browse/YARN-3945
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacityscheduler
> Affects Versions: 2.7.1
> Reporter: Naganarasimha G R
> Assignee: Naganarasimha G R
> Attachments: YARN-3945.20150728-1.patch, YARN-3945.20150729-1.patch,
> YARN-3945.V1.003.patch
>
>
> maxApplicationsPerUser is currently calculated based on the formula
> {{maxApplicationsPerUser = (int)(maxApplications * (userLimit / 100.0f) *
> userLimitFactor)}} but description of userlimit is
> {quote}
> Each queue enforces a limit on the percentage of resources allocated to a
> user at any given time, if there is demand for resources. The user limit can
> vary between a minimum and maximum value.{color:red} The the former (the
> minimum value) is set to this property value {color} and the latter (the
> maximum value) depends on the number of users who have submitted
> applications. For e.g., suppose the value of this property is 25. If two
> users have submitted applications to a queue, no single user can use more
> than 50% of the queue resources. If a third user submits an application, no
> single user can use more than 33% of the queue resources. With 4 or more
> users, no user can use more than 25% of the queues resources. A value of 100
> implies no user limits are imposed. The default is 100. Value is specified as
> a integer.
> {quote}
> configuration related to minimum limit should not be made used in a formula
> to calculate max applications for a user
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
