I think it's not valid. Multiply it by ULF seems not reasonable, I think it should be:
max(1, maxApplications * max(userlimit/100, 1/#activeUsers)) Assuming admin setups a very large ULF (e.g. 100), maxApplicationsPerUser can be much more than maxApplications of a queue. Also, multiply ULF to compute userAMResourceLimit seems not valid too, it can lead to a single user can run too much applications than expected, which we should avoid. Thoughts? Thanks, Wangda On Thu, Jul 16, 2015 at 12:53 AM, Naganarasimha G R (Naga) < [email protected]> wrote: > Hi Folks , > Came across one scenario where in maxApplications @ cluster level(2 > node) was set to a low value like 10 and based on capacity configuration > for a particular queue it was coming to 2 as value, but further while > calculating maxApplicationsPerUser formula used is : > > *maxApplicationsPerUser = (int)(maxApplications * (userLimit / 100.0f) * > userLimitFactor);* > > but the definition of *userLimit *in the documentation is : > *Each queue enforces a limit on the percentage of resources allocated to a > user at any given time, if there is demand for resources. The user limit > can vary between a minimum and maximum value. The the former (the minimum > value) is set to this property value and the latter (the maximum value) > depends on the number of users who have submitted applications. For e.g., > suppose the value of this property is 25. If two users have submitted > applications to a queue, no single user can use more than 50% of the queue > resources. If a third user submits an application, no single user can use > more than 33% of the queue resources. With 4 or more users, no user can use > more than 25% of the queues resources. A value of 100 implies no user > limits are imposed. The default is 100. Value is specified as a integer.* > > So was wondering how a* minimum limit is made used in a formula to > calculate max applications for a user*, suppose i set " > yarn.scheduler.capacity.<queue-path>.minimum-user-limit-percent" to *20 * > assuming > at least 20% of queue at the minimum is available for a queue but based on > the formula *maxApplicationsPerUser * is getting set to *zero. * > According to the definition of the property max is based on the current > no. of active users, so i feel this formula is wrong. > *P.S. *userLimitFactor was configured as default 1, but what i am > wondering is whether its valid to use it in combination with userlimit to > find max apps per user. > > Please correct me if my understanding is wrong? if its a bug would like > to raise and solve it . > > + Naga >
