[
https://issues.apache.org/jira/browse/YARN-5892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025713#comment-16025713
]
Sunil G commented on YARN-5892:
-------------------------------
Thanks [~eepayne]
bq.which then multiplies the value of userLimitResource by the appropriate
user's weight before returning it
I think I am fine here as we multiply with real weight of user, this will help
to bring UL to correct value. Thanks for explaining in detail. I also have one
more doubt now.
{code}
Resource userSpecificUserLimit =
Resources.multiplyAndNormalizeUp(resourceCalculator,
userLimitResource, weight, lQueue.getMinimumAllocation());
{code}
I think we could use multiplyAndNormalizeDown here. I have 2 reasons for this.
1) Ideally we allow atleast one container (extra) for a user given UL is
lesser. So it might be fine to use multiplyAndNormalizeDown given we are not
breaking a valid use case.
We do a > check here, not >=
{code:title=LeafQueue#canAssignToUser}
if (Resources.greaterThan(resourceCalculator, clusterResource,
user.getUsed(nodePartition), limit)) {
...
{code}
2) weight_user1=0.1, weight_user2=0.1. Now consider userLimitResource is some
how 10GB and minimumAllocation is 4GB. In this case, both user1 and 2 will get
UL as 4GB. This will help each user to get 2 containers each. I assume we have
queue elasticity and other queue has some more resources. In this case, I feel
we do not need to award a user with 2 containers, correct.?
Please correct me if I am wrong.
bq.I think the code is within locks everwhere it is used.
Yes. I did check the code detail. We are fine here, below code was not having
lock which confused me, but its caller has correct lock.
{{UsersManager.addUser(String userName, User user)}}
> Capacity Scheduler: Support user-specific minimum user limit percent
> --------------------------------------------------------------------
>
> Key: YARN-5892
> URL: https://issues.apache.org/jira/browse/YARN-5892
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacityscheduler
> Reporter: Eric Payne
> Assignee: Eric Payne
> Attachments: Active users highlighted.jpg, YARN-5892.001.patch,
> YARN-5892.002.patch, YARN-5892.003.patch, YARN-5892.004.patch,
> YARN-5892.005.patch, YARN-5892.006.patch, YARN-5892.007.patch,
> YARN-5892.008.patch, YARN-5892.009.patch, YARN-5892.010.patch,
> YARN-5892.012.patch, YARN-5892.013.patch
>
>
> Currently, in the capacity scheduler, the {{minimum-user-limit-percent}}
> property is per queue. A cluster admin should be able to set the minimum user
> limit percent on a per-user basis within the queue.
> This functionality is needed so that when intra-queue preemption is enabled
> (YARN-4945 / YARN-2113), some users can be deemed as more important than
> other users, and resources from VIP users won't be as likely to be preempted.
> For example, if the {{getstuffdone}} queue has a MULP of 25 percent, but user
> {{jane}} is a power user of queue {{getstuffdone}} and needs to be guaranteed
> 75 percent, the properties for {{getstuffdone}} and {{jane}} would look like
> this:
> {code}
> <property>
>
> <name>yarn.scheduler.capacity.root.getstuffdone.minimum-user-limit-percent</name>
> <value>25</value>
> </property>
> <property>
>
> <name>yarn.scheduler.capacity.root.getstuffdone.jane.minimum-user-limit-percent</name>
> <value>75</value>
> </property>
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
