[
https://issues.apache.org/jira/browse/YARN-5892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025425#comment-16025425
]
Eric Payne commented on YARN-5892:
----------------------------------
Thank you for the reviews.
bq. allUsersTimesWeights will be less than 1. I think in this case UL value is
higher
[~sunilg], I think this is similar to the question I answered
[above|https://issues.apache.org/jira/browse/YARN-5892?focusedCommentId=15972782&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15972782],
but I'll restate it here for the sake of clarity.
Having a combined sum of weights < 1 does not cause UL to be too large. This is
because {{userLimitResource}} (the return value of {{computeUserLimit}}) is
only ever used by {{getComputedResourceLimitFor\[Active|All\]Users}}, which
then multiplies the value of {{userLimitResource}} by the appropriate user's
weight before returning it. This will result in the correct value of userLimit
for each specific user. When the sum of active user(s)'s weight(s) is < 1, then
it is true that {{userLimitResource}} is larger than the actual number of
resources used. However, {{userLimitResource}} is just an intermediate value.
bq. In UserManager, do we also need to lock while updating
"activeUsersTimesWeights".
Can you please clarify where you see it being read or written outside the lock?
I think the code is within locks everwhere it is used.
bq. 1) Could you move CapacitySchedulerQueueManager#updateUserWeights to
LeafQueue#setupQueueConfigs.
[~leftnoteasy], good optimization. I will make this change, do testing and
await [~sunilg]'s response before submitting a new patch.
> Capacity Scheduler: Support user-specific minimum user limit percent
> --------------------------------------------------------------------
>
> Key: YARN-5892
> URL: https://issues.apache.org/jira/browse/YARN-5892
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacityscheduler
> Reporter: Eric Payne
> Assignee: Eric Payne
> Attachments: Active users highlighted.jpg, YARN-5892.001.patch,
> YARN-5892.002.patch, YARN-5892.003.patch, YARN-5892.004.patch,
> YARN-5892.005.patch, YARN-5892.006.patch, YARN-5892.007.patch,
> YARN-5892.008.patch, YARN-5892.009.patch, YARN-5892.010.patch,
> YARN-5892.012.patch, YARN-5892.013.patch
>
>
> Currently, in the capacity scheduler, the {{minimum-user-limit-percent}}
> property is per queue. A cluster admin should be able to set the minimum user
> limit percent on a per-user basis within the queue.
> This functionality is needed so that when intra-queue preemption is enabled
> (YARN-4945 / YARN-2113), some users can be deemed as more important than
> other users, and resources from VIP users won't be as likely to be preempted.
> For example, if the {{getstuffdone}} queue has a MULP of 25 percent, but user
> {{jane}} is a power user of queue {{getstuffdone}} and needs to be guaranteed
> 75 percent, the properties for {{getstuffdone}} and {{jane}} would look like
> this:
> {code}
> <property>
>
> <name>yarn.scheduler.capacity.root.getstuffdone.minimum-user-limit-percent</name>
> <value>25</value>
> </property>
> <property>
>
> <name>yarn.scheduler.capacity.root.getstuffdone.jane.minimum-user-limit-percent</name>
> <value>75</value>
> </property>
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
