[
https://issues.apache.org/jira/browse/YARN-5892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Payne updated YARN-5892:
-----------------------------
Attachment: YARN-5892.007.patch
[~leftnoteasy],
Attaching a YARN-5892.007.patch, that addresses all of the above comments.
I have a couple of concerns, however.
1) I took out the code that allows the weight property to be inherited from
parent queues. I felt that it was inefficient (and maybe risky) because it
parsed through the weight conf property string several times for each user.
This patch requires the weight property to be set at the leaf queue only. If
it's all the same to you, I would like to add the inheritance feature in a
separate JIRA.
2) The other concern is larger, I think. Currently, the weight is applied to
the user limit in {{getComputedResourceLimitForActiveUsers}}. However, the
logic that prevents the user limit from falling below the minimum user limit
percent is done down at the {{computeUserLimit}} level. Which means that if a
user's weight is < 1, the user's user limit will fall below the minimum user
limit percent.
I can think of a couple of ways to get around this, but I would like to get
your opinion first. Also, CC-ing [~jlowe]
> Capacity Scheduler: Support user-specific minimum user limit percent
> --------------------------------------------------------------------
>
> Key: YARN-5892
> URL: https://issues.apache.org/jira/browse/YARN-5892
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacityscheduler
> Reporter: Eric Payne
> Assignee: Eric Payne
> Attachments: Active users highlighted.jpg, YARN-5892.001.patch,
> YARN-5892.002.patch, YARN-5892.003.patch, YARN-5892.004.patch,
> YARN-5892.005.patch, YARN-5892.006.patch, YARN-5892.007.patch
>
>
> Currently, in the capacity scheduler, the {{minimum-user-limit-percent}}
> property is per queue. A cluster admin should be able to set the minimum user
> limit percent on a per-user basis within the queue.
> This functionality is needed so that when intra-queue preemption is enabled
> (YARN-4945 / YARN-2113), some users can be deemed as more important than
> other users, and resources from VIP users won't be as likely to be preempted.
> For example, if the {{getstuffdone}} queue has a MULP of 25 percent, but user
> {{jane}} is a power user of queue {{getstuffdone}} and needs to be guaranteed
> 75 percent, the properties for {{getstuffdone}} and {{jane}} would look like
> this:
> {code}
> <property>
>
> <name>yarn.scheduler.capacity.root.getstuffdone.minimum-user-limit-percent</name>
> <value>25</value>
> </property>
> <property>
>
> <name>yarn.scheduler.capacity.root.getstuffdone.jane.minimum-user-limit-percent</name>
> <value>75</value>
> </property>
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
