[
https://issues.apache.org/jira/browse/YARN-2669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16305336#comment-16305336
]
Carlos Martinez Moller commented on YARN-2669:
----------------------------------------------
Doing this kind of replacement (james.smith to james_dot_smith) adds some
limitations to automating queue placement.
I don't know if they were possible before... if they weren't it would be also
good to consider this enhancement.
The Fair Scheduler allows for Subpool management which is very useful in
certain scenarios, where two or more levels are used.
Scenario:
Let's imagine that due to budgets I want to split the cluster in two main
queues: DepartmentA (30% of the cluster) and DepartmentB (70% of the cluster)
And then split the queue DeparmentA in two: ProjectA(40%) and ProjectB (60%)
Using secondary groups we could have created a Unix Group
"DeparmentA.ProjectA" and assign it to the user executing Batches for this
department.
This allows for an easy/centralized management of Queues when SubQueues are
used:
- Single database of which users work on which pools (/etc/group/LDAP)
- Easy to reallocate/redesign the solution by changing the secondary groups of
users
But this Jira seems to think on a flat Queue solution, I believe Subpools are
not considered (other than the group.username kind)
With the solution adopted on this Jira we are forced to specify the Queue for
each Job and do Queue management for each job. In the case of redesigning the
Pool distribution have many things to change in mind. Not everything comes from
Linux shell and uses the same technology, sometimes is JDBC executions on Hive,
etc so the workaround is not as easy as modifying the .profile of an user to
specify a queue.
I could not think for a Workaround to this.
> FairScheduler: queue names shouldn't allow periods
> --------------------------------------------------
>
> Key: YARN-2669
> URL: https://issues.apache.org/jira/browse/YARN-2669
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Wei Yan
> Assignee: Wei Yan
> Fix For: 2.7.0
>
> Attachments: YARN-2669-1.patch, YARN-2669-2.patch, YARN-2669-3.patch,
> YARN-2669-4.patch, YARN-2669-5.patch
>
>
> For an allocation file like:
> {noformat}
> <allocations>
> <queue name="root.q1">
> <minResources>4096mb,4vcores</minResources>
> </queue>
> </allocations>
> {noformat}
> Users may wish to config minResources for a queue with full path "root.q1".
> However, right now, fair scheduler will treat this configureation for the
> queue with full name "root.root.q1". We need to print out a warning msg to
> notify users about this.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
