[ https://issues.apache.org/jira/browse/YARN-2411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14100763#comment-14100763 ]
Hudson commented on YARN-2411: ------------------------------ FAILURE: Integrated in Hadoop-Hdfs-trunk #1841 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1841/]) YARN-2411. Support simple user and group mappings to queues. Contributed by Ram Venkatesh (jianhe: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1618542) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/conf/capacity-scheduler.xml * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestQueueMappings.java > [Capacity Scheduler] support simple user and group mappings to queues > --------------------------------------------------------------------- > > Key: YARN-2411 > URL: https://issues.apache.org/jira/browse/YARN-2411 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacityscheduler > Reporter: Ram Venkatesh > Assignee: Ram Venkatesh > Fix For: 2.6.0 > > Attachments: YARN-2411-2.patch, YARN-2411.1.patch, YARN-2411.3.patch, > YARN-2411.4.patch, YARN-2411.5.patch > > > YARN-2257 has a proposal to extend and share the queue placement rules for > the fair scheduler and the capacity scheduler. This is a good long term > solution to streamline queue placement of both schedulers but it has core > infra work that has to happen first and might require changes to current > features in all schedulers along with corresponding configuration changes, if > any. > I would like to propose a change with a smaller scope in the capacity > scheduler that addresses the core use cases for implicitly mapping jobs that > have the default queue or no queue specified to specific queues based on the > submitting user and user groups. It will be useful in a number of real-world > scenarios and can be migrated over to the unified scheme when YARN-2257 > becomes available. > The proposal is to add two new configuration options: > yarn.scheduler.capacity.queue-mappings-override.enable > A boolean that controls if user-specified queues can be overridden by the > mapping, default is false. > and, > yarn.scheduler.capacity.queue-mappings > A string that specifies a list of mappings in the following format (default > is "" which is the same as no mapping) > <map_specifier>:<source_attribute>:<queue_name>[,<map_specifier>:<source_attribute>:<queue_name>]* > map_specifier := user (u) | group (g) > source_attribute := user | group | %user > queue_name := the name of the mapped queue | %user | %primary_group > The mappings will be evaluated left to right, and the first valid mapping > will be used. If the mapped queue does not exist, or the current user does > not have permissions to submit jobs to the mapped queue, the submission will > fail. > Example usages: > 1. user1 is mapped to queue1, group1 is mapped to queue2 > u:user1:queue1,g:group1:queue2 > 2. To map users to queues with the same name as the user: > u:%user:%user > I am happy to volunteer to take this up. -- This message was sent by Atlassian JIRA (v6.2#6252)