[ https://issues.apache.org/jira/browse/YARN-2411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ram Venkatesh updated YARN-2411: -------------------------------- Attachment: YARN-2411.4.patch [~wangda.tan] thank you for your comments. I agree, it is better to check and reject the mapping upfront if it refers to a non-existent or non-leaf queue. Uploading patch with this change. > [Capacity Scheduler] support simple user and group mappings to queues > --------------------------------------------------------------------- > > Key: YARN-2411 > URL: https://issues.apache.org/jira/browse/YARN-2411 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacityscheduler > Reporter: Ram Venkatesh > Assignee: Ram Venkatesh > Attachments: YARN-2411-2.patch, YARN-2411.1.patch, YARN-2411.3.patch, > YARN-2411.4.patch > > > YARN-2257 has a proposal to extend and share the queue placement rules for > the fair scheduler and the capacity scheduler. This is a good long term > solution to streamline queue placement of both schedulers but it has core > infra work that has to happen first and might require changes to current > features in all schedulers along with corresponding configuration changes, if > any. > I would like to propose a change with a smaller scope in the capacity > scheduler that addresses the core use cases for implicitly mapping jobs that > have the default queue or no queue specified to specific queues based on the > submitting user and user groups. It will be useful in a number of real-world > scenarios and can be migrated over to the unified scheme when YARN-2257 > becomes available. > The proposal is to add two new configuration options: > yarn.scheduler.capacity.queue-mappings-override.enable > A boolean that controls if user-specified queues can be overridden by the > mapping, default is false. > and, > yarn.scheduler.capacity.queue-mappings > A string that specifies a list of mappings in the following format (default > is "" which is the same as no mapping) > <map_specifier>:<source_attribute>:<queue_name>[,<map_specifier>:<source_attribute>:<queue_name>]* > map_specifier := user (u) | group (g) > source_attribute := user | group | %user > queue_name := the name of the mapped queue | %user | %primary_group > The mappings will be evaluated left to right, and the first valid mapping > will be used. If the mapped queue does not exist, or the current user does > not have permissions to submit jobs to the mapped queue, the submission will > fail. > Example usages: > 1. user1 is mapped to queue1, group1 is mapped to queue2 > u:user1:queue1,g:group1:queue2 > 2. To map users to queues with the same name as the user: > u:%user:%user > I am happy to volunteer to take this up. -- This message was sent by Atlassian JIRA (v6.2#6252)