Ram Venkatesh created YARN-2411:
-----------------------------------

             Summary: [Capacity Scheduler] support simple user and group 
mappings to queues
                 Key: YARN-2411
                 URL: https://issues.apache.org/jira/browse/YARN-2411
             Project: Hadoop YARN
          Issue Type: Improvement
          Components: capacityscheduler
            Reporter: Ram Venkatesh


YARN-2257 has a proposal to extend and share the queue placement rules for the 
fair scheduler and the capacity scheduler. This is a good long term solution to 
streamline queue placement of both schedulers but it has core infra work that 
has to happen first and might require changes to current features in all 
schedulers along with corresponding configuration changes, if any. 

I would like to propose a change with a smaller scope in the capacity scheduler 
that addresses the core use cases for implicitly mapping jobs to queues based 
on the submitting user and user groups. It will be useful in a number of 
real-world scenarios and can be migrated over to the unified scheme when 
YARN-2257 becomes available.

The proposal is to add two new configuration options:

yarn.scheduler.capacity.queue-mappings.enable 
A boolean that controls if queue mappings are enabled, default is false.

and,
yarn.scheduler.capacity.queue-mappings
A string that specifies a list of mappings in the following format:

<map_specifier>:<source_attribute>:<queue_name>[,<map_specifier>:<source_attribute>:<queue_name>]*
map_specifier := user (u) | group (g)
source_attribute := user | group | %user
queue_name := the name of the mapped queue | %user | %primary_group

The mappings will be evaluated left to right, and the first valid mapping will 
be used. If the mapped queue does not exist, or the current user does not have 
permissions to submit jobs to the mapped queue, the submission will fail.

Example usages:
1. user1 is mapped to queue1, group1 is mapped to queue2
u:user1:queue1,g:group1:queue2

2. To map users to queues with the same name as the user:
u:%user:%user

I am happy to volunteer to take this up.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Reply via email to