Ram Venkatesh updated YARN-2411:

    Attachment: YARN-2411.3.patch

Thank you [~jianhe] for your comments, I have addressed both of them in this 

> [Capacity Scheduler] support simple user and group mappings to queues
> ---------------------------------------------------------------------
>                 Key: YARN-2411
>                 URL: https://issues.apache.org/jira/browse/YARN-2411
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: capacityscheduler
>            Reporter: Ram Venkatesh
>            Assignee: Ram Venkatesh
>         Attachments: YARN-2411-2.patch, YARN-2411.1.patch, YARN-2411.3.patch
> YARN-2257 has a proposal to extend and share the queue placement rules for 
> the fair scheduler and the capacity scheduler. This is a good long term 
> solution to streamline queue placement of both schedulers but it has core 
> infra work that has to happen first and might require changes to current 
> features in all schedulers along with corresponding configuration changes, if 
> any. 
> I would like to propose a change with a smaller scope in the capacity 
> scheduler that addresses the core use cases for implicitly mapping jobs that 
> have the default queue or no queue specified to specific queues based on the 
> submitting user and user groups. It will be useful in a number of real-world 
> scenarios and can be migrated over to the unified scheme when YARN-2257 
> becomes available.
> The proposal is to add two new configuration options:
> yarn.scheduler.capacity.queue-mappings-override.enable 
> A boolean that controls if user-specified queues can be overridden by the 
> mapping, default is false.
> and,
> yarn.scheduler.capacity.queue-mappings
> A string that specifies a list of mappings in the following format (default 
> is "" which is the same as no mapping)
> <map_specifier>:<source_attribute>:<queue_name>[,<map_specifier>:<source_attribute>:<queue_name>]*
> map_specifier := user (u) | group (g)
> source_attribute := user | group | %user
> queue_name := the name of the mapped queue | %user | %primary_group
> The mappings will be evaluated left to right, and the first valid mapping 
> will be used. If the mapped queue does not exist, or the current user does 
> not have permissions to submit jobs to the mapped queue, the submission will 
> fail.
> Example usages:
> 1. user1 is mapped to queue1, group1 is mapped to queue2
> u:user1:queue1,g:group1:queue2
> 2. To map users to queues with the same name as the user:
> u:%user:%user
> I am happy to volunteer to take this up.

This message was sent by Atlassian JIRA

Reply via email to