Wangda Tan commented on YARN-2257:

Hi [~Patrick Liu],
Thanks for creating this JIRA, I think it's a very useful feature and should be 
considered to release soon.

To [~sandyr], I think it's better to make it as a basic module of YARN, other 
scheduler can all benefit from it. W can simply implement a module can read 
from configuration, which will return queue of following sequence:
- If queue defined in ApplicationSubmissionContext, use it,
- If userA->queueB defined, use queueB
- If groupA (userA belongs groupA)->queueB defined, use queueB
- use default queue otherwise.

We may need consider to create a separated service to load updated user->queue 
mapping periodical as well. 
Any thoughts?


> Add user to queue mappings to automatically place users' apps into specific 
> queues
> ----------------------------------------------------------------------------------
>                 Key: YARN-2257
>                 URL: https://issues.apache.org/jira/browse/YARN-2257
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: scheduler
>            Reporter: Patrick Liu
>            Assignee: Vinod Kumar Vavilapalli
>              Labels: features
> Currently, the fair-scheduler supports two modes, default queue or individual 
> queue for each user.
> Apparently, the default queue is not a good option, because the resources 
> cannot be managed for each user or group.
> However, individual queue for each user is not good enough. Especially when 
> connecting yarn with hive. There will be increasing hive users in a corporate 
> environment. If we create a queue for a user, the resource management will be 
> hard to maintain.
> I think the problem can be solved like this:
> 1. Define user->queue mapping in Fair-Scheduler.xml. Inside each queue, use 
> aclSubmitApps to control user's ability.
> 2. Each time a user submit an app to yarn, if the user has mapped to a queue, 
> the app will be scheduled to that queue; otherwise, the app will be submitted 
> to default queue.
> 3. If the user cannot pass aclSubmitApps limits, the app will not be accepted.

This message was sent by Atlassian JIRA

Reply via email to