Sandy Ryza commented on YARN-3635:

[~leftnoteasy], apologies for this quick drive-by review - I am currently 

The JIRA appears to be lacking a design-doc and I wasn't able to find 
documentation in the patch.  The patch should ultimately include some detailed 
documentation, but I don't want to ask this of you before OKing the approach.  
In light of this, a few questions:
* What steps are required for the Fair Scheduler to integrate with this?
* Is a common way of configuration proposed?
* How does this differ from the current Fair Scheduler model?  To summarize:
** The FS model consists of a sequence of placement rules that the app is 
passed through.
** Each placement rule gets the chance to assign the app to a queue, reject the 
app, or pass.  If it passes, the next rule gets a chance.
** A placement rule can base its decision on:
*** The submitting user.
*** The set of groups the submitting user belongs to.
*** The queue requested in the app submission.
*** A set of configuration options that are specific to the rule.
*** The set of queues given in the Fair Scheduler configuration.
** Rules are marked as "terminal" if they will never pass.  This helps to avoid 
misconfigurations where users place rules after terminal rules.
** Rules have a "create" attribute which determines whether they can create a 
new queue or whether they must assign to existing queues.
** Currently the set of placement rules is limited to what's implemented in 
YARN.  I.e. there's no public pluggable rule support.
I noticed from Vinod's comment that this patch follows a similar structure.  
Are there places where my summary would not describe what's going on in this 

> Get-queue-mapping should be a common interface of YarnScheduler
> ---------------------------------------------------------------
>                 Key: YARN-3635
>                 URL: https://issues.apache.org/jira/browse/YARN-3635
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: scheduler
>            Reporter: Wangda Tan
>            Assignee: Tan, Wangda
>         Attachments: YARN-3635.1.patch, YARN-3635.2.patch, YARN-3635.3.patch, 
> YARN-3635.4.patch, YARN-3635.5.patch, YARN-3635.6.patch
> Currently, both of fair/capacity scheduler support queue mapping, which makes 
> scheduler can change queue of an application after submitted to scheduler.
> One issue of doing this in specific scheduler is: If the queue after mapping 
> has different maximum_allocation/default-node-label-expression of the 
> original queue, {{validateAndCreateResourceRequest}} in RMAppManager checks 
> the wrong queue.
> I propose to make the queue mapping as a common interface of scheduler, and 
> RMAppManager set the queue after mapping before doing validations.

This message was sent by Atlassian JIRA

Reply via email to