Wangda Tan commented on YARN-2003:

Hi [~sunilg],
Thanks for working on this. I took a quick look at YARN-2003 / YARN-2004. Some 
overall comments.

First I want to describe my thinkings of how RM side workflow look like:
- When application submit to RMAppManager, it will simply pass priority set by 
user to RMApp (see (1))
- RMApp will finally create APP_ATTEMPT_ADDED, queue itself will normalize 
priority (reject it / convert it from label to number, etc. And set new 
priority to RMApp), set priority to SchedulerApplicationAttempt
- Scheduler uses priority in SchedulerApplicationAttempt and Queue to make 
scheduling decisions
- If user ask for changing application priority, or admin changes priority 
configuration, event may need send to scheduler to update inner 
- When user requires priority of application via CLI/Web-UI, 
ApplicationPriorityManager will convert number to label (if possible) and show 
to user.

(1). We don't have to add too much logic here, and if we can simply handle it 
inside scheduler, when configuration changed (like label->integer priority 
mapping changed), it can be handled by scheduler itself.

Back to your patches, several major differences are:
- RMAppManager takes responsibility to check ACL for priority, which I think 
not proper (ACL is always managed by scheduler queues) and not correct (when 
queue configuration changed, you cannot do rechecking application priority if 
new app attempt created.).
- As same as above, queues should check ACLs for priority.
- You may not need add priority to SchedulerApplication, it seems not necessary 
to me.

> Support to process Job priority from Submission Context in 
> AppAttemptAddedSchedulerEvent [RM side]
> --------------------------------------------------------------------------------------------------
>                 Key: YARN-2003
>                 URL: https://issues.apache.org/jira/browse/YARN-2003
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Sunil G
>            Assignee: Sunil G
>         Attachments: 0001-YARN-2003.patch, 0002-YARN-2003.patch, 
> 0003-YARN-2003.patch, 0004-YARN-2003.patch
> AppAttemptAddedSchedulerEvent should be able to receive the Job Priority from 
> Submission Context and store.
> Later this can be used by Scheduler.

This message was sent by Atlassian JIRA

Reply via email to