[
https://issues.apache.org/jira/browse/YARN-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14326197#comment-14326197
]
Sunil G commented on YARN-2004:
-------------------------------
Hi Jason, thank you for sharing the thoughts.
In one way, we need not have to think abt headroom and userlimit. Still I would
like to share 2 scenarios
1. Similar to MAPREDUCE-314. A job j1 is submitted with lower priority and
finished its map tasks, reducers are running. later j2 and j3 came in and took
over cluster resources. if a map is failed, by loosing some map o/p, there are
no chances of getting a resource for j1 till j2 and j3 releases resources and
not allocating it. In a -ve scenario, j1 will starve for much longer. This was
one of the intention to temporarily pause demand from j2 and j4 for a while and
spare some resources for j1.
2. User Limit: Assume the factor is 25, and 4 users can take 25% each from
cluster. 5th user has to wait. Assume the highest priority app is submitted by
5th user. He may not get resources untill demand from first 4 users(for
existing apps) are over. Do you feel this is to be handled?
> Priority scheduling support in Capacity scheduler
> -------------------------------------------------
>
> Key: YARN-2004
> URL: https://issues.apache.org/jira/browse/YARN-2004
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacityscheduler
> Reporter: Sunil G
> Assignee: Sunil G
> Attachments: 0001-YARN-2004.patch
>
>
> Based on the priority of the application, Capacity Scheduler should be able
> to give preference to application while doing scheduling.
> Comparator<FiCaSchedulerApp> applicationComparator can be changed as below.
>
> 1. Check for Application priority. If priority is available, then return
> the highest priority job.
> 2. Otherwise continue with existing logic such as App ID comparison and
> then TimeStamp comparison.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
