[
https://issues.apache.org/jira/browse/YARN-8495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533902#comment-16533902
]
Yufei Gu commented on YARN-8495:
--------------------------------
Thanks [~Dillon.] for filing this. The implementation conflicts with the
preemption inside a queue, which assumes each application splits the resource
evenly, so that scheduler will kill the containers from the applications who
use more resource than its fair share. In a case that a low priority app A uses
less than its fair share and a high priority app B uses more than its fair
share. This priority scheduling will assign containers to B, while the
preemption will kill B's containers and intend to give them to A. A live lock
happens.
> Priority scheduling support in FairShare scheduler
> --------------------------------------------------
>
> Key: YARN-8495
> URL: https://issues.apache.org/jira/browse/YARN-8495
> Project: Hadoop YARN
> Issue Type: Wish
> Components: fairscheduler
> Reporter: Dillon Zhang
> Priority: Major
> Attachments: YARN-8495.001.patch
>
>
> In production environment, priority scheduling is of vital importance to us
> as we have lots of queues for different departments, then create applications
> but some of them are not so important as others, so we must guarantee the
> import ones to supply service.
> Based on the priority of the application, Fair Scheduler should be able to
> give preference to application while scheduling.
> Comparator<FairShareComparator> applicationComparator can be changed as
> below.
> 1. Check for Application priority. If priority is available, then return the
> highest priority job.
> 2. Otherwise continue with existing logic such as Fair Share comparison and
> App ID comparison and then TimeStamp comparison.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
