Wei Yan commented on YARN-2395:

bq. Do we want to introduce backward incompatible change by removing the 
existing global "fairSharePreemptionTimeout" config ?
Backward incompatible happens. Here the problem is, if we introduce per-queue 
preemption timeout, the old "fairSharePreemptionTimeout" always be 
incompatible. To solve this problem, we can check whether the old 
"fairSharePreemptionTimeout" exist in the alloc file; if existing and without 
"defaultFairSharePreemptionTimeout" config, we can treat the 
"fairSharePreemptionTimeout" as the default one.

bq. Just curious as to why we decided to inherit timeouts ?
Consider a scenario where we have one high priority parent queue and low 
priority parent queue, and each parent queue will create multiple user queues 
runtime. The high priority queues want to take the resource back as soon as 
possible, using a small preemption timeout; while the low priority queues are 
ok for a larger timeout. This cannot be achieved if we only have one default 
preemption timeout config.

> FairScheduler: Preemption timeout should be configurable per queue
> ------------------------------------------------------------------
>                 Key: YARN-2395
>                 URL: https://issues.apache.org/jira/browse/YARN-2395
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: fairscheduler
>            Reporter: Ashwin Shankar
>            Assignee: Wei Yan
>         Attachments: YARN-2395-1.patch, YARN-2395-2.patch, YARN-2395-3.patch, 
> YARN-2395-3.patch
> Currently in fair scheduler, the preemption logic considers fair share 
> starvation only at leaf queue level. This jira is created to implement it at 
> the parent queue as well.
> It involves :
> 1. Making "check for fair share starvation" and "amount of resource to 
> preempt"  recursive such that they traverse the queue hierarchy from root to 
> leaf.
> 2. Currently fairSharePreemptionTimeout is a global config. We could make it 
> configurable on a per queue basis,so that we can specify different timeouts 
> for parent queues.

This message was sent by Atlassian JIRA

Reply via email to