Ashwin Shankar commented on YARN-2394:

[~ywskycn], [~kasha]
Inheriting fairSharePreemptionThreshold wouldn't work for us. Do you guys have 
a use case for inheriting this config ?
We have the following requirement :
root.parentUserQueue : fair share = 80% fairSharePreemptionThreshold=1
root.parentUserQueue.user(1-100) : defaultFairSharePreemptionThreshold=0.5

Basically when parentUserQueue is starved, we want to preempt more 
aggressively(threshold=1) from its siblings under root.
While when a user queue is starved, we want to preempt less aggressively from 
other user queues(assuming other user queues are above fair share).
We can do this by setting fairSharePreemptionThreshold=1 at parentUserQueue and 
defaultFairSharePreemptionThreshold=0.5 which would be used by user queues.
You might say that we can override fairSharePreemptionThreshold to 0.5 in the 
user queues, but these queues are created on the fly when a new user 
submits an app and NOT through the alloc xml and there are hundreds of them, so 
its not very feasible to override each one of them in the config.

Would it be okay if we stick to version-1 of the patch which doesn't inherit 
threshold ? 

> Fair Scheduler : ability to configure fairSharePreemptionThreshold per queue
> ----------------------------------------------------------------------------
>                 Key: YARN-2394
>                 URL: https://issues.apache.org/jira/browse/YARN-2394
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: fairscheduler
>            Reporter: Ashwin Shankar
>            Assignee: Wei Yan
>         Attachments: YARN-2394-1.patch, YARN-2394-2.patch
> Preemption based on fair share starvation happens when usage of a queue is 
> less than 50% of its fair share. This 50% is hardcoded. We'd like to make 
> this configurable on a per queue basis, so that we can choose the threshold 
> at which we want to preempt. Calling this config 
> "fairSharePreemptionThreshold". 

This message was sent by Atlassian JIRA

Reply via email to