Sandy Ryza commented on YARN-2073:

There are some situations where preemption with free resources on the cluster 
is the right thing to do.

For example, if I'm requesting 2 GB containers, I have no resources, and 100 
nodes on the cluster each have 1GB remaining, containers should get preempted 
on my behalf.

There are also cases arising from requests with strict locality - the cluster 
might have resources available because I'm waiting on a subset of nodes.  (In 
this case, we'd probably want to make sure preemption only happens on the nodes 
being waited for; otherwise we'd kill containers needlessly).

If the goal is to make sure that we aren't preempting on behalf of an 
application that's actually receiving resources, it might also be worth 
considering time-based approaches. E.g. only preempt on behalf of an 
application that hasn't received resources in some amount of time.

> FairScheduler starts preempting resources even with free resources on the 
> cluster
> ---------------------------------------------------------------------------------
>                 Key: YARN-2073
>                 URL: https://issues.apache.org/jira/browse/YARN-2073
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: scheduler
>    Affects Versions: 2.4.0
>            Reporter: Karthik Kambatla
>            Assignee: Karthik Kambatla
>            Priority: Critical
>         Attachments: yarn-2073-0.patch, yarn-2073-1.patch, yarn-2073-2.patch
> Preemption should kick in only when the currently available slots don't match 
> the request. 

This message was sent by Atlassian JIRA

Reply via email to