Andrey Stepachev commented on YARN-2073:

I read code and don't cleanly understand why inequality uses Available 
resources instead of Allocated?
With current code preemption performed only when cluster utilised under 
Suppose we have occupied 8GB of 10GB. And default preemption threshold == 0.8f.
>From description I thought that preemption should occur, but it doesn't.
Code below gives us: 
0.8 < availableMb / maxMem
0.8 < (10-8) / 10
0.8 < 0.2 
      return (preemptionUtilizationThreshold < Math.max(
          (float) rootMetrics.getAvailableMB() / clusterResource.getMemory(),
          (float) rootMetrics.getAvailableVirtualCores() /

Is that was an intention for this parameter (and it should be documented for 
non native speakers to be 
more clear, say: preemption will not performed if utilisation is above this 
parameter) or here was a mistake?

> Fair Scheduler: Add a utilization threshold to prevent preempting resources 
> when cluster is free
> ------------------------------------------------------------------------------------------------
>                 Key: YARN-2073
>                 URL: https://issues.apache.org/jira/browse/YARN-2073
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: scheduler
>    Affects Versions: 2.4.0
>            Reporter: Karthik Kambatla
>            Assignee: Karthik Kambatla
>            Priority: Critical
>             Fix For: 2.5.0
>         Attachments: yarn-2073-0.patch, yarn-2073-1.patch, yarn-2073-2.patch, 
> yarn-2073-3.patch, yarn-2073-4.patch
> Preemption should kick in only when the currently available slots don't match 
> the request. 

This message was sent by Atlassian JIRA

Reply via email to