Allen Wittenauer commented on YARN-2413:

I might have missed something, but there appears to be a very bad bug here.  
Given the following settings:

          "mapreduce.map.cpu.vcores": 10
          "yarn.scheduler.maximum-allocation-vcores": 220
          "mapreduce.reduce.cpu.vcores": 10
          "yarn.app.mapreduce.am.resource.cpu-vcores": 10
          "yarn.scheduler.minimum-allocation-vcores": 10
          "yarn.nodemanager.resource.cpu-vcores": 221

The resource manager is only allocating 1 vcore per container:

2014-08-12 15:49:31,809 INFO 
org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerNode: Assigned 
container container_1407883573269_0001_01_000001 of capacity <memory:2048, 
vCores:1> on host, which has 1 containers, <memory:2048, 
vCores:1> used and <memory:2050, vCores:220> available after allocation

... which is clearly wrong, as the number of vcores requested should have been 
10 and the available remaining should have been 211.

> capacity scheduler will overallocate vcores
> -------------------------------------------
>                 Key: YARN-2413
>                 URL: https://issues.apache.org/jira/browse/YARN-2413
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: scheduler
>    Affects Versions: 3.0.0, 2.2.0
>            Reporter: Allen Wittenauer
>            Priority: Critical
> It doesn't appear that the capacity scheduler is properly allocation vcores 
> when making scheduling decisions, which results in overallocation of CPU 
> resources.

This message was sent by Atlassian JIRA

Reply via email to