Sunil G commented on YARN-2305:

1. Yes [~leftnoteasy], GUI display of 106% is similar to YARN-2285. It can be 
tackled there.

As mentioned in earlier comment, Total MB in GUI is internally sum of 

a. *LeafQueue#usedResources* is sum of used and reserved memory.  
But *CSQueueUtils#updateQueueStatistics()* code may give a -ve value in case of 
reservation which sets availableMB in QueueMetrics.
Resource available = Resources.subtract(queueLimit, usedResources);

If this comes as -ve, then *availableMB* is set as 0.

b. *allocatedMB*: This is set when a container is really allocated. This is the 
real queue usage.

In above scenario, it should have come as 
But due to reservation, allocatedMB became 0. Hence total shown as 15.
I feel instead of showing Total as *allocated+available*, we can show 
*clusterResource* here. Any particular reason why we need like 
allocated+available, thoughts?

> When a container is in reserved state then total cluster memory is displayed 
> wrongly.
> -------------------------------------------------------------------------------------
>                 Key: YARN-2305
>                 URL: https://issues.apache.org/jira/browse/YARN-2305
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: resourcemanager
>    Affects Versions: 2.4.1
>            Reporter: J.Andreina
>            Assignee: Sunil G
>         Attachments: Capture.jpg
> ENV Details:
> =========  
>          3 queues  :  a(50%),b(25%),c(25%) ---> All max utilization is set to 
> 100
>          2 Node cluster with total memory as 16GB
> TestSteps:
> =========
>           Execute following 3 jobs with different memory configurations for 
> Map , reducer and AM task
>                   ./yarn jar wordcount-sleep.jar -Dmapreduce.job.queuename=a 
> -Dwordcount.map.sleep.time=2000 -Dmapreduce.map.memory.mb=2048 
> -Dyarn.app.mapreduce.am.resource.mb=1024 -Dmapreduce.reduce.memory.mb=2048 
> /dir8 /preempt_85 (application_1405414066690_0023)
>                  ./yarn jar wordcount-sleep.jar -Dmapreduce.job.queuename=b 
> -Dwordcount.map.sleep.time=2000 -Dmapreduce.map.memory.mb=2048 
> -Dyarn.app.mapreduce.am.resource.mb=2048 -Dmapreduce.reduce.memory.mb=2048 
> /dir2 /preempt_86 (application_1405414066690_0025)
>                  ./yarn jar wordcount-sleep.jar -Dmapreduce.job.queuename=c 
> -Dwordcount.map.sleep.time=2000 -Dmapreduce.map.memory.mb=1024 
> -Dyarn.app.mapreduce.am.resource.mb=1024 -Dmapreduce.reduce.memory.mb=1024 
> /dir2 /preempt_62
> Issue
> =====
>               when 2GB memory is in reserved state  totoal memory is shown as 
> 15GB and used as 15GB  ( while total memory is 16GB)

This message was sent by Atlassian JIRA

Reply via email to