[ 
https://issues.apache.org/jira/browse/YARN-2305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14065238#comment-14065238
 ] 

Sunil G commented on YARN-2305:
-------------------------------

1. Yes [~leftnoteasy], GUI display of 106% is similar to YARN-2285. It can be 
tackled there.
2. 

As mentioned in earlier comment, Total MB in GUI is internally sum of 
availableMB+allottedMB.

a. *LeafQueue#usedResources* is sum of used and reserved memory.  
But *CSQueueUtils#updateQueueStatistics()* code may give a -ve value in case of 
reservation which sets availableMB in QueueMetrics.
{code}
Resource available = Resources.subtract(queueLimit, usedResources);
{code}

If this comes as -ve, then *availableMB* is set as 0.

b. *allocatedMB*: This is set when a container is really allocated. This is the 
real queue usage.

In above scenario, it should have come as 
{noformat}15(availableMb)+1(allocatedMB)=16{noformat}
But due to reservation, allocatedMB became 0. Hence total shown as 15.
I feel instead of showing Total as *allocated+available*, we can show 
*clusterResource* here. Any particular reason why we need like 
allocated+available, thoughts?


> When a container is in reserved state then total cluster memory is displayed 
> wrongly.
> -------------------------------------------------------------------------------------
>
>                 Key: YARN-2305
>                 URL: https://issues.apache.org/jira/browse/YARN-2305
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: resourcemanager
>    Affects Versions: 2.4.1
>            Reporter: J.Andreina
>            Assignee: Sunil G
>         Attachments: Capture.jpg
>
>
> ENV Details:
> =========  
>          3 queues  :  a(50%),b(25%),c(25%) ---> All max utilization is set to 
> 100
>          2 Node cluster with total memory as 16GB
> TestSteps:
> =========
>           Execute following 3 jobs with different memory configurations for 
> Map , reducer and AM task
>                   ./yarn jar wordcount-sleep.jar -Dmapreduce.job.queuename=a 
> -Dwordcount.map.sleep.time=2000 -Dmapreduce.map.memory.mb=2048 
> -Dyarn.app.mapreduce.am.resource.mb=1024 -Dmapreduce.reduce.memory.mb=2048 
> /dir8 /preempt_85 (application_1405414066690_0023)
>                  ./yarn jar wordcount-sleep.jar -Dmapreduce.job.queuename=b 
> -Dwordcount.map.sleep.time=2000 -Dmapreduce.map.memory.mb=2048 
> -Dyarn.app.mapreduce.am.resource.mb=2048 -Dmapreduce.reduce.memory.mb=2048 
> /dir2 /preempt_86 (application_1405414066690_0025)
>  
>                  ./yarn jar wordcount-sleep.jar -Dmapreduce.job.queuename=c 
> -Dwordcount.map.sleep.time=2000 -Dmapreduce.map.memory.mb=1024 
> -Dyarn.app.mapreduce.am.resource.mb=1024 -Dmapreduce.reduce.memory.mb=1024 
> /dir2 /preempt_62
> Issue
> =====
>               when 2GB memory is in reserved state  totoal memory is shown as 
> 15GB and used as 15GB  ( while total memory is 16GB)
>  



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Reply via email to