[ 
https://issues.apache.org/jira/browse/YARN-4538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15082564#comment-15082564
 ] 

Bibin A Chundatt commented on YARN-4538:
----------------------------------------

[~sunilg]/[~rohithsharma]

Thank you for looking into the issue. I am not sure its the same issue. 
Currently the issue looks like in {{QueueMetrics#_decrPendingResources}} and 
{{QueueMetrics#_incrPendingResources}}

{noformat}
  private void _decrPendingResources(int containers, Resource res) {
    // if #container = 0, means change container resource
    pendingContainers.decr(containers);
    pendingMB.decr(res.getMemory() * Math.max(containers, 1));
    pendingVCores.decr(res.getVirtualCores() * Math.max(containers, 1));
  }

  private void _incrPendingResources(int containers, Resource res) {
    pendingContainers.incr(containers);
    pendingMB.incr(res.getMemory() * containers);
    pendingVCores.incr(res.getVirtualCores() * containers);
  }
{noformat}

For increase and decrease the logic looks different.

{noformat}
[{Priority: 20, Capability: <memory:1024, vCores:1>, # Containers: 1, Location: 
*, Relax Locality: true, Node Label Expression: }]  
[{Priority: 20, Capability: <memory:1024, vCores:1>, # Containers: 0, Location: 
*, Relax Locality: true, Node Label Expression: }]  
{noformat}

The resource request in {{AppSchedulingInfo#updateResourceRequests}} is as 
above. And causes pendingMB and PendingVcore to be negative.
When ever allocation request contains capacity as non zero value and 
container=0 the the pending resource calculation will go wrong.

Looks related to YARN-1651. 
I will sync {{QueueMetrics#_incrPendingResources}} also same as 
{{QueueMetrics#_decrPendingResources}} and upload a patch soon

Thoughts??

> QueueMetrics pending  cores and memory metrics wrong
> ----------------------------------------------------
>
>                 Key: YARN-4538
>                 URL: https://issues.apache.org/jira/browse/YARN-4538
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Bibin A Chundatt
>            Assignee: Bibin A Chundatt
>
> Submit 2 application to default queue 
> Check queue metrics for pending cores and memory
> {noformat}
>     List<QueueInfo> allQueues = client.getChildQueueInfos("root");
>     for (QueueInfo queueInfo : allQueues) {
>       QueueStatistics quastats = queueInfo.getQueueStatistics();
>       System.out.println(quastats.getPendingVCores());
>       System.out.println(quastats.getPendingMemoryMB());
>     }
> {noformat}
> *Output :*
> -20
> -20480



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to