[
https://issues.apache.org/jira/browse/YARN-4538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15082564#comment-15082564
]
Bibin A Chundatt commented on YARN-4538:
----------------------------------------
[~sunilg]/[~rohithsharma]
Thank you for looking into the issue. I am not sure its the same issue.
Currently the issue looks like in {{QueueMetrics#_decrPendingResources}} and
{{QueueMetrics#_incrPendingResources}}
{noformat}
private void _decrPendingResources(int containers, Resource res) {
// if #container = 0, means change container resource
pendingContainers.decr(containers);
pendingMB.decr(res.getMemory() * Math.max(containers, 1));
pendingVCores.decr(res.getVirtualCores() * Math.max(containers, 1));
}
private void _incrPendingResources(int containers, Resource res) {
pendingContainers.incr(containers);
pendingMB.incr(res.getMemory() * containers);
pendingVCores.incr(res.getVirtualCores() * containers);
}
{noformat}
For increase and decrease the logic looks different.
{noformat}
[{Priority: 20, Capability: <memory:1024, vCores:1>, # Containers: 1, Location:
*, Relax Locality: true, Node Label Expression: }]
[{Priority: 20, Capability: <memory:1024, vCores:1>, # Containers: 0, Location:
*, Relax Locality: true, Node Label Expression: }]
{noformat}
The resource request in {{AppSchedulingInfo#updateResourceRequests}} is as
above. And causes pendingMB and PendingVcore to be negative.
When ever allocation request contains capacity as non zero value and
container=0 the the pending resource calculation will go wrong.
Looks related to YARN-1651.
I will sync {{QueueMetrics#_incrPendingResources}} also same as
{{QueueMetrics#_decrPendingResources}} and upload a patch soon
Thoughts??
> QueueMetrics pending cores and memory metrics wrong
> ----------------------------------------------------
>
> Key: YARN-4538
> URL: https://issues.apache.org/jira/browse/YARN-4538
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Bibin A Chundatt
> Assignee: Bibin A Chundatt
>
> Submit 2 application to default queue
> Check queue metrics for pending cores and memory
> {noformat}
> List<QueueInfo> allQueues = client.getChildQueueInfos("root");
> for (QueueInfo queueInfo : allQueues) {
> QueueStatistics quastats = queueInfo.getQueueStatistics();
> System.out.println(quastats.getPendingVCores());
> System.out.println(quastats.getPendingMemoryMB());
> }
> {noformat}
> *Output :*
> -20
> -20480
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
