[
https://issues.apache.org/jira/browse/YARN-4484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated YARN-4484:
--------------------------
Attachment: 0004-YARN-4484.patch
Hi [~leftnoteasy]
Thank you for sharing the comments.
bq.a queue has >0 available resource from default partition and another
partition, when container allocated/released, we should modify available
resource from both partitions
I have test case named {{testQueueMetricsWithLabelsOnDefaultLabelNode}} for
handling scenario where we has Default and another label configured for one
Queue. And it shares few resource from default-partition and from partition=x.
And I am getting expected results.
To clear out the scenario:
1. 1 node with 10GB for label "x" and another node for "default_partition"
2. Allocated 1GB AM container from node2 ("default_partition")
3. Allocated 5 * 1GB containers from node1 (label = x)
4. Allocate 1GB container from node2 ("default_partition")
Now capacity distribution is as follows.
QueueA
---> label X to take 50%
---> Default Partition to take 25%
So QueueA can get 5Gb from Node1(partition x) and 2.5GB from
(default_partition).
Test verifies below for Queue A (metrics)
Total Available resource is 0.5GB (this from node2)
Total Allocated resource is 7GB
This metric calculation is updated only on QueueMetrics. For label, we are
already handling in QueuCapacity. Ref:: {{CSQueueUtils#updateUsedCapacity}}
Updating test case by adding more clarity and rebasing. Could you please help
to check the same.
> Available Resource calculation for a queue is not correct when used with
> labels
> -------------------------------------------------------------------------------
>
> Key: YARN-4484
> URL: https://issues.apache.org/jira/browse/YARN-4484
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler
> Affects Versions: 2.7.1
> Reporter: Sunil G
> Assignee: Sunil G
> Attachments: 0001-YARN-4484.patch, 0002-YARN-4484.patch,
> 0003-YARN-4484-v2.patch, 0003-YARN-4484.patch, 0004-YARN-4484.patch
>
>
> To calculate available resource for a queue, we have to get the total
> resource allocated for all labels in queue compare to its usage.
> Also address the comments given in
> [YARN-4304-comments|https://issues.apache.org/jira/browse/YARN-4304?focusedCommentId=15064874&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15064874
> ] given by [~leftnoteasy] for same.
> ClusterMetrics related issues will also get handled once we fix this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
