[
https://issues.apache.org/jira/browse/YARN-11414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17739689#comment-17739689
]
ASF GitHub Bot commented on YARN-11414:
---------------------------------------
tomicooler commented on PR #5293:
URL: https://github.com/apache/hadoop/pull/5293#issuecomment-1618880984
Hi,
I can help in reviewing the code, but I'm not familiar with ClusterMetrics
or FairScheduler.
```
2. Currently, ClusterMetricsInfo shows available or allocated memory only
for Default Partition. I don't think this is by intention, if so there has to
be at least one test case failed with this patch. From the name
ClusterMetricsInfo, it is Cluster Wide, showing only for Default Partition does
not look right and also does not help clusters where some/all nodes are part of
non-default partitions.
```
Agreed. I also think, that we should add a new test case that validates the
behaviour.
E.g.: in the `TestRMWebServicesCapacitySched`:
```
@Test
public void testClusterMetrics() throws Exception {
try (MockRM rm = createRM(createConfigWithNodeLabels())){
rm.registerNode("h1:1234", 32 * GB, 32); // multiple nodes with some
labels that makes sense
assertJsonResponse(resource().path("ws/v1/cluster/metrics")
.accept(MediaType.APPLICATION_JSON).get(ClientResponse.class),
"webapp/scheduler-response.json");
}
}
```
```
3. Without this patch, currently i see there is no way to retrieve the
Cluster Wide Available and Allocated Memory. But we have ways to get it for
Partition level already through QueueMetrics . Having both ClusterMetrics and
QueueMetrics showing for Default Partition seems redundant.
```
I think the `availableMB` and `allocatedMB` can be calculated from the
`ClusterMetrics` for CapacityScheduler without this change, e.g.:
```
availableMB = totalClusterResourcesAcrossPartition .getMemorySize() -
totalUsedResourcesAcrossPartition.getMemorySize()
allocatedMB = totalUsedResourcesAcrossPartition.getMemorySize()
```
> ClusterMetricsInfo shows wrong availableMB when node labels enabled
> --------------------------------------------------------------------
>
> Key: YARN-11414
> URL: https://issues.apache.org/jira/browse/YARN-11414
> Project: Hadoop YARN
> Issue Type: Bug
> Affects Versions: 3.3.3
> Reporter: Prabhu Joseph
> Assignee: Ashutosh Gupta
> Priority: Major
> Labels: pull-request-available
>
> ClusterMetricsInfo shows wrong availableMB when node labels enabled. It shows
> availableMB of Default Partition alone.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
