tomicooler commented on PR #5293:
URL: https://github.com/apache/hadoop/pull/5293#issuecomment-1618880984
Hi,
I can help in reviewing the code, but I'm not familiar with ClusterMetrics
or FairScheduler.
```
2. Currently, ClusterMetricsInfo shows available or allocated memory only
for Default Partition. I don't think this is by intention, if so there has to
be at least one test case failed with this patch. From the name
ClusterMetricsInfo, it is Cluster Wide, showing only for Default Partition does
not look right and also does not help clusters where some/all nodes are part of
non-default partitions.
```
Agreed. I also think, that we should add a new test case that validates the
behaviour.
E.g.: in the `TestRMWebServicesCapacitySched`:
```
@Test
public void testClusterMetrics() throws Exception {
try (MockRM rm = createRM(createConfigWithNodeLabels())){
rm.registerNode("h1:1234", 32 * GB, 32); // multiple nodes with some
labels that makes sense
assertJsonResponse(resource().path("ws/v1/cluster/metrics")
.accept(MediaType.APPLICATION_JSON).get(ClientResponse.class),
"webapp/scheduler-response.json");
}
}
```
```
3. Without this patch, currently i see there is no way to retrieve the
Cluster Wide Available and Allocated Memory. But we have ways to get it for
Partition level already through QueueMetrics . Having both ClusterMetrics and
QueueMetrics showing for Default Partition seems redundant.
```
I think the `availableMB` and `allocatedMB` can be calculated from the
`ClusterMetrics` for CapacityScheduler without this change, e.g.:
```
availableMB = totalClusterResourcesAcrossPartition .getMemorySize() -
totalUsedResourcesAcrossPartition.getMemorySize()
allocatedMB = totalUsedResourcesAcrossPartition.getMemorySize()
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
