[
https://issues.apache.org/jira/browse/YARN-9457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szilard Nemeth updated YARN-9457:
---------------------------------
Component/s: fairscheduler
> Integrate custom resource metrics better for FairScheduler
> ----------------------------------------------------------
>
> Key: YARN-9457
> URL: https://issues.apache.org/jira/browse/YARN-9457
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: fairscheduler
> Reporter: Szilard Nemeth
> Priority: Major
>
> YARN-8842 added
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetricsForCustomResources.
> This class stores all metrics data for custom resource types.
> A field is there in QueueMetrics to hold an object of this class.
> Similarly, YARN-9322 added FSQueueMetricsForCustomResources and added an
> object of this class to FSQueueMetrics.
> This jira is about to investigate how it is possible to integrate
> QueueMetricsForCustomResources into QueueMetrics and
> FSQueueMetricsForCustomResources into FSQueueMetrics.
> The trick is that the Metrics annotation
> (org.apache.hadoop.metrics2.annotation.Metric) is used to expose values on
> JMX.
> We need to implement a mechanism where QueueMetrics / FSQueueMetrics classes
> do contain a field of the custom resource values which is a map of resource
> names as keys, and longs as values.
> This way, we don't need the new classes (QueueMetricsForCustomResources and
> FSQueueMetricsForCustomResources), the code could be much cleaner and
> consistent.
> The hardest part possibly is to find a way to expose metrics values from a
> map. We obviously can't use the Metrics annotation so a mechanism is required
> to expose the values on JMX.
> For a quick search, I haven't found any way like this in the code
> [~wilfreds]: Are you aware of any way to expose values like this?
> Most probably, we need to check how the Metrics annotation is processed,
> understand the whole flow and check what is the underlying mechanism of the
> metrics propagation to the JMX interface.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
