[
https://issues.apache.org/jira/browse/YARN-6492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16094003#comment-16094003
]
Naganarasimha G R commented on YARN-6492:
-----------------------------------------
Thanks [[email protected]] for the patch, and sorry for the delay in
responding.
I had following high level comments on the approach and scenarios covered:
# User metrics needs to be updated under the partition Queue metric. As we need
to capture for each queue under a partition how much resource has been utlized
by a user.
# Sorry to retract on the approach, when going through the code i was fealing
the approach of the Queue structure is little different from how its displayed
in the ui and processed during scheduling as well. Currently we are trying to
capture within a queue whats the partition queue's queue metrics. IMO it should
be for each partition we should have the queue hierarchy i.e the existing
QueueMetrics with user metrics. This would give a better structure
earlier planned approach
{code}
QueueMetric
metrics
Usermetrics
Partition Metrics
UserMetrics
childQueueMetrics
{code}
IMO it should be some thing like
{code}
Partitions
PartitionMetric
QueueMetric
metrics
Usermetrics
childQueueMetrics
PartitionMetric
QueueMetric
...
Partitions
QueueMetric //existing for default metrics to maintain compatability
metrics
Usermetrics
{code}
But at the same time we should not break the compatability. Thoughts?
May be you can attach the existing structure for a simple queue hierarchy for
others to visualize.
Thoughts [~sunilg] , [~jlowe] ?
Implementation
# Ensure the metrics are updated up the tree hierarchy from the leaf till the
root on every update on partition metric. I think its taken care but just add
test and ensure in manual testing too
# QueueMetrics.java : New overloaded constructor can make a call to existing
constructor internally with required arguments instead of both methods doing
the update.
Would go through the patch in more detail once the approach gets finalized.
> Generate queue metrics for each partition
> -----------------------------------------
>
> Key: YARN-6492
> URL: https://issues.apache.org/jira/browse/YARN-6492
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacity scheduler
> Reporter: Jonathan Hung
> Assignee: Manikandan R
> Attachments: YARN-6492.001.patch
>
>
> We are interested in having queue metrics for all partitions. Right now each
> queue has one QueueMetrics object which captures metrics either in default
> partition or across all partitions. (After YARN-6467 it will be in default
> partition)
> But having the partition metrics would be very useful.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
