[
https://issues.apache.org/jira/browse/YARN-6492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16989920#comment-16989920
]
Manikandan R commented on YARN-6492:
------------------------------------
{quote}Otherwise metrics for default partition will be updated when this is
called for non-null partition.
{quote}
No. When _allocateResources has been called for very first time (from other
classes), it does the metrics computation only for that corresponding LeafQueue
(using CSQueueMetrics object), nothing to do with partition metrics at this
moment. Partition * Queue metrics and Partition metrics computation will come
into picture only when QueueMetrics#getPartitionQueueMetrics and
PartitionQueueMetrics#getPartitionQueueMetrics is triggered respectively inside
alllocateResources().
Partition * Queue metrics intent is to derive
{code:java}
"name" :
"Hadoop:service=ResourceManager,name=PartitionQueueMetrics,partition=x,q0=root,q1=a"
...{code}
in JMX o/p.
Partition metrics intent is to derive
{code:java}
"name" :
"Hadoop:service=ResourceManager,name=PartitionQueueMetrics,partition=x"
...{code}
in JMX o/p.
QueueMetrics#getPartitionQueueMetrics purpose is to register the Partition *
Queue metrics into metric system , whereas,
PartitionQueueMetrics#getPartitionQueueMetrics purpose is to register Partition
metrics into metric system.
Below line in allocateResources() calls appropriate method implementation of
two different class.
{code:java}
QueueMetrics partitionQueueMetrics = getPartitionQueueMetrics(partition);{code}
Current flow in patch is, QueueMetrics#getPartitionQueueMetrics is triggered
first to compute metrics for Partition * Queue metrics and when control reaches
getPartitionQueueMetrics, it triggers getPartitionQueueMetrics method of
PartitionQueueMetrics class and takes care of only partition metrics object
creation and registration.
As of now, codebase consider empty string or null as "default" partition. So,
to make this clear and to avoid confusion, we are using "default" for empty
string/null/default partition while registering into the metric system. For an
instance,
{code:java}
"name" :
"Hadoop:service=ResourceManager,name=PartitionQueueMetrics,partition=default"
...{code}
and
{code:java}
"name" :
"Hadoop:service=ResourceManager,name=PartitionQueueMetrics,partition=default,q0=root,q1=a"
...{code}
...
In addition,
{code:java}
"name" : "Hadoop:service=ResourceManager,name=QueueMetrics,q0=root,q1=a"
...{code}
also has been retained to provide backward compatibility.
Hope this answers your first 3 comments. Please refer earlier comments
(especially,
https://issues.apache.org/jira/browse/YARN-6492?focusedCommentId=16113219&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16113219)
to visualise the JMX o/p structure.
#4 can be addressed.
> Generate queue metrics for each partition
> -----------------------------------------
>
> Key: YARN-6492
> URL: https://issues.apache.org/jira/browse/YARN-6492
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacity scheduler
> Reporter: Jonathan Hung
> Assignee: Manikandan R
> Priority: Major
> Attachments: PartitionQueueMetrics_default_partition.txt,
> PartitionQueueMetrics_x_partition.txt, PartitionQueueMetrics_y_partition.txt,
> YARN-6492.001.patch, YARN-6492.002.patch, YARN-6492.003.patch,
> YARN-6492.004.patch, YARN-6492.005.WIP.patch, YARN-6492.006.WIP.patch,
> YARN-6492.007.WIP.patch, partition_metrics.txt
>
>
> We are interested in having queue metrics for all partitions. Right now each
> queue has one QueueMetrics object which captures metrics either in default
> partition or across all partitions. (After YARN-6467 it will be in default
> partition)
> But having the partition metrics would be very useful.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
