[
https://issues.apache.org/jira/browse/YARN-6492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16977070#comment-16977070
]
Jonathan Hung commented on YARN-6492:
-------------------------------------
[[email protected]] thanks for working on this feature. I'm still going
thru the patch, I have some comments though:
* in pSourceName, how come we split partition by Q_SPLITTER? I think we don't
need to do any splitting here (there should only be one partition)
* I don't see PartitionQueueMetrics#forQueue invoked anywhere except a test
case, unless I missed it? Do we need this? The metrics registration seems to
happen in QueueMetrics#getPartitionQueueMetrics. Also, how come we call
getQueueMetrics().put(partition, metrics)? I think it should be keyed by
partition + queueName
* Do we need a separate getPartitionMetrics? Can we track a partition's
metrics via that partition + root queue?
* For setAvailableResourcesToUser - how come we add this bit?
{noformat}
if (parent != null) {
parent.setAvailableResourcesToUser(partition, user, limit);
}{noformat}
* It seems in some methods, e.g. incrPendingResources, decrPendingResources,
etc. we need to add some metrics inheritance from child queue to parent queue
(e.g, this is what's done for default partition:
{noformat}
_decrPendingResources(containers, res);
QueueMetrics userMetrics = getUserMetrics(user);
if (userMetrics != null) {
userMetrics.decrPendingResources(partition, user, containers, res);
}
if (parent != null) {
parent.decrPendingResources(partition, user, containers, res);
} {noformat}
> Generate queue metrics for each partition
> -----------------------------------------
>
> Key: YARN-6492
> URL: https://issues.apache.org/jira/browse/YARN-6492
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacity scheduler
> Reporter: Jonathan Hung
> Assignee: Manikandan R
> Priority: Major
> Attachments: PartitionQueueMetrics_default_partition.txt,
> PartitionQueueMetrics_x_partition.txt, PartitionQueueMetrics_y_partition.txt,
> YARN-6492.001.patch, YARN-6492.002.patch, YARN-6492.003.patch,
> YARN-6492.004.patch, YARN-6492.005.WIP.patch, YARN-6492.006.WIP.patch,
> partition_metrics.txt
>
>
> We are interested in having queue metrics for all partitions. Right now each
> queue has one QueueMetrics object which captures metrics either in default
> partition or across all partitions. (After YARN-6467 it will be in default
> partition)
> But having the partition metrics would be very useful.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
