[
https://issues.apache.org/jira/browse/YARN-6492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16977625#comment-16977625
]
Manikandan R commented on YARN-6492:
------------------------------------
[~jhung] Thanks for your review.
{quote}in pSourceName, how come we split partition by Q_SPLITTER? I think we
don't need to do any splitting here (there should only be one partition){\quote}
Yes, there is no need for split as there is going to be only one partition.
However, we can keep the method pSourceName just to generate strings like
"partition=x" instead of "p0=x" from readability perspective.
{quote}I don't see PartitionQueueMetrics#forQueue invoked anywhere except a
test case, unless I missed it? Do we need this?{\quote}
Yes, test case is the only place. But idea is to use the method whenever any
new partitions gets added in the future. As soon as any partition gets added,
we might need to generate the metrics for all partition * queue combinations.
Hence "public static" method has been introduced. However, "Adding New
partition" code flow need to be revisit to ensure behaviour would be smooth.
For example, What would be metrics value for newly added partition till
corresponding mapped queue resource has been utilized?
{quote}The metrics registration seems to happen in
QueueMetrics#getPartitionQueueMetrics. Also, how come we call
getQueueMetrics().put(partition, metrics)? I think it should be keyed by
partition + queueName{\quote}
PartitionQueueMetrics#forQueue registration happens then and there.
QueueMetrics#getPartitionQueueMetrics has been used to register partition *
queue metrics as part of any metrics computation.
QueueMetrics#getPartitionMetrics has been used to register partition metrics as
part of any metrics computation.
{quote}Do we need a separate getPartitionMetrics? Can we track a partition's
metrics via that partition + root queue?{\quote}
Seems to be better option. I will think through.
{quote}For setAvailableResourcesToUser - how come we add this bit?{\quote}
Are you saying, that piece of code would end up in doubling up user available
value as we navigate the queue up from the bottom of the queue path?
{quote}maybe we need to overwrite the parent field in QueueMetrics inside
PartitionQueueMetrics?{\quote}
Sorry, didn't understand this. Can you explain in detail?
> Generate queue metrics for each partition
> -----------------------------------------
>
> Key: YARN-6492
> URL: https://issues.apache.org/jira/browse/YARN-6492
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacity scheduler
> Reporter: Jonathan Hung
> Assignee: Manikandan R
> Priority: Major
> Attachments: PartitionQueueMetrics_default_partition.txt,
> PartitionQueueMetrics_x_partition.txt, PartitionQueueMetrics_y_partition.txt,
> YARN-6492.001.patch, YARN-6492.002.patch, YARN-6492.003.patch,
> YARN-6492.004.patch, YARN-6492.005.WIP.patch, YARN-6492.006.WIP.patch,
> partition_metrics.txt
>
>
> We are interested in having queue metrics for all partitions. Right now each
> queue has one QueueMetrics object which captures metrics either in default
> partition or across all partitions. (After YARN-6467 it will be in default
> partition)
> But having the partition metrics would be very useful.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
