[
https://issues.apache.org/jira/browse/YARN-6492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17101990#comment-17101990
]
Jonathan Hung edited comment on YARN-6492 at 5/7/20, 7:28 PM:
--------------------------------------------------------------
[[email protected]], thanks. Seems you missed uploading
PartitionQueueMetrics class.
I definitely think we should address #2, #3, and #4 in this JIRA. I don't think
#3 is addressed by YARN-9767. For example it edits the tests in the same way,
i.e. {noformat}assertEquals(10 * GB,
leafQueueA.getMetrics().getAvailableMB());{noformat} is changed to
{noformat}assertEquals(22 * GB,
leafQueueA.getMetrics().getAvailableMB());{noformat}, but this assert should
still be 0 GB, since the default partition has no resources. IMO the bottom
line is that after this JIRA is committed, the existing QueueMetrics should
still only contain metrics for default partition, and partitioned queue metrics
should only be in the newly added metrics. It will get very confusing if we
break this behavior in this JIRA and then patch it in another. What do you
think?
Also, regarding your first point in YARN-9767 about non exclusive node labels,
this issue seems to exist even before YARN-6492, so I think we can address this
issue in YARN-9767.
was (Author: jhung):
[[email protected]], thanks. Seems you missed uploading
PartitionQueueMetrics class.
I definitely think we should address #2, #3, and #4 in this JIRA. I don't think
#3 is addressed by YARN-9767. For example it edits the tests in the same way,
i.e. {noformat}assertEquals(10 * GB,
leafQueueA.getMetrics().getAvailableMB());{noformat} is changed to
{noformat}assertEquals(22 * GB,
leafQueueA.getMetrics().getAvailableMB());{noformat}, but this assert should
still be 0 GB, since the default partition has no resources. IMO the bottom
line is that after this JIRA is committed, the existing QueueMetrics should
still only contain metrics for default partition, and partitioned queue metrics
should only be in the newly added metrics. What do you think?
> Generate queue metrics for each partition
> -----------------------------------------
>
> Key: YARN-6492
> URL: https://issues.apache.org/jira/browse/YARN-6492
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacity scheduler
> Reporter: Jonathan Hung
> Assignee: Manikandan R
> Priority: Major
> Attachments: PartitionQueueMetrics_default_partition.txt,
> PartitionQueueMetrics_x_partition.txt, PartitionQueueMetrics_y_partition.txt,
> YARN-6492.001.patch, YARN-6492.002.patch, YARN-6492.003.patch,
> YARN-6492.004.patch, YARN-6492.005.WIP.patch, YARN-6492.006.WIP.patch,
> YARN-6492.007.WIP.patch, YARN-6492.008.WIP.patch, partition_metrics.txt
>
>
> We are interested in having queue metrics for all partitions. Right now each
> queue has one QueueMetrics object which captures metrics either in default
> partition or across all partitions. (After YARN-6467 it will be in default
> partition)
> But having the partition metrics would be very useful.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
