[
https://issues.apache.org/jira/browse/YARN-7728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324165#comment-16324165
]
Eric Payne commented on YARN-7728:
----------------------------------
Thanks a lot for the comments, [~sunilg].
bq. n 3.0, we support multiple types, and this covers only cpu and memory. So
could we cover preemption metrics also in case of multi resources.
I agree with this in principle. However, I made a conscious decision not to do
this. There are a couple of difficulties that I see. First, this is not done
for other resource metrics in QueueMetrics (or any of the other system metrics
I could find). The resource metrics only cover memory and vcores. Second,
making the metric names match the resource names is a little difficult if the
resource names could be dynamic. Because of these two things, I feel that
solving this should be done all at the same time in a more general JIRA.
{quote}
One more doubt is with aggregateVcoreSecondsPreempted. MutableCounterLong is
used for this. But under one queue, we ll have multiple containers gets
preempted and each container resource size vary drastically. So are we looking
for an aggregate resource among all preempted containers in a given time ?
{quote}
I don't think I understand the question. The metrics are updated when each
container is preempted, and the value keeps increasing over time. Similar to
memory, it's basically a metric of total lost (virtual) cpu cycles due to
preemption since the RM was started.
{quote}
aggregateMegabyteSecondsPreempted: MegaByte seems a bit confusing, MemoryMB is
used in another places as well. Could we use something similar (like prepending
memory)
{quote}
Good point. I will update a new patch.
> Expose and expand container preemptions in Capacity Scheduler queue metrics
> ---------------------------------------------------------------------------
>
> Key: YARN-7728
> URL: https://issues.apache.org/jira/browse/YARN-7728
> Project: Hadoop YARN
> Issue Type: Improvement
> Affects Versions: 2.9.0, 2.8.3, 3.0.0
> Reporter: Eric Payne
> Assignee: Eric Payne
> Attachments: YARN-7728.001.patch
>
>
> YARN-1047 exposed queue metrics for the number of preempted containers to the
> fair scheduler. I would like to also expose these to the capacity scheduler
> and add metrics for the amount of lost memory seconds and vcore seconds.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
