[
https://issues.apache.org/jira/browse/HADOOP-8706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13436554#comment-13436554
]
Ming Ma commented on HADOOP-8706:
---------------------------------
@Aaron, It seems MutableRate is used to collect things like latency. It
generates a bunch of derived metrics such as mean and stand deviation. It
doesn't seem to target ops / sec, although the name has rate in it. MutableRate
seems to be similar to MetricsTimeVaryingRate in metrics 1.0.
@Andy. We use ganglia. Do you know if there is a way to change ganglia to
calculate derivative of any given metrics or something similar?
Some more background. Before we applied this fix to MetricsTimeVaryingLong in
metrics 1.0 in our internal branch, we find it hard to understand why NN in a
shared cluster sometimes became really slow, later we understood there are some
bad client code doing lots of NN operation; the monotonic metric of NN file
operations aren't obvious in ganglia. After the fix is applied to production
cluster, it makes root cause analysis much easier.
> Provide rate metrics based on counter value
> -------------------------------------------
>
> Key: HADOOP-8706
> URL: https://issues.apache.org/jira/browse/HADOOP-8706
> Project: Hadoop Common
> Issue Type: Improvement
> Components: metrics
> Reporter: Ming Ma
> Attachments: HADOOP-8706.patch
>
>
> In production clusters, it is more useful to have ops / sec instead of
> increasing counter value. Take NameNodeMetrics.getBlockLocations as an
> example, its current type is MutableCounterLong and thus the value is
> increasing all the time. Quite often "num of getBlockLocations" per second is
> more interesting for analysis. Further I found most of the MutableCounterLong
> in NamenodeMetrics and DataNodeMetrics are more useful if they are expressed
> in terms of ops / sec.
> I looked at all the metrics objects provided in metrics 2.0, couldn't find
> such type.
> FYI, hbase has its own MetricsRate object based on metrics 1.0 for this
> purpose.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
