[
https://issues.apache.org/jira/browse/HADOOP-7630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13107541#comment-13107541
]
Eric Yang commented on HADOOP-7630:
-----------------------------------
Luke, It used to be 5 seconds blurb rate (Simon Plugin) and 60 seconds
reporting rate (Simon Aggregator). Simon Aggregator calculates the wavelet
transformation of the output samples. Unfortunately, Simon plugin udp datagram
does not scale well for having large emission sources. Instead of losing blurb
(udp datagram), I changed the blurb rate to 60 seconds around August 2008 on Y!
clusters. The Simon plugin is only doing add and average of samples, it does
not have discrete wavelet transformation algorithm built-in. Consequently, It
is better to have the publish/subscribe rate to be the same as the blurb
emission rate so the result is more accurate. More averaging computed at the
MetricsSource, the less precision is retained at the source.
Are you concerning that the metrics might overflow if the publish rate is at 60
seconds? In 2008, I had also audited all metrics are using double to prevent
overflow in short fraction of time. There are few metrics added, but the new
metrics do not look like they would overflow neither. As a side benefit, by
reducing the period, it is less amount of cycle spend in metrics monitoring,
which makes the system more efficient.
> hadoop-metrics2.properties should have a property *.period set to a default
> value foe metrics
> ---------------------------------------------------------------------------------------------
>
> Key: HADOOP-7630
> URL: https://issues.apache.org/jira/browse/HADOOP-7630
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Reporter: Arpit Gupta
> Assignee: Eric Yang
> Fix For: 0.20.205.0, 0.23.0
>
> Attachments: HADOOP-7630-trunk.patch, HADOOP-7630.patch
>
>
> currently the hadoop-metrics2.properties file does not have a value set for
> *.period
> This property is useful for metrics to determine when the property will
> refresh. We should set it to default of 60
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
