[ https://issues.apache.org/jira/browse/HADOOP-8541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13412803#comment-13412803 ]
Hudson commented on HADOOP-8541: -------------------------------- Integrated in Hadoop-Mapreduce-trunk #1134 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1134/]) HADOOP-8541. Better high-percentile latency metrics. Contributed by Andrew Wang. (Revision 1360501) Result = SUCCESS atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1360501 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/dev-support/findbugsExcludeFile.xml * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/MetricsRegistry.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/MutableQuantiles.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/util/Quantile.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/util/SampleQuantiles.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/metrics2/lib/TestMutableMetrics.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/metrics2/util/TestSampleQuantiles.java > Better high-percentile latency metrics > -------------------------------------- > > Key: HADOOP-8541 > URL: https://issues.apache.org/jira/browse/HADOOP-8541 > Project: Hadoop Common > Issue Type: Improvement > Components: metrics > Affects Versions: 2.0.0-alpha > Reporter: Andrew Wang > Assignee: Andrew Wang > Fix For: 2.0.1-alpha > > Attachments: hadoop-8541-1.patch, hadoop-8541-2.patch, > hadoop-8541-3.patch, hadoop-8541-4.patch, hadoop-8541-5.patch, > hadoop-8541-6.patch > > > Based on discussion in HBASE-6261 and with some HDFS devs, I'd like to make > better high-percentile latency metrics a part of hadoop-common. > I've already got a working implementation of [1], an efficient algorithm for > estimating quantiles on a stream of values. It allows you to specify > arbitrary quantiles to track (e.g. 50th, 75th, 90th, 95th, 99th), along with > tight error bounds. This estimator can be snapshotted and reset periodically > to get a feel for how these percentiles are changing over time. > I propose creating a new MutableQuantiles class that does this. [1] isn't > completely without overhead (~1MB memory for reasonably sized windows), which > is why I hesitate to add it to the existing MutableStat class. > [1] Cormode, Korn, Muthukrishnan, and Srivastava. "Effective Computation of > Biased Quantiles over Data Streams" in ICDE 2005. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira