[
https://issues.apache.org/jira/browse/HBASE-6261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13416680#comment-13416680
]
Andrew Wang commented on HBASE-6261:
------------------------------------
Sorry I haven't had time to push on this more. I talked with Jon Hsieh last
week about doing a more convincing analysis of the performance of the new
MutableQuantiles class from HADOOP-8541 vs the existing reservoir-sampling
histogram method. I'll try to get that done within a week.
I'm also not sure about the right course of action at getting it used in HBase.
Stack indicated way back on the mailing list that he was okay waiting for a
hadoop-common version bump, which is kind of a long timescale. If people really
urgently want this, we could just copy the code over and then refactor it away
when it's released in hadoop-common.
> Better approximate high-percentile percentile latency metrics
> -------------------------------------------------------------
>
> Key: HBASE-6261
> URL: https://issues.apache.org/jira/browse/HBASE-6261
> Project: HBase
> Issue Type: New Feature
> Reporter: Andrew Wang
> Assignee: Andrew Wang
> Labels: metrics
> Attachments: Latencyestimation.pdf
>
>
> The existing reservoir-sampling based latency metrics in HBase are not
> well-suited for providing accurate estimates of high-percentile (e.g. 90th,
> 95th, or 99th) latency. This is a well-studied problem in the literature (see
> [1] and [2]), the question is determining which methods best suit our needs
> and then implementing it.
> Ideally, we should be able to estimate these high percentiles with minimal
> memory and CPU usage as well as minimal error (e.g. 1% error on 90th, or .1%
> on 99th). It's also desirable to provide this over different time-based
> sliding windows, e.g. last 1 min, 5 mins, 15 mins, and 1 hour.
> I'll note that this would also be useful in HDFS, or really anywhere latency
> metrics are kept.
> [1] http://www.cs.rutgers.edu/~muthu/bquant.pdf
> [2] http://infolab.stanford.edu/~manku/papers/04pods-sliding.pdf
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
