HBase has some nice metrics stuff that's not part of hadoop-common, including a nifty reservoir sampling histogram / stats class. Would definitely be nice to pull in.
There's also the MutableQuantiles class in hadoop-common if you want to calculate a more accurate histogram. If you're looking for yet more metrics, Coda Hale's library comes recommended: http://metrics.codahale.com/ On Fri, May 10, 2013 at 9:46 AM, Steve Loughran <ste...@hortonworks.com>wrote: > On 9 May 2013 20:39, <sya...@stevendyates.com> wrote: > > > > >> Unless there are existing bits of this stuff lurking somewhere in the > >> Hadoop codebase that I haven't noticed, these could be copied into > hadoop > >> core. Reviewing the code as it is would be welcome > >> > >> https://github.com/**steveloughran/hadoop-trunk/** > >> tree/stevel/HADOOP-8545-swift/**hadoop-tools/hadoop-openstack/** > >> src/main/java/org/apache/**hadoop/fs/swift/util< > https://github.com/steveloughran/hadoop-trunk/tree/stevel/HADOOP-8545-swift/hadoop-tools/hadoop-openstack/src/main/java/org/apache/hadoop/fs/swift/util > > > >> > >> > > Thank you Steve, if you don't mind me taking a look at and possibly using > > your stats classes with this issue that would be great. > > > > I will review and get back in touch > > > > great! > > This is what a post-test-run dump of the stats looks like > 2013-05-09 16:35:19,878 [main] INFO HEAD getObjectMetadata-newest count=3 > total=0.732s mean=0.244s stddev=0.000s min=0.171s max=0.287s > 2013-05-09 16:35:19,879 [main] INFO DELETE count=10 total=4.594s > mean=0.459s stddev=0.001s min=0.188s max=2.526s > 2013-05-09 16:35:19,879 [main] INFO PUT count=1 total=6.253s mean=6.253s > stddev=0.000s min=6.253s max=6.253s > 2013-05-09 16:35:19,880 [main] INFO GET findObjects count=2 total=0.443s > mean=0.222s stddev=0.000s min=0.220s max=0.223s > 2013-05-09 16:35:19,880 [main] INFO HEAD createContainer count=1 > total=0.324s mean=0.324s stddev=0.000s min=0.324s max=0.324s > 2013-05-09 16:35:19,881 [main] INFO HEAD getObjectMetadata-newest-FAIL > count=2 total=10.755s mean=5.378s stddev=0.007s min=0.178s max=10.577s > 2013-05-09 16:35:19,881 [main] INFO PUT upload count=9 total=2.546s > mean=0.283s stddev=0.000s min=0.216s max=0.377s > 2013-05-09 16:35:19,881 [main] INFO POST authentication count=1 > total=1.096s mean=1.096s stddev=0.000s min=1.096s max=1.096s >
