[
https://issues.apache.org/jira/browse/HBASE-9243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexandre Normand updated HBASE-9243:
-------------------------------------
Attachment: HBASE-9243-1.patch
I overlooked the fact that the {{ConsoleReporter}} doesn't print the histogram
count by default. I just added it.
I also realized something while doing that: the count of Key length actually
represented the count of key/value pairs so it was misrepresenting the count of
keys (even though the key length was still accurate). I changed this so that
the count of "Key length" accurately represents the number of keys.
Updated example (actually, it's against an entirely different hfile but you get
the idea):
{code}
Stats: Key length:
min = 29.00
max = 29.00
mean = 29.00
stddev = 0.00
median = 29.00
75% <= 29.00
95% <= 29.00
98% <= 29.00
99% <= 29.00
99.9% <= 29.00
count <= 15
Row size (bytes):
min = 6921.00
max = 455286.00
mean = 320933.67
stddev = 160821.04
median = 346302.00
75% <= 455190.00
95% <= 455286.00
98% <= 455286.00
99% <= 455286.00
99.9% <= 455286.00
count <= 15
Row size (columns):
min = 110.00
max = 7595.00
mean = 5352.73
stddev = 2685.21
median = 5775.00
75% <= 7594.00
95% <= 7595.00
98% <= 7595.00
99% <= 7595.00
99.9% <= 7595.00
count <= 15
Val length:
min = 19.00
max = 55.00
mean = 22.96
stddev = 1.48
median = 23.00
75% <= 23.00
95% <= 23.00
98% <= 23.00
99% <= 23.00
99.9% <= 23.00
count <= 80291
Key of biggest row: \x7F\x80\x00\x00[\x80\x00\x00\x00\x00\x00\x00\x0A
{code}
And Jean-Marc, about the run/shutdown sequence, I think I've got it right. The
Reporter needs to run (to do the actual output to the ByteArrayOutputStream)
and shutdown is just tidying up before terminating. There's no
{{MetricsRegistry.run}}.
> Add more useful statistics in the HFile tool
> --------------------------------------------
>
> Key: HBASE-9243
> URL: https://issues.apache.org/jira/browse/HBASE-9243
> Project: HBase
> Issue Type: Improvement
> Components: HFile
> Affects Versions: 0.96.0
> Reporter: Alexandre Normand
> Priority: Minor
> Labels: newbie
> Attachments: HBASE-9243-1.patch, HBASE-9243.patch
>
>
> The [HFile tool|http://hbase.apache.org/book/regions.arch.html#hfile_tool]
> has been very useful to us recently to get a better idea of the size of our
> rows. However, it happened frequently that we wished for more statistics to
> have a more complete picture of the distribution of the row sizes.
> [~skuehn] requested that feature often enough in private that I decided to
> give it a go.
> Here's the patch that adds more nice little stats via yammer's histograms. It
> was easy enough since {{com.yammer.metrics}} is already in hbase's
> dependencies.
> Example of the new output from {{org.apache.hadoop.hbase.io.hfile.HFile -s -f
> ...}}:
> {code}
> Stats:
> Key length:
> min = 24.00
> max = 24.00
> mean = 24.00
> stddev = 0.00
> median = 24.00
> 75% <= 24.00
> 95% <= 24.00
> 98% <= 24.00
> 99% <= 24.00
> 99.9% <= 24.00
> Row size (bytes):
> min = 33.00
> max = 33.00
> mean = 33.00
> stddev = 0.00
> median = 33.00
> 75% <= 33.00
> 95% <= 33.00
> 98% <= 33.00
> 99% <= 33.00
> 99.9% <= 33.00
> Row size (columns):
> min = 1.00
> max = 1.00
> mean = 1.00
> stddev = 0.00
> median = 1.00
> 75% <= 1.00
> 95% <= 1.00
> 98% <= 1.00
> 99% <= 1.00
> 99.9% <= 1.00
> Val length:
> min = 1.00
> max = 1.00
> mean = 1.00
> stddev = 0.00
> median = 1.00
> 75% <= 1.00
> 95% <= 1.00
> 98% <= 1.00
> 99% <= 1.00
> 99.9% <= 1.00
> Key of biggest row: \x00
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
