[
https://issues.apache.org/jira/browse/HBASE-11331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-11331:
---------------------------------
Attachment: lazy-decompress.02.0.pdf
Attaching some metrics after running v02 with the new flag enabled/disabled.
The test cluster is a single machine, configured with 12g heap and 0.6 for
blockcache, so slighly over 7g mem available for cache. The TestTable is snappy
compressed, written with PE --size=100. Shell {{status 'simple'}} reports
{{compressionRatio=0.2437}}. RandomRead test is run with --size=25, so with
feature enabled, we should be at ~6g of compressed blocks.
h3. client perceived perf
With flag=false, I see slightly higher throughput (~7.5k vs ~8k) and slightly
less gc (360ms vs 325ms).
h3. system load
CPU load looks slightly higher with flag=true (user time diff is negligible,
system time more noticeable, no difference in iowait).
h3. blockcache utilization
There are definitely cache evictions happening with flag=false that do not
happen with flag=true. The number of blocks cached is definitely higher with
flag=true (~17.5k vs ~21.5k).
h3. IO
No difference in reported IO requests. Probably it's the OS buffer cache to the
rescue here.
> [blockcache] lazy block decompression
> -------------------------------------
>
> Key: HBASE-11331
> URL: https://issues.apache.org/jira/browse/HBASE-11331
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: Nick Dimiduk
> Assignee: Nick Dimiduk
> Attachments: HBASE-11331.00.patch, HBASE-11331.01.patch,
> HBASE-11331.02.patch, HBASE-11331LazyBlockDecompressperfcompare.pdf,
> lazy-decompress.02.0.pdf
>
>
> Maintaining data in its compressed form in the block cache will greatly
> increase our effective blockcache size and should show a meaning improvement
> in cache hit rates in well designed applications. The idea here is to lazily
> decompress/decrypt blocks when they're consumed, rather than as soon as
> they're pulled off of disk.
> This is related to but less invasive than HBASE-8894.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
