[
https://issues.apache.org/jira/browse/HBASE-11331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14109817#comment-14109817
]
Nick Dimiduk commented on HBASE-11331:
--------------------------------------
bq. Would be interesting to see how it does in a long running test?
Today's results were from 90m warmup + 40m test. The 16h run I posted on
Thursday [0] seemed stable. No slope in those graphs. This is just PE mind you,
no concurrent writes.
bq. On the blockCacheCount, this is indication of our caching more blocks? 3x?
Yes, exactly. It's purely compression ratio manifest in memory. All this is
using SNAPPY and the reported compression ratio is 0.2473, thus the ~3x
increase. Probably using GZ would have better BC utilization buy higher CPU
load (should translate to a standard compression benchmark).
[0]:
https://issues.apache.org/jira/secure/attachment/12663479/v03-20g-045g-true-16h.pdf
> [blockcache] lazy block decompression
> -------------------------------------
>
> Key: HBASE-11331
> URL: https://issues.apache.org/jira/browse/HBASE-11331
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: Nick Dimiduk
> Assignee: Nick Dimiduk
> Attachments: HBASE-11331.00.patch, HBASE-11331.01.patch,
> HBASE-11331.02.patch, HBASE-11331.03.patch, HBASE-11331.04.patch,
> HBASE-11331.05.patch, HBASE-11331LazyBlockDecompressperfcompare.pdf,
> hbase-hbase-master-hor17n36.gq1.ygridcore.net.log, lazy-decompress.02.0.pdf,
> lazy-decompress.02.1.json, lazy-decompress.02.1.pdf, v03-20g-045g-false.pdf,
> v03-20g-045g-true-16h.pdf, v03-20g-045g-true.pdf
>
>
> Maintaining data in its compressed form in the block cache will greatly
> increase our effective blockcache size and should show a meaning improvement
> in cache hit rates in well designed applications. The idea here is to lazily
> decompress/decrypt blocks when they're consumed, rather than as soon as
> they're pulled off of disk.
> This is related to but less invasive than HBASE-8894.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
