[
https://issues.apache.org/jira/browse/HBASE-11331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14129242#comment-14129242
]
Nick Dimiduk commented on HBASE-11331:
--------------------------------------
Thanks [~enis], [~stack] for having a look.
bq. Add to the simplification of block cache config a note that we need to
unify the configuration args?
Alright, I'll open a ticket.
bq. Needs fat release note and on commit, add something to the refguide in the
block cache section else I'm afraid folks won't come across this feature.
Release note I'd planned, just what's in the commit message; you want more than
this?
{noformat}
When hbase.block.data.cachecompressed=true, DATA (and ENCODED_DATA) blocks are
cached in the BlockCache in their on-disk format. This is different from the
default behavior, which decompresses and decrypts a block before caching.
{noformat}
I'll open a docs ticket for updating the book.
bq. Is this right? Assuming HeapByteBuffer here.
Alas, yes. There's a number of assumptions baked into HFileBlock about
HeapByteArrays. The whole read-ahead of the next block's header feature is
based on this supposition (look for reads beyond limit without checking
capacity; this is how I ran into the serialization bug in the first place).
> [blockcache] lazy block decompression
> -------------------------------------
>
> Key: HBASE-11331
> URL: https://issues.apache.org/jira/browse/HBASE-11331
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: Nick Dimiduk
> Assignee: Nick Dimiduk
> Fix For: 2.0.0, 0.98.7, 0.99.1
>
> Attachments: HBASE-11331.00.patch, HBASE-11331.01.patch,
> HBASE-11331.02.patch, HBASE-11331.03.patch, HBASE-11331.04.patch,
> HBASE-11331.05-0.98.patch, HBASE-11331.05.patch, HBASE-11331.06-0.98.patch,
> HBASE-11331.06.patch, HBASE-11331.07-0.98.patch, HBASE-11331.07.patch,
> HBASE-11331LazyBlockDecompressperfcompare.pdf,
> hbase-hbase-master-hor17n36.gq1.ygridcore.net.log, lazy-decompress.02.0.pdf,
> lazy-decompress.02.1.json, lazy-decompress.02.1.pdf, v03-20g-045g-false.pdf,
> v03-20g-045g-true-16h.pdf, v03-20g-045g-true.pdf
>
>
> Maintaining data in its compressed form in the block cache will greatly
> increase our effective blockcache size and should show a meaning improvement
> in cache hit rates in well designed applications. The idea here is to lazily
> decompress/decrypt blocks when they're consumed, rather than as soon as
> they're pulled off of disk.
> This is related to but less invasive than HBASE-8894.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
