[
https://issues.apache.org/jira/browse/HBASE-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12861948#action_12861948
]
Jonathan Gray commented on HBASE-2500:
--------------------------------------
My preference would be to add all of these to the existing block cache LRU.
The open question then would be whether they are treated like other normal data
blocks, or whether we add additional block types. The design of the LRU
actually makes it quite easy to add additional types (currently there is single
access block, multiple access block, and in-memory block), and there is not
much overhead involved. The additional overhead is done in the background
eviction thread so doesn't add anything to the put/get methods.
If there was a new block type or two, like "index" and/or "meta", then these
would be able to have their own slice of the total block cache pie. The LRU
gives each of the different block types their own percentage of the total LRU
size. Each is able to grow above it's slice if others are under-utilized, the
LRU just guarantees that during an eviction that block type will be given at
least the specified slice size.
I think we'll also need to add code to re-fetch block indexes / meta blocks if
they are not in the block cache.
> Block indexes and meta blocks should be put into the LRU
> --------------------------------------------------------
>
> Key: HBASE-2500
> URL: https://issues.apache.org/jira/browse/HBASE-2500
> Project: Hadoop HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: Jonathan Gray
> Fix For: 0.20.5, 0.21.0
>
>
> Currently we are not tracking the heap usage of block indexes. We're just
> relying on the fact that we leave a large percentage of total heap as part of
> "everything else" not in memstores or the block cache. The "everything else"
> portion should be left for ephemeral/transient memory usage, not long-lived
> objects like block indexes. Introduction of blooms will extend this issue to
> meta blocks in general.
> All of this should be moved into an LRU. Either the existing block cache LRU
> or a new LRU.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
