[
https://issues.apache.org/jira/browse/HBASE-17819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398136#comment-16398136
]
Anoop Sam John commented on HBASE-17819:
----------------------------------------
Actually this patch included 2 parts.
1. For file backed BC, we might not need some of the newly added states in
Bucket entry. The refCount stuff. We can easily fix this.
2. We have an extra NavigableSet at BucketCache layer to track the blocks per
HFiles. This is used to evict those blocks when this file is invalidate
(because of compaction). The patch was trying to even avoid this also. In that
path, we changed this compacted away block eviction to an asyn model. A extra
dedicated thread will be handling this job. This will need more discussion
So what I plan now is split this patch into 2. Get the 1st one reviewed and
committed soon. That is easy thing. Then will put up patch for the #2 part
alone and we can discuss specifically that. Sounds good?
> Reduce the heap overhead for BucketCache
> ----------------------------------------
>
> Key: HBASE-17819
> URL: https://issues.apache.org/jira/browse/HBASE-17819
> Project: HBase
> Issue Type: Sub-task
> Components: BucketCache
> Reporter: Anoop Sam John
> Assignee: Anoop Sam John
> Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-17819_V1.patch, HBASE-17819_V2.patch,
> HBASE-17819_V3.patch
>
>
> We keep Bucket entry map in BucketCache. Below is the math for heapSize for
> the key , value into this map.
> BlockCacheKey
> ---------------
> String hfileName - Ref - 4
> long offset - 8
> BlockType blockType - Ref - 4
> boolean isPrimaryReplicaBlock - 1
> Total = 12 (Object) + 17 = 29
> BucketEntry
> ------------
> int offsetBase - 4
> int length - 4
> byte offset1 - 1
> byte deserialiserIndex - 1
> long accessCounter - 8
> BlockPriority priority - Ref - 4
> volatile boolean markedForEvict - 1
> AtomicInteger refCount - 16 + 4
> long cachedTime - 8
> Total = 12 (Object) + 51 = 63
> ConcurrentHashMap Map.Entry - 40
> blocksByHFile ConcurrentSkipListSet Entry - 40
> Total = 29 + 63 + 80 = 172
> For 10 million blocks we will end up having 1.6GB of heap size.
> This jira aims to reduce this as much as possible
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
