[
https://issues.apache.org/jira/browse/HBASE-23277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16973901#comment-16973901
]
Anoop Sam John commented on HBASE-23277:
----------------------------------------
In case of Bucket Cache we maintain another Map which maps Hfile name vs blocks
so as to avoid the entire original cache map iteration for this eviction. Ya
with a cost of more heap memory need. I believe I had a jira to discuss even
for Bucket Cache we can avoid this. The extra heap space req matters a lot
when we have huge Bucket Cache size usage.
Agree to the points by Andy. That might be the reason.
One thing we can do here is , when we read HFile blocks for the compaction (by
compaction thread), we can avoid treating it as a hit to make it recently used.
So we are doing all probable things to downgrade these blocks and possibly get
evicted when next eviction op happens. Also as in the desc says may be for
much bigger sized files (only), we can do this proactive eviction after
compaction. Even the current status wrt free space available for the cache can
be another decision point in deciding whether to do a proactive evict or not.
This issue is wrt On heap LRUCache only I believe. Lets do the Bucket Cache
issue in other old jira.
> CompactedHFilesDischarger may lead to high cpu usage
> -----------------------------------------------------
>
> Key: HBASE-23277
> URL: https://issues.apache.org/jira/browse/HBASE-23277
> Project: HBase
> Issue Type: Brainstorming
> Reporter: junfei liang
> Priority: Minor
> Attachments: compated_discharger.png
>
>
> in one of online cluster(RS:16C72G, hbase version is 2.x) , cpu average
> usage is 30%, but cpu usage may above 80% occasionally. by a enchanced
> 'top -H' script, i find 'RS_COMPACTED_FILES_DISCHARGER' threads can use
> 50%+ cpu ( 10 * 80% ).
>
> !compated_discharger.png|width=1168,height=454!
>
> by jstack , we found most of stacks is at.
> "org.apache.hadoop.hbase.io.hfile.LruBlockCache#evictBlocksByHfileName"
> public int evictBlocksByHfileName(String hfileName) {
> int numEvicted = 0;
> for (BlockCacheKey key : map.keySet()) {
> if (key.getHfileName().equals(hfileName)) {
> if (evictBlock(key))
> ++numEvicted;
> }
> }
> if (victimHandler != null) {
> numEvicted += victimHandler.evictBlocksByHfileName(hfileName);
> }
> return numEvicted;
> }
>
> when close storefile, now we evict it's blocks from cache, but we have to
> traverse the whole blockcache to locate the block cached, and can lead to
> high cpu usage.
> do we need evict blockcache when close file, even for a small file ( size
> less than 10MB)? it's a lru block cache, the unused block will be
> automatically removed from cache when blockcache usage is high?
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
