[
https://issues.apache.org/jira/browse/HBASE-23277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16973592#comment-16973592
]
Nick Dimiduk commented on HBASE-23277:
--------------------------------------
Fancy script :)
Proactive eviction of blocks is probably not necessary -- the normal expiration
should eject them in due time. Maybe [~larsh] or [~apurtell] have some memory
as to why we do this proactively. If eviction is necessary, maybe we need to
add an in-memory index of hfilename to list of blocks (or use one that exists
already, if it exists already).
> CompactedHFilesDischarger may lead to high cpu usage
> -----------------------------------------------------
>
> Key: HBASE-23277
> URL: https://issues.apache.org/jira/browse/HBASE-23277
> Project: HBase
> Issue Type: Brainstorming
> Reporter: junfei liang
> Priority: Minor
> Attachments: compated_discharger.png
>
>
> in one of online cluster(RS:16C72G, hbase version is 2.x) , cpu average
> usage is 30%, but cpu usage may above 80% occasionally. by a enchanced
> 'top -H' script, i find 'RS_COMPACTED_FILES_DISCHARGER' threads can use
> 50%+ cpu ( 10 * 80% ).
>
> !compated_discharger.png|width=1168,height=454!
>
> by jstack , we found most of stacks is at.
> "org.apache.hadoop.hbase.io.hfile.LruBlockCache#evictBlocksByHfileName"
> public int evictBlocksByHfileName(String hfileName) {
> int numEvicted = 0;
> for (BlockCacheKey key : map.keySet()) {
> if (key.getHfileName().equals(hfileName)) {
> if (evictBlock(key))
> ++numEvicted;
> }
> }
> if (victimHandler != null) {
> numEvicted += victimHandler.evictBlocksByHfileName(hfileName);
> }
> return numEvicted;
> }
>
> when close storefile, now we evict it's blocks from cache, but we have to
> traverse the whole blockcache to locate the block cached, and can lead to
> high cpu usage.
> do we need evict blockcache when close file, even for a small file ( size
> less than 10MB)? it's a lru block cache, the unused block will be
> automatically removed from cache when blockcache usage is high?
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
