[
https://issues.apache.org/jira/browse/HBASE-23887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043470#comment-17043470
]
Danil Lipovoy edited comment on HBASE-23887 at 2/24/20 3:16 PM:
----------------------------------------------------------------
Thank you for response!
We don't have BucketCache, because need a lot of memory for other applications.
Unfortunatly I don't understand what exactly mean the case 3.
Looks like you mean when "L1 and internal L2".
If it is correct then since 2.0.0 version BucketCache can be deployed only
offheap or file and it is not actual now.
But I confused, becase in the last line you wrote "case 3, L2's memory are not
managed by JVM", so it is looks like when L2 is offheap.
In this case you absolutely right and this will not effect for L2, but if the
idea ok, can we can go step by step? I mean now implemet impovment for L1,
later add the same logic for L2.
was (Author: pustota):
Thank you for reponce!
We don't have BucketCache, because need a lot of memory for other applications.
Unfortunatly I don't understand what exactly mean the case 3.
Looks like you mean when "L1 and internal L2".
If it is correct then since 2.0.0 version BucketCache can be deployed only
offheap or file and it is not actual now.
But I confused, becase in the last line you wrote "case 3, L2's memory are not
managed by JVM", so it is looks like when L2 is offheap.
In this case you absolutely right and this will not effect for L2, but if the
idea ok, can we can go step by step? I mean now implemet impovment for L1,
later add the same logic for L2.
> BlockCache performance improve
> ------------------------------
>
> Key: HBASE-23887
> URL: https://issues.apache.org/jira/browse/HBASE-23887
> Project: HBase
> Issue Type: New Feature
> Reporter: Danil Lipovoy
> Priority: Minor
> Attachments: cmp.png
>
>
> Hi!
> I first time here, correct me please if something wrong.
> I want propose how to improve performance when data in HFiles much more than
> BlockChache (usual story in BigData). The idea - caching only part of DATA
> blocks. It is good becouse LruBlockCache starts to work and save huge amount
> of GC. See the picture in attachment with test below. Requests per second is
> higher, GC is lower.
>
> The key point of the code:
> Added the parameter: *hbase.lru.cache.data.block.percent* which by default =
> 100
>
> But if we set it 0-99, then will work the next logic:
>
>
> {code:java}
> public void cacheBlock(BlockCacheKey cacheKey, Cacheable buf, boolean
> inMemory) {
> if (cacheDataBlockPercent != 100 && buf.getBlockType().isData())
> if (cacheKey.getOffset() % 100 >= cacheDataBlockPercent)
> return;
> ...
> // the same code as usual
> }
> {code}
>
>
> Descriptions of the test:
> 4 nodes E5-2698 v4 @ 2.20GHz, 700 Gb Mem.
> 4 RegionServers
> 4 tables by 64 regions by 1.88 Gb data in each = 600 Gb total (only FAST_DIFF)
> Total BlockCache Size = 48 Gb (8 % of data in HFiles)
> Random read in 20 threads
>
> I am going to make Pull Request, hope it is right way to make some
> contribution in this cool product.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
