[
https://issues.apache.org/jira/browse/HBASE-23279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16977777#comment-16977777
]
Viraj Jasani edited comment on HBASE-23279 at 11/19/19 7:33 PM:
----------------------------------------------------------------
ROW_INDEX_V1 indeed seems to be taking more space for BucketCache.
Tried running these commands separately on fresh local cluster:
{code:java}
bin/hbase ltt -init_only -data_block_encoding ROW_INDEX_V1
bin/hbase ltt -skip_init -write 1:10:300 -read 100:200 -num_keys 100000
-multiput -multiget_batchsize 20
echo "flush 'cluster_test'" | bin/hbase shell
bin/hbase ltt -skip_init -read 100:200 -num_keys 100000 -multiget_batchsize
20{code}
{code:java}
bin/hbase ltt -init_only -data_block_encoding NONE
bin/hbase ltt -skip_init -write 1:10:300 -read 100:200 -num_keys 100000
-multiput -multiget_batchsize 20
echo "flush 'cluster_test'" | bin/hbase shell
bin/hbase ltt -skip_init -read 100:200 -num_keys 100000 -multiget_batchsize 20
{code}
L1 Cache Size Limit: 805.9 MB
L2 Cache Size Limit: 4.1 GB
Stats for 1st case(ROW_INDEX_V1):
{code:java}
L2 Block Count: 432
Size of Blocks: 38.1 MB (DataBlocks Size: 28.58 MB)
L1+L2 Combined Size: 38.8 MB{code}
Stats for 2nd case(NONE):
{code:java}
L2 Block Count: 432
Size of Blocks: 27.3 MB (DataBlocks Size: 26.86 MB)
L1+L2 Combined Size: 28 MB
{code}
was (Author: vjasani):
ROW_INDEX_V1 indeed seems to be taking more space for BucketCache.
Tried running these commands separately on fresh local cluster:
{code:java}
bin/hbase ltt -init_only -data_block_encoding ROW_INDEX_V1
bin/hbase ltt -skip_init -write 1:10:300 -read 100:200 -num_keys 100000
-multiput -multiget_batchsize 20
echo "flush 'cluster_test'" | bin/hbase shell
bin/hbase ltt -skip_init -read 100:200 -num_keys 100000 -multiget_batchsize
20{code}
{code:java}
bin/hbase ltt -init_only -data_block_encoding NONE
bin/hbase ltt -skip_init -write 1:10:300 -read 100:200 -num_keys 100000
-multiput -multiget_batchsize 20
echo "flush 'cluster_test'" | bin/hbase shell
bin/hbase ltt -skip_init -read 100:200 -num_keys 100000 -multiget_batchsize 20
{code}
L1 Cache Size Limit: 805.9 MB
L2 Cache Size Limit: 4.1 GB
Stats for 1st case(ROW_INDEX_V1):
{code:java}
L2 Block Count: 432
Size of Blocks: 38.1 MB (DataBlocks Size: 28.58 MB)
L1+L2 Combined Size: 38.8 MB{code}
Stats for 2nd case(NONE):
{code:java}
L2 Block Count: 432
Size of Blocks: 27.3 MB (DataBlocks Size: 26.86 MB)
L1+L2 Combined Size: 28 MB
{code}
> Switch default block encoding to ROW_INDEX_V1
> ---------------------------------------------
>
> Key: HBASE-23279
> URL: https://issues.apache.org/jira/browse/HBASE-23279
> Project: HBase
> Issue Type: Wish
> Affects Versions: 3.0.0, 2.3.0
> Reporter: Lars Hofhansl
> Assignee: Viraj Jasani
> Priority: Minor
> Fix For: 3.0.0, 2.3.0
>
> Attachments: HBASE-23279.master.000.patch
>
>
> Currently we set both block encoding and compression to NONE.
> ROW_INDEX_V1 has many advantages and (almost) no disadvantages (the hfiles
> are slightly larger about 3% or so). I think that would a better default than
> NONE.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
