[
https://issues.apache.org/jira/browse/HBASE-16594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15476741#comment-15476741
]
binlijin edited comment on HBASE-16594 at 9/12/16 2:15 AM:
-----------------------------------------------------------
I do test with one of our very important table and the most requests on this
table is random get. This table have 5 column family(why have so many families,
this is for history reason.)
I get one region's data and do the random get performance on a regionserver.
This region's detail information is:
{code}
number of row : 3463153
5 family: a,b,c,d,f
family a : avgKeyLen=54,avgValueLen=12 entries=234100060
length=4369389736(4.07GB)
family b : avgKeyLen=53,avgValueLen=10 entries=51913519
length=981625160(936MB)
family c : avgKeyLen=50,avgValueLen=6 entries=14864860
length=273820502(261MB)
family d : avgKeyLen=50,avgValueLen=6 entries=141422679
length=3216604161(3GB)
family f : avgKeyLen=38,avgValueLen=13 entries=73084074
length=1174375801(1.09GB)
avg cells per row
family a : 67.6
family b : 15
family c : 4.3
family d : 40.8
family f : 21.1
BlockSize＝8k COMPRESSION=LZO RegionSize=9.33GB DATA_BLOCK_ENCODING => 'NONE'
BlockSize＝16k COMPRESSION=LZO RegionSize=8.52GB DATA_BLOCK_ENCODING => 'NONE'
BlockSize＝32k COMPRESSION=LZO RegionSize=7.81GB DATA_BLOCK_ENCODING => 'NONE'
BlockSize＝64k COMPRESSION=LZO RegionSize=7.74GB DATA_BLOCK_ENCODING => 'NONE'
BlockSize＝32k COMPRESSION=LZO RegionSize=7.84GB DATA_BLOCK_ENCODING =>
'ROW_INDEX_V1'
BlockSize＝32k COMPRESSION=LZO RegionSize=6.24GB DATA_BLOCK_ENCODING =>
'ROW_INDEX_V2'
{code}
was (Author: aoxiang):
I do test with one of our very important table. This table have 5 column
family(why have so many families, this is for history reason.)
I get one region's data and do the random get performance on a regionserver.
This region's detail information is:
{code}
number of row : 3463153
5 family: a,b,c,d,f
family a : avgKeyLen=54,avgValueLen=12 entries=234100060
length=4369389736(4.07GB)
family b : avgKeyLen=53,avgValueLen=10 entries=51913519
length=981625160(936MB)
family c : avgKeyLen=50,avgValueLen=6 entries=14864860
length=273820502(261MB)
family d : avgKeyLen=50,avgValueLen=6 entries=141422679
length=3216604161(3GB)
family f : avgKeyLen=38,avgValueLen=13 entries=73084074
length=1174375801(1.09GB)
avg cells per row
family a : 67.6
family b : 15
family c : 4.3
family d : 40.8
family f : 21.1
BlockSize＝8k COMPRESSION=LZO RegionSize=9.33GB DATA_BLOCK_ENCODING => 'NONE'
BlockSize＝16k COMPRESSION=LZO RegionSize=8.52GB DATA_BLOCK_ENCODING => 'NONE'
BlockSize＝32k COMPRESSION=LZO RegionSize=7.81GB DATA_BLOCK_ENCODING => 'NONE'
BlockSize＝64k COMPRESSION=LZO RegionSize=7.74GB DATA_BLOCK_ENCODING => 'NONE'
BlockSize＝32k COMPRESSION=LZO RegionSize=7.84GB DATA_BLOCK_ENCODING =>
'ROW_INDEX_V1'
BlockSize＝32k COMPRESSION=LZO RegionSize=6.24GB DATA_BLOCK_ENCODING =>
'ROW_INDEX_V2'
{code}
> ROW_INDEX_V2 DBE
> ----------------
>
> Key: HBASE-16594
> URL: https://issues.apache.org/jira/browse/HBASE-16594
> Project: HBase
> Issue Type: Sub-task
> Components: Performance
> Reporter: binlijin
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-16594-master_v1.patch, HBASE-16594-master_v2.patch
>
>
> See HBASE-16213, ROW_INDEX_V1 DataBlockEncoding.
> ROW_INDEX_V1 is the first version which have no storage optimization,
> ROW_INDEX_V2 do storage optimization: store every row only once, store column
> family only once in a HFileBlock.
> ROW_INDEX_V1 is :
> /**
> * Store cells following every row's start offset, so we can binary search to
> a row's cells.
> *
> * Format:
> * flat cells
> * integer: number of rows
> * integer: row0's offset
> * integer: row1's offset
> * ....
> * integer: dataSize
> *
> */
> ROW_INDEX_V2 is :
> * row1 qualifier timestamp type value tag
> * qualifier timestamp type value tag
> * qualifier timestamp type value tag
> * row2 qualifier timestamp type value tag
> * row3 qualifier timestamp type value tag
> * qualifier timestamp type value tag
> * ....
> * integer: number of rows
> * integer: row0's offset
> * integer: row1's offset
> * ....
> * column family
> * integer: dataSize
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
