Re: Cut Size

[ShaoFeng Shi]

Ok... let me explain; The default cut is 5GB per region; If the cube's
estimated size is 3000GB, then there should be 600 regions; While, since
"max-region-count" is set to 500, so the region number in one HTable will
be controlled at <= 500; so finally it will split the regions by 3000/500 =
6GB. This is the cap; it controll the max region count in one table.

For "kylin.storage.hbase.hfile-size-gb", see
https://issues.apache.org/jira/browse/KYLIN-1323

2016-12-12 20:22 GMT+08:00 Alberto Ramón :

> "it will do a cap" I dont't know what cap. this means  :)
>
> Then what is the function of "kylin.storage.hbase.hfile-size-gb=2"
>
> 2016-12-12 2:58 GMT+01:00 ShaoFeng Shi :
>
>> when you have hfile-size-gb, you re-split HFile using max-region-count
>> and region-cut-gb ?
>>
>> --> Yes; Kylin will estimate the total size, then divide by
>> "regino-cut-gb" to get the region number; If the region number exceeds
>> "max-region-count", it will do a cap.
>>
>> Medium , small, . ..  is deprecated (KYLIN-1669
>> )?
>> --> Yes, that marker has been removed; Will use same split configuration
>> for all cubes; If user want to customize, he can overwrite the config
>> values at cube level.
>>
>> 2016-12-08 21:27 GMT+08:00 Alberto Ramón :
>>
>>> I'm reading this MailList
>>> 
>>> and have some doubts (Example
>>> 
>>> ):
>>>
>>> region-cut-gb
>>> max-region-count
>>> hfile-size-gb
>>>
>>> when you have hfile-size-gb, you re-split HFile using max-region-count
>>> and region-cut-gb ?? or is for normal ingest, Kylin 1323?
>>>
>>> Medium , small, . ..  is deprecated (KYLIN-1669
>>> )? "# E.g, for cube
>>> whose capacity be marked as "SMALL", split region per 10GB by default"
>>> (From Example)
>>>
>>
>>
>>
>> --
>> Best regards,
>>
>> Shaofeng Shi 史少锋
>>
>>
>


-- 
Best regards,

Shaofeng Shi 史少锋

Re: Cut Size

[Alberto Ramón]

"it will do a cap" I dont't know what cap. this means  :)

Then what is the function of "kylin.storage.hbase.hfile-size-gb=2"

2016-12-12 2:58 GMT+01:00 ShaoFeng Shi :

> when you have hfile-size-gb, you re-split HFile using max-region-count and
> region-cut-gb ?
>
> --> Yes; Kylin will estimate the total size, then divide by
> "regino-cut-gb" to get the region number; If the region number exceeds
> "max-region-count", it will do a cap.
>
> Medium , small, . ..  is deprecated (KYLIN-1669
> )?
> --> Yes, that marker has been removed; Will use same split configuration
> for all cubes; If user want to customize, he can overwrite the config
> values at cube level.
>
> 2016-12-08 21:27 GMT+08:00 Alberto Ramón :
>
>> I'm reading this MailList
>> 
>> and have some doubts (Example
>> 
>> ):
>>
>> region-cut-gb
>> max-region-count
>> hfile-size-gb
>>
>> when you have hfile-size-gb, you re-split HFile using max-region-count
>> and region-cut-gb ?? or is for normal ingest, Kylin 1323?
>>
>> Medium , small, . ..  is deprecated (KYLIN-1669
>> )? "# E.g, for cube
>> whose capacity be marked as "SMALL", split region per 10GB by default"
>> (From Example)
>>
>
>
>
> --
> Best regards,
>
> Shaofeng Shi 史少锋
>
>