[
https://issues.apache.org/jira/browse/HBASE-10501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898540#comment-13898540
]
Lars Hofhansl edited comment on HBASE-10501 at 2/12/14 12:21 AM:
-----------------------------------------------------------------
Starting with flush size means that every flush will cause a split until we see
multiple regions of a table on an RS.
Maybe just starting with 2*flushsize would avert that.
was (Author: lhofhansl):
Starting flush size means that every flush will cause a split until we see
multiple regions of a table on an RS.
Maybe just starting with 2*flushsize would avert that.
> Make IncreasingToUpperBoundRegionSplitPolicy configurable
> ---------------------------------------------------------
>
> Key: HBASE-10501
> URL: https://issues.apache.org/jira/browse/HBASE-10501
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Attachments: 10501-0.94.txt
>
>
> During some (admittedly artificial) load testing we found a large amount
> split activity, which we tracked down the
> IncreasingToUpperBoundRegionSplitPolicy.
> The current logic is this (from the comments):
> "regions that are on this server that all are of the same table, squared,
> times the region flush size OR the maximum region split size, whichever is
> smaller"
> So with a flush size of 128mb and max file size of 20gb, we'd need 13 region
> of the same table on an RS to reach the max size.
> With 10gb file sized it is still 9 regions of the same table.
> Considering that the number of regions that an RS can carry is limited and
> there might be multiple tables, this should be more configurable.
> I think the squaring is smart and we do not need to change it.
> We could
> * Make the start size configurable and default it to the flush size
> * Add multiplier for the initial size, i.e. start with n * flushSize
> * Also change the default to start with 2*flush size
> Of course one can override the default split policy, but these seem like
> simple tweaks.
> Or we could instead set the goal of how many regions of the same table would
> need to be present in order to reach the max size. In that case we'd start
> with maxSize/goal^2. So if max size is 20gb and the goal is three we'd start
> with 20g/9 = 2.2g for the initial region size.
> [~stack], I'm especially interested in your opinion.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
