[ https://issues.apache.org/jira/browse/HBASE-10501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13904390#comment-13904390 ]
Lars Hofhansl commented on HBASE-10501: --------------------------------------- Currently IncreasingToUpperBoundRegionSplitPolicy is not configurable. If using another RegionSplitPolicy it's fine. You sound a bit concerned. You'd rather not change it? Or change is too aggressive? I think the key is that there is no scenario now that would split earlier than before (because that could lead to a split storm). As along as we have a table sparsely distributed (i.e. 0 or 1 regions on a given region server) the behavior would be very similar, we'd split at 256m instead of 128m. Just that as soon as we see more regions on a regionserer we should start to assume that the table is already somewhat distributed across the cluster. > Make IncreasingToUpperBoundRegionSplitPolicy configurable > --------------------------------------------------------- > > Key: HBASE-10501 > URL: https://issues.apache.org/jira/browse/HBASE-10501 > Project: HBase > Issue Type: Bug > Reporter: Lars Hofhansl > Assignee: Lars Hofhansl > Priority: Critical > Fix For: 0.96.2, 0.98.1, 0.99.0, 0.94.17 > > Attachments: 10501-0.94-v2.txt, 10501-0.94-v3.txt, 10501-0.94-v4.txt, > 10501-0.94.txt, 10501-trunk-v2.txt, 10501-trunk.txt > > > During some (admittedly artificial) load testing we found a large amount > split activity, which we tracked down the > IncreasingToUpperBoundRegionSplitPolicy. > The current logic is this (from the comments): > "regions that are on this server that all are of the same table, squared, > times the region flush size OR the maximum region split size, whichever is > smaller" > So with a flush size of 128mb and max file size of 20gb, we'd need 13 region > of the same table on an RS to reach the max size. > With 10gb file sized it is still 9 regions of the same table. > Considering that the number of regions that an RS can carry is limited and > there might be multiple tables, this should be more configurable. > I think the squaring is smart and we do not need to change it. > We could > * Make the start size configurable and default it to the flush size > * Add multiplier for the initial size, i.e. start with n * flushSize > * Also change the default to start with 2*flush size > Of course one can override the default split policy, but these seem like > simple tweaks. > Or we could instead set the goal of how many regions of the same table would > need to be present in order to reach the max size. In that case we'd start > with maxSize/goal^2. So if max size is 20gb and the goal is three we'd start > with 20g/9 = 2.2g for the initial region size. > [~stack], I'm especially interested in your opinion. -- This message was sent by Atlassian JIRA (v6.1.5#6160)