[jira] [Commented] (HBASE-15324) Jitter may cause desiredMaxFileSize overflow in ConstantSizeRegionSplitPolicy and trigger unexpected split

[huaxiang sun (JIRA)]

    [ 
https://issues.apache.org/jira/browse/HBASE-15324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15651817#comment-15651817
 ] 

huaxiang sun commented on HBASE-15324:
--------------------------------------

Hi [~liyu], have one question about the fix. Why the overflow is only checked 
when (this.jitterRate > EPSILON). I modified a bit of the test case, it can get 
a minus desiredMaxFileSize (overflow). Should the overflow be checked for all 
the cases? Thanks.
{code}
  @Test
  public void testConstantSizePolicyWithJitter() throws IOException {
    conf.set(HConstants.HBASE_REGION_SPLIT_POLICY_KEY,
      ConstantSizeRegionSplitPolicy.class.getName());
    htd.setMaxFileSize(Long.MAX_VALUE);
    boolean positiveJitter = false;
    ConstantSizeRegionSplitPolicy policy = null;
    long value = 0;
    while (value >= 0) {
      policy = (ConstantSizeRegionSplitPolicy) 
RegionSplitPolicy.create(mockRegion, conf);
      //positiveJitter = policy.positiveJitterRate();
      value = policy.getDesiredMaxFileSize();
      if (value < 0) {
        System.out.println(policy.getDesiredMaxFileSize());
        System.out.println(policy.getJitterRate());
      }
    }

    // add a store
    HStore mockStore = Mockito.mock(HStore.class);
    Mockito.doReturn(2000L).when(mockStore).getSize();
    Mockito.doReturn(true).when(mockStore).canSplit();
    stores.add(mockStore);
    // Jitter shouldn't cause overflow when HTableDescriptor.MAX_FILESIZE set 
to Long.MAX_VALUE
    assertFalse(policy.shouldSplit());
  }
{code}

For the failed case, values are:
-9223365302346055681 (getDesiredMaxFileSize)
7.301568984985352E-7 (jitter rate)

> Jitter may cause desiredMaxFileSize overflow in ConstantSizeRegionSplitPolicy 
> and trigger unexpected split
> ----------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-15324
>                 URL: https://issues.apache.org/jira/browse/HBASE-15324
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 2.0.0, 1.1.3
>            Reporter: Yu Li
>            Assignee: Yu Li
>             Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.5, 1.1.8
>
>         Attachments: HBASE-15324.patch, HBASE-15324_v2.patch, 
> HBASE-15324_v3.patch, HBASE-15324_v3.patch
>
>
> We introduce jitter for region split decision in HBASE-13412, but the 
> following line in {{ConstantSizeRegionSplitPolicy}} may cause long value 
> overflow if MAX_FILESIZE is specified to Long.MAX_VALUE:
> {code}
> this.desiredMaxFileSize += (long)(desiredMaxFileSize * (RANDOM.nextFloat() - 
> 0.5D) * jitter);
> {code}
> In our case we specify MAX_FILESIZE to Long.MAX_VALUE to prevent target 
> region to split.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)