[jira] [Commented] (GORA-506) Investigate timing of HBase tests

[Kiyonari Harigae (JIRA)]

    [ 
https://issues.apache.org/jira/browse/GORA-506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15897023#comment-15897023
 ] 

Kiyonari Harigae commented on GORA-506:
---------------------------------------

I tried to test locally, result as follows

Total: 634 sec
HBaseClusterSingleton#truncateAllTables: 472 sec

The most occupation was HBaseClusterSingleton#truncateAllTables,
whose percentage was about 74%.

I think that because of the implementation of HBaseTestingUtility#truncateTable 
differs between 0.98.8 and 1.2.3.
At 0.98.8 the implementation of this method was delete each rows, whereas 1.2.3 
implemented as disableTable -> Truncate.
Instead, at 1.2.3 can be satisfy the requirement by using deleteTableData.
Improvement to 2min40sec by using deleteTableData.
It is close to at gora-0.6.1.

HBaseTestingUtility#truncateTable
(hbase-server-0.98.8-hadoop2-tests)

  public HTable truncateTable(byte[] tableName)
    throws IOException
  {
    return truncateTable(TableName.valueOf(tableName));
  }
  
  public HTable truncateTable(TableName tableName)
    throws IOException
  {
    HTable table = new HTable(getConfiguration(), tableName);
    Scan scan = new Scan();
    ResultScanner resScan = table.getScanner(scan);
    for (Result res : resScan)
    {
      Delete del = new Delete(res.getRow());
      table.delete(del);
    }
    resScan = table.getScanner(scan);
    resScan.close();
    return table;
  }

HBaseTestingUtility#truncateTable
(hbase-server-1.2.3-tests)

  public HTable truncateTable(TableName tableName, boolean preserveRegions)
    throws IOException
  {
    Admin admin = getHBaseAdmin();
    if (!admin.isTableDisabled(tableName)) {
      admin.disableTable(tableName);
    }
    admin.truncateTable(tableName, preserveRegions);
    return new HTable(getConfiguration(), tableName);
  }
  
  public HTable truncateTable(TableName tableName)
    throws IOException
  {
    return truncateTable(tableName, false);
  }
  
  public HTable truncateTable(byte[] tableName, boolean preserveRegions)
    throws IOException
  {
    return truncateTable(TableName.valueOf(tableName), preserveRegions);
  }
  
  public HTable truncateTable(byte[] tableName)
    throws IOException
  {
    return truncateTable(tableName, false);
  }

Instead use follow:

  public HTable deleteTableData(byte[] tableName)
    throws IOException
  {
    return deleteTableData(TableName.valueOf(tableName));
  }
  
  public HTable deleteTableData(TableName tableName)
    throws IOException
  {
    HTable table = new HTable(getConfiguration(), tableName);
    Scan scan = new Scan();
    ResultScanner resScan = table.getScanner(scan);
    for (Result res : resScan)
    {
      Delete del = new Delete(res.getRow());
      table.delete(del);
    }
    resScan = table.getScanner(scan);
    resScan.close();
    return table;
  }


> Investigate timing of HBase tests
> ---------------------------------
>
>                 Key: GORA-506
>                 URL: https://issues.apache.org/jira/browse/GORA-506
>             Project: Apache Gora
>          Issue Type: Test
>          Components: gora-hbase
>    Affects Versions: 0.7
>            Reporter: Lewis John McGibbney
>            Priority: Critical
>
> GORA-443 * fixed * an upgrade to HBase 1.2.3, there is a significant amount 
> of time however spent on the tests.
> We need to investigate why and fix.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)