[
https://issues.apache.org/jira/browse/TRAFODION-2272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15563712#comment-15563712
]
ASF GitHub Bot commented on TRAFODION-2272:
-------------------------------------------
GitHub user DaveBirdsall opened a pull request:
https://github.com/apache/incubator-trafodion/pull/753
[TRAFODION-2272] Insure adequate hbase.client.scanner.timeout.period
UPDATE STATISTICS on large tables fails on some cluster installations
because the hbase.client.scanner.timeout.period setting is too low. The symptom
is that a sample scan fails with a SocketTimeoutException. This change adds
code to insure the setting is at least 1 hour.
This is viewed as a short-term fix. The setting is hard-coded in the
Trafodion HBase client code. Very soon there will be another change that will
abstract this out into some sort of properties file mechanism that is easier to
manipulate.
Too, as soon as we migrate to HBase 1.1-based distributions, the HBase
heartbeat protocol should eliminate the need for this particular change.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/DaveBirdsall/incubator-trafodion Trafodion2272
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-trafodion/pull/753.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #753
----
commit 670d9edc49e5a1d8fbe1a94824ce5e334110ee0b
Author: Dave Birdsall <[email protected]>
Date: 2016-10-10T22:12:28Z
[TRAFODION-2272] Insure adequate hbase.client.scanner.timeout.period
----
> UPDATE STATS on 5.5 billion row UNALIGNED table fails with HBase error 706,
> Retries Exhausted
> ---------------------------------------------------------------------------------------------
>
> Key: TRAFODION-2272
> URL: https://issues.apache.org/jira/browse/TRAFODION-2272
> Project: Apache Trafodion
> Issue Type: Bug
> Components: sql-cmp
> Affects Versions: 2.1-incubating
> Environment: Cluster, CDH 5.4 distribution
> Reporter: David Wayne Birdsall
> Assignee: David Wayne Birdsall
>
> Symptom:
> update statistics for table trafodion.javabench.oe_orderline_18432 on every
> column, (OL_W_ID, OL_I_ID), (OL_D_ID, OL_W_ID), (OL_D_ID, OL_I_ID) sample;
> command fails with the following:
> ** ERROR[9200] UPDATE STATISTICS for table
> TRAFODION.JAVABENCH.OE_ORDERLINE_18432 encountered an error (8448) from
> statement HSCursor::fetchRowset(). [2016-09-20 01:15:13]
> *** ERROR[8448] Unable to access Hbase interface. Call to
> ExpHbaseInterface::nextRow returned error HBASE_ACCESS_ERROR(-706). Cause:
> java.util.concurrent.ExecutionException:
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> attempts=4, exceptions:
> Tue Sep 20 01:13:06 UTC 2016, null, java.net.SocketTimeoutException:
> callTimeout=60000, callDuration=60305: row '?. [2016-09-20 01:15:13]
> *** ERROR[9200] UPDATE STATISTICS for table
> TRAFODION.JAVABENCH.OE_ORDERLINE_18432 encountered an error (8448) from
> statement READ_COLS_INTO_MEM. [2016-09-20 01:15:13]
> *** ERROR[8448] Unable to access Hbase interface. Call to
> ExpHbaseInterface::nextRow returned error HBASE_ACCESS_ERROR(-706). Cause:
> java.util.concurrent.ExecutionException:
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> attempts=4, exceptions:
> Tue Sep 20 01:13:06 UTC 2016, null, java.net.SocketTimeoutException:
> callTimeout=60000, callDuration=60305: row '?. [2016-09-20 01:15:13]
> The table in question was unaligned format, having 5.5 billion rows. The DDL
> for the table is below:
> CREATE TABLE TRAFODION.JAVABENCH.OE_ORDERLINE_18432
> (
> OL_O_ID INT NO DEFAULT NOT NULL NOT DROPPABLE
> SERIALIZED
> , OL_D_ID INT NO DEFAULT NOT NULL NOT DROPPABLE
> SERIALIZED
> , OL_W_ID INT NO DEFAULT NOT NULL NOT DROPPABLE
> SERIALIZED
> , OL_NUMBER INT NO DEFAULT NOT NULL NOT DROPPABLE
> SERIALIZED
> , OL_I_ID INT NO DEFAULT NOT NULL NOT DROPPABLE
> SERIALIZED
> , OL_SUPPLY_W_ID INT NO DEFAULT NOT NULL NOT DROPPABLE
> SERIALIZED
> , OL_DELIVERY_D TIMESTAMP(6) DEFAULT NULL NOT SERIALIZED
> , OL_QUANTITY NUMERIC(2, 0) NO DEFAULT NOT NULL NOT
> DROPPABLE SERIALIZED
> , OL_AMOUNT NUMERIC(6, 2) NO DEFAULT NOT NULL NOT
> DROPPABLE SERIALIZED
> , OL_DIST_INFO CHAR(24) CHARACTER SET ISO88591 COLLATE
> DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED
> , PRIMARY KEY (OL_W_ID ASC, OL_D_ID ASC, OL_O_ID ASC, OL_NUMBER ASC)
> )
> SALT USING 64 PARTITIONS
> ON (OL_W_ID)
> ;
> The cluster in question had 8 nodes. Memory was not an issue during the test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
