[
https://issues.apache.org/jira/browse/TRAFODION-2272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15565695#comment-15565695
]
ASF GitHub Bot commented on TRAFODION-2272:
-------------------------------------------
Github user selvaganesang commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/753#discussion_r82815496
--- Diff: core/sql/src/main/java/org/trafodion/sql/HBaseClient.java ---
@@ -159,6 +159,17 @@ public HBaseClient() {
static public Connection getConnection() throws IOException {
+ // On some distributions, the hbase.client.scanner.timeout.period
setting is
+ // too small, resulting in annoying SocketTimeoutExceptions during
operations
+ // such as UPDATE STATISTICS on very large tables. On CDH 5.4.5 in
particular
+ // we have seen this. Unfortunately Cloudera Manager does not
allow us to
+ // change this setting, and setting it manually in hbase-site.xml
doesn't work
+ // because a later Cloudera Manager deploy would just overwrite
it. So, we
+ // programmatically check the setting here and insure it is at
least 1 hour.
+ long configuredTimeout =
config.getLong("hbase.client.scanner.timeout.period",0);
+ if (configuredTimeout < 3600000 /* 1 hour */)
+ config.setLong("hbase.client.scanner.timeout.period",3600000);
--- End diff --
Connection is also created at line 184 also. Can you please change it to
call this method so that all connections have this property configured.
> UPDATE STATS on 5.5 billion row UNALIGNED table fails with HBase error 706,
> Retries Exhausted
> ---------------------------------------------------------------------------------------------
>
> Key: TRAFODION-2272
> URL: https://issues.apache.org/jira/browse/TRAFODION-2272
> Project: Apache Trafodion
> Issue Type: Bug
> Components: sql-cmp
> Affects Versions: 2.1-incubating
> Environment: Cluster, CDH 5.4 distribution
> Reporter: David Wayne Birdsall
> Assignee: David Wayne Birdsall
>
> Symptom:
> update statistics for table trafodion.javabench.oe_orderline_18432 on every
> column, (OL_W_ID, OL_I_ID), (OL_D_ID, OL_W_ID), (OL_D_ID, OL_I_ID) sample;
> command fails with the following:
> ** ERROR[9200] UPDATE STATISTICS for table
> TRAFODION.JAVABENCH.OE_ORDERLINE_18432 encountered an error (8448) from
> statement HSCursor::fetchRowset(). [2016-09-20 01:15:13]
> *** ERROR[8448] Unable to access Hbase interface. Call to
> ExpHbaseInterface::nextRow returned error HBASE_ACCESS_ERROR(-706). Cause:
> java.util.concurrent.ExecutionException:
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> attempts=4, exceptions:
> Tue Sep 20 01:13:06 UTC 2016, null, java.net.SocketTimeoutException:
> callTimeout=60000, callDuration=60305: row '?. [2016-09-20 01:15:13]
> *** ERROR[9200] UPDATE STATISTICS for table
> TRAFODION.JAVABENCH.OE_ORDERLINE_18432 encountered an error (8448) from
> statement READ_COLS_INTO_MEM. [2016-09-20 01:15:13]
> *** ERROR[8448] Unable to access Hbase interface. Call to
> ExpHbaseInterface::nextRow returned error HBASE_ACCESS_ERROR(-706). Cause:
> java.util.concurrent.ExecutionException:
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> attempts=4, exceptions:
> Tue Sep 20 01:13:06 UTC 2016, null, java.net.SocketTimeoutException:
> callTimeout=60000, callDuration=60305: row '?. [2016-09-20 01:15:13]
> The table in question was unaligned format, having 5.5 billion rows. The DDL
> for the table is below:
> CREATE TABLE TRAFODION.JAVABENCH.OE_ORDERLINE_18432
> (
> OL_O_ID INT NO DEFAULT NOT NULL NOT DROPPABLE
> SERIALIZED
> , OL_D_ID INT NO DEFAULT NOT NULL NOT DROPPABLE
> SERIALIZED
> , OL_W_ID INT NO DEFAULT NOT NULL NOT DROPPABLE
> SERIALIZED
> , OL_NUMBER INT NO DEFAULT NOT NULL NOT DROPPABLE
> SERIALIZED
> , OL_I_ID INT NO DEFAULT NOT NULL NOT DROPPABLE
> SERIALIZED
> , OL_SUPPLY_W_ID INT NO DEFAULT NOT NULL NOT DROPPABLE
> SERIALIZED
> , OL_DELIVERY_D TIMESTAMP(6) DEFAULT NULL NOT SERIALIZED
> , OL_QUANTITY NUMERIC(2, 0) NO DEFAULT NOT NULL NOT
> DROPPABLE SERIALIZED
> , OL_AMOUNT NUMERIC(6, 2) NO DEFAULT NOT NULL NOT
> DROPPABLE SERIALIZED
> , OL_DIST_INFO CHAR(24) CHARACTER SET ISO88591 COLLATE
> DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED
> , PRIMARY KEY (OL_W_ID ASC, OL_D_ID ASC, OL_O_ID ASC, OL_NUMBER ASC)
> )
> SALT USING 64 PARTITIONS
> ON (OL_W_ID)
> ;
> The cluster in question had 8 nodes. Memory was not an issue during the test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
