[
https://issues.apache.org/jira/browse/IMPALA-13256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17870119#comment-17870119
]
ASF subversion and git services commented on IMPALA-13256:
----------------------------------------------------------
Commit 806f4157db3d986badd962b194d9862cfebc2433 in impala's branch
refs/heads/master from wzhou-code
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=806f4157d ]
IMPALA-13256: Support more than 2G rows for COUNT(*) on jdbc table
Function DatabaseAccessor.getTotalNumberOfRecords() is defined with
return data type as int. This caused Impala to throw an exception when
executing COUNT(*) query for jdbc tables with more than 2G rows.
This patch fixed the issue by changing the function return type as
long. It also made number of rows in each TRowBatch fetched from jdbc
data source not to exceed 2G.
Testing:
- Passed core-test and tpcds test for jdbc tables.
- Manually created a jdbc table jdbc_table with more than 2G rows,
verified that query 'select count(*) from jdbc_table' returned
correct number of rows. Detailed steps were added in the comments
of IMPALA-13256.
Change-Id: I47db58300cbe3270bab07da02c3fcde6d7072334
Reviewed-on: http://gerrit.cloudera.org:8080/21617
Reviewed-by: Michael Smith <[email protected]>
Reviewed-by: Abhishek Rawat <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> SQLDataException in external JDBC table query request
> -----------------------------------------------------
>
> Key: IMPALA-13256
> URL: https://issues.apache.org/jira/browse/IMPALA-13256
> Project: IMPALA
> Issue Type: Sub-task
> Reporter: YifanZhang
> Assignee: Pranav Yogi Lodha
> Priority: Major
> Fix For: Impala 4.5.0
>
>
> Logs from impala coordinator node:
> {code}
> I0726 21:15:51.142773 3202116 GenericJdbcDatabaseAccessor.java:86]
> b34bb1ec996a2df9:a00f489200000000] Query to execute is [SELECT COUNT(*) FROM
> (SELECT * FROM test_table) tmptable]
> E0726 21:15:51.287364 3202116 GenericJdbcDatabaseAccessor.java:101]
> b34bb1ec996a2df9:a00f489200000000] Caught exception while trying to get the
> number of records
> Java exception follows:
> java.sql.SQLDataException: Value '3379400792' is outside of valid range for
> type java.lang.Integer
> at
> com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:114)
> at
> com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
> at
> com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:89)
> at
> com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:63)
> at
> com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:73)
> at
> com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:92)
> at
> com.mysql.cj.jdbc.result.ResultSetImpl.getObject(ResultSetImpl.java:1431)
> at com.mysql.cj.jdbc.result.ResultSetImpl.getInt(ResultSetImpl.java:830)
> at
> org.apache.commons.dbcp2.DelegatingResultSet.getInt(DelegatingResultSet.java:623)
> at
> org.apache.commons.dbcp2.DelegatingResultSet.getInt(DelegatingResultSet.java:623)
> at
> org.apache.impala.extdatasource.jdbc.dao.GenericJdbcDatabaseAccessor.getTotalNumberOfRecords(GenericJdbcDatabaseAccessor.java:92)
> at
> org.apache.impala.extdatasource.jdbc.JdbcDataSource.buildQueryAndExecute(JdbcDataSource.java:335)
> at
> org.apache.impala.extdatasource.jdbc.JdbcDataSource.open(JdbcDataSource.java:157)
> at
> org.apache.impala.extdatasource.ExternalDataSourceExecutor.open(ExternalDataSourceExecutor.java:274)
> at
> org.apache.impala.extdatasource.ExternalDataSourceExecutor.open(ExternalDataSourceExecutor.java:217)
> Caused by: com.mysql.cj.exceptions.NumberOutOfRange: Value '3379400792' is
> outside of valid range for type java.lang.Integer
> at
> com.mysql.cj.result.IntegerValueFactory.createFromLong(IntegerValueFactory.java:62)
> at
> com.mysql.cj.result.IntegerValueFactory.createFromLong(IntegerValueFactory.java:44)
> at
> com.mysql.cj.protocol.a.MysqlTextValueDecoder.decodeInt8(MysqlTextValueDecoder.java:126)
> at
> com.mysql.cj.protocol.result.AbstractResultsetRow.decodeAndCreateReturnValue(AbstractResultsetRow.java:113)
> at
> com.mysql.cj.protocol.result.AbstractResultsetRow.getValueFromBytes(AbstractResultsetRow.java:243)
> at
> com.mysql.cj.protocol.a.result.ByteArrayRow.getValue(ByteArrayRow.java:91)
> at
> com.mysql.cj.jdbc.result.ResultSetImpl.getObject(ResultSetImpl.java:1324)
> ... 8 more {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
