[
https://issues.apache.org/jira/browse/HIVE-26350?focusedWorklogId=797160&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-797160
]
ASF GitHub Bot logged work on HIVE-26350:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 02/Aug/22 09:09
Start Date: 02/Aug/22 09:09
Worklog Time Spent: 10m
Work Description: zabetak commented on code in PR #3470:
URL: https://github.com/apache/hive/pull/3470#discussion_r935307984
##########
jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/dao/GenericJdbcDatabaseAccessor.java:
##########
@@ -89,23 +92,76 @@ public List<String> getColumnNames(Configuration conf)
throws HiveJdbcDatabaseAc
ResultSetMetaData metadata = rs.getMetaData();
int numColumns = metadata.getColumnCount();
- List<String> columnNames = new ArrayList<String>(numColumns);
+ List<T> columnMeta = new ArrayList<>(numColumns);
for (int i = 0; i < numColumns; i++) {
- columnNames.add(metadata.getColumnName(i + 1));
+ columnMeta.add(colAccessor.get(metadata, i + 1));
}
- return columnNames;
+ return columnMeta;
}
catch (Exception e) {
- LOGGER.error("Error while trying to get column names.", e);
- throw new HiveJdbcDatabaseAccessException("Error while trying to get
column names: " + e.getMessage(), e);
+ throw new HiveJdbcDatabaseAccessException("", e);
Review Comment:
The choice boils down to which stacktrace we prefer to see when a problem
occurs. Below, we have the three alternatives mentioned here:
**Alternative 1**
```java
throw new HiveJdbcDatabaseAccessException("", e);
org.apache.hive.storage.jdbc.exception.HiveJdbcDatabaseAccessException:
at
org.apache.hive.storage.jdbc.dao.GenericJdbcDatabaseAccessor.getColumnMetadata(GenericJdbcDatabaseAccessor.java:103)
at
org.apache.hive.storage.jdbc.dao.GenericJdbcDatabaseAccessor.getColumnTypes(GenericJdbcDatabaseAccessor.java:118)
at
org.apache.hive.storage.jdbc.dao.TestGenericJdbcDatabaseAccessor.testGetColumnTypes_fieldListQuery(TestGenericJdbcDatabaseAccessor.java:113)
Caused by: org.h2.jdbc.JdbcSQLSyntaxErrorException: Table "T_STRATEGY" not
found; SQL statement:
select name,referrer from t_strategy LIMIT 1 [42102-210]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:521)
at org.h2.message.DbException.getJdbcSQLException(DbException.java:496)
at org.h2.message.DbException.get(DbException.java:227)
at org.h2.message.DbException.get(DbException.java:203)
at
org.h2.command.Parser.getTableOrViewNotFoundDbException(Parser.java:8410)
at
org.h2.command.Parser.getTableOrViewNotFoundDbException(Parser.java:8381)
at org.h2.command.Parser.readTableOrView(Parser.java:8370)
at org.h2.command.Parser.readTablePrimary(Parser.java:1866)
at org.h2.command.Parser.readTableReference(Parser.java:2342)
at org.h2.command.Parser.parseSelectFromPart(Parser.java:2793)
at org.h2.command.Parser.parseSelect(Parser.java:2900)
at org.h2.command.Parser.parseQueryPrimary(Parser.java:2783)
at org.h2.command.Parser.parseQueryTerm(Parser.java:2650)
at org.h2.command.Parser.parseQueryExpressionBody(Parser.java:2629)
at
org.h2.command.Parser.parseQueryExpressionBodyAndEndOfQuery(Parser.java:2622)
at org.h2.command.Parser.parseQueryExpression(Parser.java:2615)
at org.h2.command.Parser.parseQuery(Parser.java:2576)
at org.h2.command.Parser.parsePrepared(Parser.java:707)
at org.h2.command.Parser.parse(Parser.java:674)
at org.h2.command.Parser.parse(Parser.java:644)
at org.h2.command.Parser.prepareCommand(Parser.java:551)
at org.h2.engine.SessionLocal.prepareLocal(SessionLocal.java:615)
at org.h2.engine.SessionLocal.prepareCommand(SessionLocal.java:553)
at org.h2.jdbc.JdbcConnection.prepareCommand(JdbcConnection.java:1116)
at
org.h2.jdbc.JdbcPreparedStatement.<init>(JdbcPreparedStatement.java:92)
at org.h2.jdbc.JdbcConnection.prepareStatement(JdbcConnection.java:288)
at
org.apache.commons.dbcp2.DelegatingConnection.prepareStatement(DelegatingConnection.java:303)
at
org.apache.commons.dbcp2.DelegatingConnection.prepareStatement(DelegatingConnection.java:303)
at
org.apache.hive.storage.jdbc.dao.GenericJdbcDatabaseAccessor.getColumnMetadata(GenericJdbcDatabaseAccessor.java:90)
... 46 more
```
**Alternative 2**
```java
throw new HiveJdbcDatabaseAccessException(e);
org.apache.hive.storage.jdbc.exception.HiveJdbcDatabaseAccessException:
org.h2.jdbc.JdbcSQLSyntaxErrorException: Table "T_STRATEGY" not found; SQL
statement:
select name,referrer from t_strategy LIMIT 1 [42102-210]
at
org.apache.hive.storage.jdbc.dao.GenericJdbcDatabaseAccessor.getColumnMetadata(GenericJdbcDatabaseAccessor.java:103)
at
org.apache.hive.storage.jdbc.dao.GenericJdbcDatabaseAccessor.getColumnTypes(GenericJdbcDatabaseAccessor.java:118)
at
org.apache.hive.storage.jdbc.dao.TestGenericJdbcDatabaseAccessor.testGetColumnTypes_fieldListQuery(TestGenericJdbcDatabaseAccessor.java:113)
Caused by: org.h2.jdbc.JdbcSQLSyntaxErrorException: Table "T_STRATEGY" not
found; SQL statement:
select name,referrer from t_strategy LIMIT 1 [42102-210]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:521)
at org.h2.message.DbException.getJdbcSQLException(DbException.java:496)
at org.h2.message.DbException.get(DbException.java:227)
at org.h2.message.DbException.get(DbException.java:203)
at
org.h2.command.Parser.getTableOrViewNotFoundDbException(Parser.java:8410)
at
org.h2.command.Parser.getTableOrViewNotFoundDbException(Parser.java:8381)
at org.h2.command.Parser.readTableOrView(Parser.java:8370)
at org.h2.command.Parser.readTablePrimary(Parser.java:1866)
at org.h2.command.Parser.readTableReference(Parser.java:2342)
at org.h2.command.Parser.parseSelectFromPart(Parser.java:2793)
at org.h2.command.Parser.parseSelect(Parser.java:2900)
at org.h2.command.Parser.parseQueryPrimary(Parser.java:2783)
at org.h2.command.Parser.parseQueryTerm(Parser.java:2650)
at org.h2.command.Parser.parseQueryExpressionBody(Parser.java:2629)
at
org.h2.command.Parser.parseQueryExpressionBodyAndEndOfQuery(Parser.java:2622)
at org.h2.command.Parser.parseQueryExpression(Parser.java:2615)
at org.h2.command.Parser.parseQuery(Parser.java:2576)
at org.h2.command.Parser.parsePrepared(Parser.java:707)
at org.h2.command.Parser.parse(Parser.java:674)
at org.h2.command.Parser.parse(Parser.java:644)
at org.h2.command.Parser.prepareCommand(Parser.java:551)
at org.h2.engine.SessionLocal.prepareLocal(SessionLocal.java:615)
at org.h2.engine.SessionLocal.prepareCommand(SessionLocal.java:553)
at org.h2.jdbc.JdbcConnection.prepareCommand(JdbcConnection.java:1116)
at
org.h2.jdbc.JdbcPreparedStatement.<init>(JdbcPreparedStatement.java:92)
at org.h2.jdbc.JdbcConnection.prepareStatement(JdbcConnection.java:288)
at
org.apache.commons.dbcp2.DelegatingConnection.prepareStatement(DelegatingConnection.java:303)
at
org.apache.commons.dbcp2.DelegatingConnection.prepareStatement(DelegatingConnection.java:303)
at
org.apache.hive.storage.jdbc.dao.GenericJdbcDatabaseAccessor.getColumnMetadata(GenericJdbcDatabaseAccessor.java:90)
... 27 more
```
**Alternative 3**
```java
throw new HiveJdbcDatabaseAccessException("Error while trying to get column
metadata", e);
org.apache.hive.storage.jdbc.exception.HiveJdbcDatabaseAccessException:
Error while trying to get column metadata
at
org.apache.hive.storage.jdbc.dao.GenericJdbcDatabaseAccessor.getColumnMetadata(GenericJdbcDatabaseAccessor.java:103)
at
org.apache.hive.storage.jdbc.dao.GenericJdbcDatabaseAccessor.getColumnTypes(GenericJdbcDatabaseAccessor.java:118)
at
org.apache.hive.storage.jdbc.dao.TestGenericJdbcDatabaseAccessor.testGetColumnTypes_fieldListQuery(TestGenericJdbcDatabaseAccessor.java:113)
Caused by: org.h2.jdbc.JdbcSQLSyntaxErrorException: Table "T_STRATEGY" not
found; SQL statement:
select name,referrer from t_strategy LIMIT 1 [42102-210]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:521)
at org.h2.message.DbException.getJdbcSQLException(DbException.java:496)
at org.h2.message.DbException.get(DbException.java:227)
at org.h2.message.DbException.get(DbException.java:203)
at
org.h2.command.Parser.getTableOrViewNotFoundDbException(Parser.java:8410)
at
org.h2.command.Parser.getTableOrViewNotFoundDbException(Parser.java:8381)
at org.h2.command.Parser.readTableOrView(Parser.java:8370)
at org.h2.command.Parser.readTablePrimary(Parser.java:1866)
at org.h2.command.Parser.readTableReference(Parser.java:2342)
at org.h2.command.Parser.parseSelectFromPart(Parser.java:2793)
at org.h2.command.Parser.parseSelect(Parser.java:2900)
at org.h2.command.Parser.parseQueryPrimary(Parser.java:2783)
at org.h2.command.Parser.parseQueryTerm(Parser.java:2650)
at org.h2.command.Parser.parseQueryExpressionBody(Parser.java:2629)
at
org.h2.command.Parser.parseQueryExpressionBodyAndEndOfQuery(Parser.java:2622)
at org.h2.command.Parser.parseQueryExpression(Parser.java:2615)
at org.h2.command.Parser.parseQuery(Parser.java:2576)
at org.h2.command.Parser.parsePrepared(Parser.java:707)
at org.h2.command.Parser.parse(Parser.java:674)
at org.h2.command.Parser.parse(Parser.java:644)
at org.h2.command.Parser.prepareCommand(Parser.java:551)
at org.h2.engine.SessionLocal.prepareLocal(SessionLocal.java:615)
at org.h2.engine.SessionLocal.prepareCommand(SessionLocal.java:553)
at org.h2.jdbc.JdbcConnection.prepareCommand(JdbcConnection.java:1116)
at
org.h2.jdbc.JdbcPreparedStatement.<init>(JdbcPreparedStatement.java:92)
at org.h2.jdbc.JdbcConnection.prepareStatement(JdbcConnection.java:288)
at
org.apache.commons.dbcp2.DelegatingConnection.prepareStatement(DelegatingConnection.java:303)
at
org.apache.commons.dbcp2.DelegatingConnection.prepareStatement(DelegatingConnection.java:303)
at
org.apache.hive.storage.jdbc.dao.GenericJdbcDatabaseAccessor.getColumnMetadata(GenericJdbcDatabaseAccessor.java:90)
... 27 more
```
Personally, I prefer _Alt1_ since it conveys the problem without redundant
information. Next in the list is _Alt3_ and last goes _Alt2_ which makes the
exception quite verbose.
@kasakrisz let me know which one you prefer and I will apply the changes.
Issue Time Tracking
-------------------
Worklog Id: (was: 797160)
Time Spent: 1h (was: 50m)
> IndexOutOfBoundsException when generating splits for external JDBC table with
> partition columns
> -----------------------------------------------------------------------------------------------
>
> Key: HIVE-26350
> URL: https://issues.apache.org/jira/browse/HIVE-26350
> Project: Hive
> Issue Type: Bug
> Components: CBO, JDBC storage handler
> Reporter: Stamatis Zampetakis
> Assignee: Soumyakanti Das
> Priority: Major
> Labels: pull-request-available
> Attachments: cbo_plan.txt, explain_plan.txt,
> jdbc_join_with_partition_table.q
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Create the following table in some JDBC database (e.g., Postgres).
> {code:sql}
> CREATE TABLE country
> (
> id int,
> name varchar(20)
> );
> {code}
> Create the following tables in Hive ensuring that the external JDBC table has
> the {{hive.sql.partitionColumn}} table property set.
> {code:sql}
> CREATE TABLE city (id int);
> CREATE EXTERNAL TABLE country
> (
> id int,
> name varchar(20)
> )
> STORED BY
> 'org.apache.hive.storage.jdbc.JdbcStorageHandler'
> TBLPROPERTIES (
> "hive.sql.database.type" = "POSTGRES",
> "hive.sql.jdbc.driver" = "org.postgresql.Driver",
> "hive.sql.jdbc.url" = "jdbc:postgresql://localhost:5432/qtestDB",
> "hive.sql.dbcp.username" = "qtestuser",
> "hive.sql.dbcp.password" = "qtestpassword",
> "hive.sql.table" = "country",
> "hive.sql.partitionColumn" = "name",
> "hive.sql.numPartitions" = "2"
> );
> {code}
> The query below fails with IndexOutOfBoundsException when the mapper scanning
> the JDBC table tries to generate the splits by exploiting the partitioning
> column.
> {code:sql}
> select country.id from country cross join city;
> {code}
> The full stack trace is given below.
> {noformat}
> java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
> at java.util.ArrayList.rangeCheck(ArrayList.java:659) ~[?:1.8.0_261]
> at java.util.ArrayList.get(ArrayList.java:435) ~[?:1.8.0_261]
> at
> org.apache.hive.storage.jdbc.JdbcInputFormat.getSplits(JdbcInputFormat.java:102)
> [hive-jdbc-handler-4.0.0-alpha-2-SNAPSHOT.jar:4.0.0-alpha-2-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.addSplitsForGroup(HiveInputFormat.java:564)
> [hive-exec-4.0.0-alpha-2-SNAPSHOT.jar:4.0.0-alpha-2-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getSplits(HiveInputFormat.java:858)
> [hive-exec-4.0.0-alpha-2-SNAPSHOT.jar:4.0.0-alpha-2-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:263)
> [hive-exec-4.0.0-alpha-2-SNAPSHOT.jar:4.0.0-alpha-2-SNAPSHOT]
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:281)
> [tez-dag-0.10.1.jar:0.10.1]
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:272)
> [tez-dag-0.10.1.jar:0.10.1]
> at java.security.AccessController.doPrivileged(Native Method)
> ~[?:1.8.0_261]
> at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_261]
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
> [hadoop-common-3.1.0.jar:?]
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:272)
> [tez-dag-0.10.1.jar:0.10.1]
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:256)
> [tez-dag-0.10.1.jar:0.10.1]
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> [guava-19.0.jar:?]
> at
> com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> [guava-19.0.jar:?]
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> [guava-19.0.jar:?]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [?:1.8.0_261]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [?:1.8.0_261]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_261]
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
