[
https://issues.apache.org/jira/browse/HIVE-24994?focusedWorklogId=579845&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-579845
]
ASF GitHub Bot logged work on HIVE-24994:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 09/Apr/21 08:39
Start Date: 09/Apr/21 08:39
Worklog Time Spent: 10m
Work Description: vnhive commented on a change in pull request #2162:
URL: https://github.com/apache/hive/pull/2162#discussion_r610452112
##########
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/DatabaseProduct.java
##########
@@ -203,7 +203,7 @@ public boolean isTableNotExistsError(SQLException e) {
* Whether the RDBMS has restrictions on IN list size (explicit, or poor
perf-based).
*/
protected boolean needsInBatching() {
- return isORACLE() || isSQLSERVER();
+ return isORACLE() || isSQLSERVER() || isPOSTGRES();
Review comment:
Setting hive.metastore.direct.sql.batch.size should have the same effect
as this fix. We can just ask the customer to set this configuration variable
instead of needing this fix.
This was the reason I did not submit a PR with this fix in the first place.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 579845)
Time Spent: 20m (was: 10m)
> get_aggr_stats_for call fail with "Tried to send an out-of-range integer"
> -------------------------------------------------------------------------
>
> Key: HIVE-24994
> URL: https://issues.apache.org/jira/browse/HIVE-24994
> Project: Hive
> Issue Type: Bug
> Components: Hive
> Reporter: Rajkumar Singh
> Assignee: Rajkumar Singh
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> aggrColStatsForPartitions call fail with the Postgres LIMIT if the no of
> partitions passed in the direct sql goes beyond the 32767
> {code:java}
> postgresql.util.PSQLException: An I/O error occurred while sending to the
> backend.
> at
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:337)
> ~[postgresql-42.2.8.jar:42.2.8]
> at
> org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:446)
> ~[postgresql-42.2.8.jar:42.2.8]
> at
> org.postgresql.jdbc.PgStatement.execute(PgStatement.java:370)
> ~[postgresql-42.2.8.jar:42.2.8]
> at
> org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:149)
> ~[postgresql-42.2.8.jar:42.2.8]
> at
> org.postgresql.jdbc.PgPreparedStatement.executeQuery(PgPreparedStatement.java:108)
> ~[postgresql-42.2.8.jar:42.2.8]
> at
> com.zaxxer.hikari.pool.ProxyPreparedStatement.executeQuery(ProxyPreparedStatement.java:52)
> ~[HikariCP-2.6.1.jar:?]
> at
> com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeQuery(HikariProxyPreparedStatement.java)
> [HikariCP-2.6.1.jar:?]
> at
> org.datanucleus.store.rdbms.ParamLoggingPreparedStatement.executeQuery(ParamLoggingPreparedStatement.java:375)
> [datanucleus-rdbms-4.1.19.jar:?]
> at
> org.datanucleus.store.rdbms.SQLController.executeStatementQuery(SQLController.java:552)
> [datanucleus-rdbms-4.1.19.jar:?]
> at
> org.datanucleus.store.rdbms.query.SQLQuery.performExecute(SQLQuery.java:645)
> [datanucleus-rdbms-4.1.19.jar:?]
> at
> org.datanucleus.store.query.Query.executeQuery(Query.java:1855)
> [datanucleus-core-4.1.17.jar:?]
> at
> org.datanucleus.store.rdbms.query.SQLQuery.executeWithArray(SQLQuery.java:807)
> [datanucleus-rdbms-4.1.19.jar:?]
> at
> org.datanucleus.api.jdo.JDOQuery.executeInternal(JDOQuery.java:368)
> [datanucleus-api-jdo-4.2.4.jar:?]
> at
> org.datanucleus.api.jdo.JDOQuery.executeWithArray(JDOQuery.java:267)
> [datanucleus-api-jdo-4.2.4.jar:?]
> at
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql.executeWithArray(MetaStoreDirectSql.java:2058)
> [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
> at
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql.executeWithArray(MetaStoreDirectSql.java:2050)
> [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
> at
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql.access$1500(MetaStoreDirectSql.java:110)
> [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
> at
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql$15$1.run(MetaStoreDirectSql.java:1530)
> [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
> at
> org.apache.hadoop.hive.metastore.Batchable.runBatched(Batchable.java:73)
> [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
> at
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql$15.run(MetaStoreDirectSql.java:1521)
> [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
> at
> org.apache.hadoop.hive.metastore.Batchable.runBatched(Batchable.java:73)
> [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
> at
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql.partsFoundForPartitions(MetaStoreDirectSql.java:1518)
> [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
> at
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql.aggrColStatsForPartitions(MetaStoreDirectSql.java:1489)
> [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
> at
> org.apache.hadoop.hive.metastore.ObjectStore$20.getSqlResult(ObjectStore.java:8966)
> [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
> at
> org.apache.hadoop.hive.metastore.ObjectStore$20.getSqlResult(ObjectStore.java:8962)
> [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
> at
> org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.run(ObjectStore.java:3757)
> [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
> at
> org.apache.hadoop.hive.metastore.ObjectStore.get_aggr_stats_for(ObjectStore.java:8981)
> [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
> at
> org.apache.hadoop.hive.metastore.ObjectStore.get_aggr_stats_for(ObjectStore.java:8951)
> [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
> at sun.reflect.GeneratedMethodAccessor65.invoke(Unknown
> Source) ~[?:?]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_261]
> at java.lang.reflect.Method.invoke(Method.java:498)
> ~[?:1.8.0_261]
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97)
> [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
> at com.sun.proxy.$Proxy37.get_aggr_stats_for(Unknown Source)
> [?:?]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_aggr_stats_for(HiveMetaStore.java:8254)
> [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
> at sun.reflect.GeneratedMethodAccessor64
> Caused by: java.io.IOException: Tried to send an out-of-range
> integer as a 2-byte value: 83666
> at org.postgresql.core.PGStream.sendInteger2(PGStream.java:266)
> ~[postgresql-42.2.8.jar:42.2.8]
> at
> org.postgresql.core.v3.QueryExecutorImpl.sendParse(QueryExecutorImpl.java:1492)
> ~[postgresql-42.2.8.jar:42.2.8]
> at
> org.postgresql.core.v3.QueryExecutorImpl.sendOneQuery(QueryExecutorImpl.java:1815)
> ~[postgresql-42.2.8.jar:42.2.8]
> at
> org.postgresql.core.v3.QueryExecutorImpl.sendQuery(QueryExecutorImpl.java:1378)
> ~[postgresql-42.2.8.jar:42.2.8]
> at
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:302)
> ~[postgresql-42.2.8.jar:42.2.8]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
