[jira] [Created] (HIVE-23483) Remove DynamicSerDe
Ashutosh Chauhan created HIVE-23483: --- Summary: Remove DynamicSerDe Key: HIVE-23483 URL: https://issues.apache.org/jira/browse/HIVE-23483 Project: Hive Issue Type: Task Reporter: Ashutosh Chauhan It is used to read thrift data files. AFAIK no one uses thrift for data serialization. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-23447) Avoid sending configs to tasks and AM which are only relevant for HS2
Ashutosh Chauhan created HIVE-23447: --- Summary: Avoid sending configs to tasks and AM which are only relevant for HS2 Key: HIVE-23447 URL: https://issues.apache.org/jira/browse/HIVE-23447 Project: Hive Issue Type: Task Components: Configuration Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan There are many configs which are only relevant for HS2. Longer term fix for this is to split HiveConf in multiple config classes relevant only for HS2, HMS, AM and tasks. And use only the objects in process where its relevant. In the interim, we can avoid configs with large value strings to send across. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-23445) Remove mapreduce.workflow.* configs
Ashutosh Chauhan created HIVE-23445: --- Summary: Remove mapreduce.workflow.* configs Key: HIVE-23445 URL: https://issues.apache.org/jira/browse/HIVE-23445 Project: Hive Issue Type: Task Components: Configuration Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan These configs were introduced in HIVE-3708 in the hope to develop tools to visualize and monitor multiple MR jobs from Hive back in a day when MR was used. Even that time in spite of these config additions, no such tools were developed AFAIK. And now MR is hardly ever used. We can get rid of these configs. That will help to reduce the size of HiveConf by a bit. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-23287) Reduce dependency on icu4j
Ashutosh Chauhan created HIVE-23287: --- Summary: Reduce dependency on icu4j Key: HIVE-23287 URL: https://issues.apache.org/jira/browse/HIVE-23287 Project: Hive Issue Type: Improvement Components: Druid integration Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Brought in transitively via druid. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-23278) Remove dependency on bouncycastle
Ashutosh Chauhan created HIVE-23278: --- Summary: Remove dependency on bouncycastle Key: HIVE-23278 URL: https://issues.apache.org/jira/browse/HIVE-23278 Project: Hive Issue Type: Improvement Components: Hive Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-23268) Eliminate beanutils transitive dependency
Ashutosh Chauhan created HIVE-23268: --- Summary: Eliminate beanutils transitive dependency Key: HIVE-23268 URL: https://issues.apache.org/jira/browse/HIVE-23268 Project: Hive Issue Type: Improvement Components: Hive Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Transitively retrieved from hadoop-commons -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-23267) Reduce dependency on groovy
Ashutosh Chauhan created HIVE-23267: --- Summary: Reduce dependency on groovy Key: HIVE-23267 URL: https://issues.apache.org/jira/browse/HIVE-23267 Project: Hive Issue Type: Improvement Components: Query Planning Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Transitively pulled where its unneeded. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-23262) Remove dependency on activemq
Ashutosh Chauhan created HIVE-23262: --- Summary: Remove dependency on activemq Key: HIVE-23262 URL: https://issues.apache.org/jira/browse/HIVE-23262 Project: Hive Issue Type: Improvement Components: HCatalog Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Activemq is a test dependency introduced to test message bus feature in hcatalog. Even when it was introduced very first, there were concerns of taking activemq as a dependency. https://issues.apache.org/jira/browse/HCATALOG-3?focusedCommentId=13035113=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-13035113 AFAIK no one uses message bus feature of HCatalog. We can remove it altogether. As a first step removing tests for it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-23241) Reduce transitive dependencies
Ashutosh Chauhan created HIVE-23241: --- Summary: Reduce transitive dependencies Key: HIVE-23241 URL: https://issues.apache.org/jira/browse/HIVE-23241 Project: Hive Issue Type: Improvement Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-22745) Config option to turn off read locks
Ashutosh Chauhan created HIVE-22745: --- Summary: Config option to turn off read locks Key: HIVE-22745 URL: https://issues.apache.org/jira/browse/HIVE-22745 Project: Hive Issue Type: Improvement Components: Transactions Reporter: Ashutosh Chauhan Although its not recommended but in perf critical scenario this option may be exercised. We have observed lock acquisition to take long time in heavily loaded system. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-22697) Remove expensive RDBMS queries in lock acquisition logic
Ashutosh Chauhan created HIVE-22697: --- Summary: Remove expensive RDBMS queries in lock acquisition logic Key: HIVE-22697 URL: https://issues.apache.org/jira/browse/HIVE-22697 Project: Hive Issue Type: Improvement Components: Transactions Reporter: Gopal Vijayaraghavan There are queries like "update NEXT_LOCK_ID set nl_next = 163084841;" which can be entirely skipped the ACID impl doesn't require globally unique lock ids because locks are marked as txn.id In a particular instance we observed this logic taking multiple seconds in mysql -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-22151) Turn off hybrid grace hash join by default
Ashutosh Chauhan created HIVE-22151: --- Summary: Turn off hybrid grace hash join by default Key: HIVE-22151 URL: https://issues.apache.org/jira/browse/HIVE-22151 Project: Hive Issue Type: Improvement Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (HIVE-22112) update jackson version in disconnected poms
Ashutosh Chauhan created HIVE-22112: --- Summary: update jackson version in disconnected poms Key: HIVE-22112 URL: https://issues.apache.org/jira/browse/HIVE-22112 Project: Hive Issue Type: Improvement Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan was updated in top level pom via HIVE-22089 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (HIVE-22090) Upgrade jetty to 9.3.27
Ashutosh Chauhan created HIVE-22090: --- Summary: Upgrade jetty to 9.3.27 Key: HIVE-22090 URL: https://issues.apache.org/jira/browse/HIVE-22090 Project: Hive Issue Type: Improvement Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (HIVE-22089) Upgrade jackson to 2.9.9
Ashutosh Chauhan created HIVE-22089: --- Summary: Upgrade jackson to 2.9.9 Key: HIVE-22089 URL: https://issues.apache.org/jira/browse/HIVE-22089 Project: Hive Issue Type: Improvement Affects Versions: 3.1.1, 3.1.0, 3.0.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (HIVE-21815) Stats in ORC file are parsed twice
Ashutosh Chauhan created HIVE-21815: --- Summary: Stats in ORC file are parsed twice Key: HIVE-21815 URL: https://issues.apache.org/jira/browse/HIVE-21815 Project: Hive Issue Type: Improvement Components: ORC Reporter: Gopal V ORC record reader unnecessarily parses stats twice -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-21427) Syslog serde
Ashutosh Chauhan created HIVE-21427: --- Summary: Syslog serde Key: HIVE-21427 URL: https://issues.apache.org/jira/browse/HIVE-21427 Project: Hive Issue Type: New Feature Components: Serializers/Deserializers Reporter: Ashutosh Chauhan It will be useful to read syslog generated log files in Hive -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-21341) Sensible defaults : hive.server2.idle.operation.timeout and hive.server2.idle.session.timeout are too high
Ashutosh Chauhan created HIVE-21341: --- Summary: Sensible defaults : hive.server2.idle.operation.timeout and hive.server2.idle.session.timeout are too high Key: HIVE-21341 URL: https://issues.apache.org/jira/browse/HIVE-21341 Project: Hive Issue Type: Improvement Components: Configuration, HiveServer2 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Defaults are too high, which results in extra resources being held too long in HS2 memory. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-21189) hive.merge.nway.joins should default to false
Ashutosh Chauhan created HIVE-21189: --- Summary: hive.merge.nway.joins should default to false Key: HIVE-21189 URL: https://issues.apache.org/jira/browse/HIVE-21189 Project: Hive Issue Type: Improvement Components: Configuration Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-20880) Update default value for hive.stats.filter.in.min.ratio
Ashutosh Chauhan created HIVE-20880: --- Summary: Update default value for hive.stats.filter.in.min.ratio Key: HIVE-20880 URL: https://issues.apache.org/jira/browse/HIVE-20880 Project: Hive Issue Type: Improvement Components: Configuration Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-20572) Change default value of hive.tez.llap.min.reducer.per.executor
Ashutosh Chauhan created HIVE-20572: --- Summary: Change default value of hive.tez.llap.min.reducer.per.executor Key: HIVE-20572 URL: https://issues.apache.org/jira/browse/HIVE-20572 Project: Hive Issue Type: Improvement Components: Configuration Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-20571) Change default value of hive.tez.dag.status.check.interval
Ashutosh Chauhan created HIVE-20571: --- Summary: Change default value of hive.tez.dag.status.check.interval Key: HIVE-20571 URL: https://issues.apache.org/jira/browse/HIVE-20571 Project: Hive Issue Type: Improvement Components: Configuration Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-20558) Change default of hive.hashtable.key.count.adjustment to 0.99
Ashutosh Chauhan created HIVE-20558: --- Summary: Change default of hive.hashtable.key.count.adjustment to 0.99 Key: HIVE-20558 URL: https://issues.apache.org/jira/browse/HIVE-20558 Project: Hive Issue Type: Improvement Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Current default is 2 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-20518) TxnHandler checkLock direct sql fail with ORA-01795 , if the table has more than 1000 partitions
Ashutosh Chauhan created HIVE-20518: --- Summary: TxnHandler checkLock direct sql fail with ORA-01795 , if the table has more than 1000 partitions Key: HIVE-20518 URL: https://issues.apache.org/jira/browse/HIVE-20518 Project: Hive Issue Type: Bug Components: Transactions Affects Versions: 2.1.0 Reporter: Rajkumar Singh Assignee: Rajkumar Singh with Oracle as Metastore, txnhandler checkLock fail with "checkLockWithRetry(181398,34773) : ORA-01795: maximum number of expressions in a list is 1000" if the write table has more than 1000 partitions. complete stacktrace {code} txn.TxnHandler (TxnHandler.java:checkRetryable(2099)) - Non-retryable error in checkLockWithRetry(181398,34773) : ORA-01795: maximum number of expressions in a list is 1000 (SQLState=42000, ErrorCode=1795) 2018-06-25 15:09:35,999 ERROR [pool-7-thread-197]: metastore.RetryingHMSHandler (RetryingHMSHandler.java:invokeInternal(203)) - MetaException(message:Unable to update transaction database java.sql.SQLSyntaxErrorException: ORA-01795: maximum number of expressions in a list is 1000 at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447) at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396) at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951) at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513) at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227) at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531) at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:195) at oracle.jdbc.driver.T4CStatement.executeForDescribe(T4CStatement.java:876) at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175) at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1296) at oracle.jdbc.driver.OracleStatement.executeQuery(OracleStatement.java:1498) at oracle.jdbc.driver.OracleStatementWrapper.executeQuery(OracleStatementWrapper.java:406) at com.jolbox.bonecp.StatementHandle.executeQuery(StatementHandle.java:464) at org.apache.hadoop.hive.metastore.txn.TxnHandler.checkLock(TxnHandler.java:2649) at org.apache.hadoop.hive.metastore.txn.TxnHandler.checkLockWithRetry(TxnHandler.java:1126) at org.apache.hadoop.hive.metastore.txn.TxnHandler.lock(TxnHandler.java:895) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.lock(HiveMetaStore.java:6123) at sun.reflect.GeneratedMethodAccessor90.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:105) at com.sun.proxy.$Proxy11.lock(Unknown Source) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$lock.getResult(ThriftHiveMetastore.java:12012) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$lock.getResult(ThriftHiveMetastore.java:11996) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:551) at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:546) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866) at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:546) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) ) at org.apache.hadoop.hive.metastore.txn.TxnHandler.checkLockWithRetry(TxnHandler.java:1131) at org.apache.hadoop.hive.metastore.txn.TxnHandler.lock(TxnHandler.java:895) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.lock(HiveMetaStore.java:6123) at sun.reflect.GeneratedMethodAccessor90.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
[jira] [Created] (HIVE-20364) Update default for hive.map.aggr.hash.min.reduction
Ashutosh Chauhan created HIVE-20364: --- Summary: Update default for hive.map.aggr.hash.min.reduction Key: HIVE-20364 URL: https://issues.apache.org/jira/browse/HIVE-20364 Project: Hive Issue Type: Improvement Components: Configuration Reporter: Nita Dembla Assignee: Ashutosh Chauhan Default value is 0.5 Lets update it to 0.99 In average case its a trade-off between cpu vs network. Erring on side of CPU is better since perf loss caused by network is usually larger. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-20260) NDV of a column shouldn't be scaled when row count is changed by filter on another column
Ashutosh Chauhan created HIVE-20260: --- Summary: NDV of a column shouldn't be scaled when row count is changed by filter on another column Key: HIVE-20260 URL: https://issues.apache.org/jira/browse/HIVE-20260 Project: Hive Issue Type: Improvement Components: Statistics Reporter: Ashutosh Chauhan HIVE-17465 introduced progressive scaling of rowcounts in presence of multiple filters. HIVE-19500 improved on that by also scaling col stats (NDV) in such scenario. However, it should pay attention to column used in filter expression and not scale for all filters. eg., consider filter a = 1 and b = 2 ndv of column b should not be scaled down by row count changes caused by a = 1 Other way to say this that ndv of a particular column should be updated at the end of computation of row count for that operator. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-20114) hive.start.cleanup.scratchdir should default to true
Ashutosh Chauhan created HIVE-20114: --- Summary: hive.start.cleanup.scratchdir should default to true Key: HIVE-20114 URL: https://issues.apache.org/jira/browse/HIVE-20114 Project: Hive Issue Type: Task Components: Configuration Reporter: Ashutosh Chauhan -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-20078) Remove ATSHook
Ashutosh Chauhan created HIVE-20078: --- Summary: Remove ATSHook Key: HIVE-20078 URL: https://issues.apache.org/jira/browse/HIVE-20078 Project: Hive Issue Type: Task Components: Hooks Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-20049) hive.groupby.limit.extrastep should be false by default
Ashutosh Chauhan created HIVE-20049: --- Summary: hive.groupby.limit.extrastep should be false by default Key: HIVE-20049 URL: https://issues.apache.org/jira/browse/HIVE-20049 Project: Hive Issue Type: Task Components: Query Planning Reporter: Ashutosh Chauhan In fact this flag is not needed since FetchTask can enforce limit there is never a reason to have another vertex purely for limit. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-20048) Consider enabling hive.fetch.task.aggr by default
Ashutosh Chauhan created HIVE-20048: --- Summary: Consider enabling hive.fetch.task.aggr by default Key: HIVE-20048 URL: https://issues.apache.org/jira/browse/HIVE-20048 Project: Hive Issue Type: Task Components: Physical Optimizer Reporter: Ashutosh Chauhan This optimization is in code base for long time. We shall consider enabling it by default. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-20045) Update hidden config list
Ashutosh Chauhan created HIVE-20045: --- Summary: Update hidden config list Key: HIVE-20045 URL: https://issues.apache.org/jira/browse/HIVE-20045 Project: Hive Issue Type: Task Components: Configuration Reporter: Ashutosh Chauhan -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19619) Allow comparisons between doubles and bigints
Ashutosh Chauhan created HIVE-19619: --- Summary: Allow comparisons between doubles and bigints Key: HIVE-19619 URL: https://issues.apache.org/jira/browse/HIVE-19619 Project: Hive Issue Type: Bug Components: Types Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Hive doesn't allow binary comparison between doubles and bigints. That is overly restrictive since all other DBs (mysql, postgres, oracle and sql server) allows that. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19396) HiveOperation is incorrectly set for analyze statement
Ashutosh Chauhan created HIVE-19396: --- Summary: HiveOperation is incorrectly set for analyze statement Key: HIVE-19396 URL: https://issues.apache.org/jira/browse/HIVE-19396 Project: Hive Issue Type: Bug Components: Query Planning Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Because we rewrite analyze to select compute_stats() operation enum gets set to Query. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19390) Useless error messages logged for dummy table stats
Ashutosh Chauhan created HIVE-19390: --- Summary: Useless error messages logged for dummy table stats Key: HIVE-19390 URL: https://issues.apache.org/jira/browse/HIVE-19390 Project: Hive Issue Type: Improvement Components: Statistics Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Queries like INSERT INTO t1 VALUES (20); gets rewritten into insert into t1 select 20 from default_db.default_tblname which later throws as compiler tries to get stats -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19383) Add ArrayList$SubList kryo serializer
Ashutosh Chauhan created HIVE-19383: --- Summary: Add ArrayList$SubList kryo serializer Key: HIVE-19383 URL: https://issues.apache.org/jira/browse/HIVE-19383 Project: Hive Issue Type: Bug Components: Query Planning Reporter: Deepesh Khandelwal Assignee: Ashutosh Chauhan Otherwise failure is encountered while trying to deserialize such a plan. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19313) TestJdbcWithDBTokenStoreNoDoAs tests are failing
Ashutosh Chauhan created HIVE-19313: --- Summary: TestJdbcWithDBTokenStoreNoDoAs tests are failing Key: HIVE-19313 URL: https://issues.apache.org/jira/browse/HIVE-19313 Project: Hive Issue Type: Sub-task Components: Test Reporter: Ashutosh Chauhan Assignee: Thejas M Nair {code} Stacktrace java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://localhost:37606/default;principal=hive/localh...@example.com;: java.net.ConnectException: Connection refused at org.apache.hive.jdbc.HiveConnection.(HiveConnection.java:252) at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107) at java.sql.DriverManager.getConnection(DriverManager.java:664) at java.sql.DriverManager.getConnection(DriverManager.java:270) at org.apache.hive.minikdc.TestJdbcWithMiniKdc.testRenewDelegationToken(TestJdbcWithMiniKdc.java:180) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) {code} Failing repeatedly in Hive QA builds. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19252) TestJdbcWithMiniKdcCookie.testCookieNegative is failing consistently
Ashutosh Chauhan created HIVE-19252: --- Summary: TestJdbcWithMiniKdcCookie.testCookieNegative is failing consistently Key: HIVE-19252 URL: https://issues.apache.org/jira/browse/HIVE-19252 Project: Hive Issue Type: Sub-task Components: Test Reporter: Ashutosh Chauhan Assignee: Thejas M Nair For last 8 builds. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19246) Update golden files for negative tests
Ashutosh Chauhan created HIVE-19246: --- Summary: Update golden files for negative tests Key: HIVE-19246 URL: https://issues.apache.org/jira/browse/HIVE-19246 Project: Hive Issue Type: Sub-task Components: Tests Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan +Error during job, obtaining debugging information... shows up in q.out due to one of recent changes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19235) Update golden files for Minimr tests
Ashutosh Chauhan created HIVE-19235: --- Summary: Update golden files for Minimr tests Key: HIVE-19235 URL: https://issues.apache.org/jira/browse/HIVE-19235 Project: Hive Issue Type: Sub-task Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-19235.patch stats update needed for 3 tests. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19232) results_cache_invalidation2 is failing
Ashutosh Chauhan created HIVE-19232: --- Summary: results_cache_invalidation2 is failing Key: HIVE-19232 URL: https://issues.apache.org/jira/browse/HIVE-19232 Project: Hive Issue Type: Sub-task Components: Test Reporter: Ashutosh Chauhan Assignee: Jason Dere Fails with plan difference on both cli as well as minillaplocal. Plan diffs looks concerning since its now longer using cache. Also, it should run only on minillaplocal -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19227) Update golden files for negative tests
Ashutosh Chauhan created HIVE-19227: --- Summary: Update golden files for negative tests Key: HIVE-19227 URL: https://issues.apache.org/jira/browse/HIVE-19227 Project: Hive Issue Type: Sub-task Components: Tests Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Now that we are able to run TestNegativeCliDriver some of golden files are found to be outdated. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19218) Upgrade to Hadoop 3.1.0
Ashutosh Chauhan created HIVE-19218: --- Summary: Upgrade to Hadoop 3.1.0 Key: HIVE-19218 URL: https://issues.apache.org/jira/browse/HIVE-19218 Project: Hive Issue Type: Task Reporter: Ashutosh Chauhan -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19198) Few flaky hcatalog tests
Ashutosh Chauhan created HIVE-19198: --- Summary: Few flaky hcatalog tests Key: HIVE-19198 URL: https://issues.apache.org/jira/browse/HIVE-19198 Project: Hive Issue Type: Sub-task Reporter: Ashutosh Chauhan TestPermsGrp : Consider removing this since hcat cli is not widely used. TestHCatPartitionPublish.testPartitionPublish TestHCatMultiOutputFormat.testOutputFormat -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19197) TestReplicationScenarios is flaky
Ashutosh Chauhan created HIVE-19197: --- Summary: TestReplicationScenarios is flaky Key: HIVE-19197 URL: https://issues.apache.org/jira/browse/HIVE-19197 Project: Hive Issue Type: Sub-task Components: repl, Test Reporter: Ashutosh Chauhan Fails once in a while. {code} java.lang.AssertionError: expected:<1> but was:<0> at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.failNotEquals(Assert.java:743) at org.junit.Assert.assertEquals(Assert.java:118) at org.junit.Assert.assertEquals(Assert.java:555) at org.junit.Assert.assertEquals(Assert.java:542) at org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.verifyResults(TestReplicationScenarios.java:3629) at org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.verifyRun(TestReplicationScenarios.java:3711) at org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.verifyRun(TestReplicationScenarios.java:3706) at org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.verifyAndReturnTblReplStatus(TestReplicationScenarios.java:3600) {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19196) TestTriggersMoveWorkloadManager is flaky
Ashutosh Chauhan created HIVE-19196: --- Summary: TestTriggersMoveWorkloadManager is flaky Key: HIVE-19196 URL: https://issues.apache.org/jira/browse/HIVE-19196 Project: Hive Issue Type: Sub-task Components: Test Reporter: Ashutosh Chauhan This is a flaky test which randomly fails. Consider improving its stability. {code} org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveConflictKill Failing for the past 1 build (Since Failed#10161 ) Took 2.4 sec. Error Message '"eventType" : "GET"' expected in STDERR capture, but not found. Stacktrace java.lang.AssertionError: '"eventType" : "GET"' expected in STDERR capture, but not found. at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.assertTrue(Assert.java:41) at org.apache.hive.jdbc.AbstractJdbcTriggersTest.runQueryWithTrigger(AbstractJdbcTriggersTest.java:169) at org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveConflictKill(TestTriggersMoveWorkloadManager.java:261) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74) {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19195) .testCliDriver[auto_sortmerge_join_2] is flaky
Ashutosh Chauhan created HIVE-19195: --- Summary: .testCliDriver[auto_sortmerge_join_2] is flaky Key: HIVE-19195 URL: https://issues.apache.org/jira/browse/HIVE-19195 Project: Hive Issue Type: Sub-task Components: Test Reporter: Ashutosh Chauhan This test is certainly flaky. Seems like it makes some assumption about available memory. Consider dropping this altogether. {code} Client Execution succeeded but contained differences (error code = 1) after executing auto_sortmerge_join_2.q 1101,1103d1100 < Hive Runtime Error: Map local work exhausted memory < FAILED: Execution Error, return code 3 from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask < ATTEMPT: Execute BackupTask: org.apache.hadoop.hive.ql.exec.mr.MapRedTask {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19194) TestDruidStorageHandler fails
Ashutosh Chauhan created HIVE-19194: --- Summary: TestDruidStorageHandler fails Key: HIVE-19194 URL: https://issues.apache.org/jira/browse/HIVE-19194 Project: Hive Issue Type: Sub-task Components: Druid integration Reporter: Ashutosh Chauhan This tests fails randomly. If its not reproducible locally consider improving its stability since it does fail once in a while on Hive QA. {code} java.lang.AssertionError: expected:<0> but was:<1> at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.failNotEquals(Assert.java:743) at org.junit.Assert.assertEquals(Assert.java:118) at org.junit.Assert.assertEquals(Assert.java:555) at org.junit.Assert.assertEquals(Assert.java:542) at org.apache.hadoop.hive.druid.TestDruidStorageHandler.testCommitMultiInsertOverwriteTable(TestDruidStorageHandler.java:414) {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19193) TestActivePassiveHA fails
Ashutosh Chauhan created HIVE-19193: --- Summary: TestActivePassiveHA fails Key: HIVE-19193 URL: https://issues.apache.org/jira/browse/HIVE-19193 Project: Hive Issue Type: Sub-task Components: HiveServer2 Reporter: Ashutosh Chauhan This looks like a flaky test. If its not reproducible locally try improving on stability of this test, since it does fail randomly with Hive QA. {code:java} java.lang.AssertionError: expected: but was: at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.failNotEquals(Assert.java:743) at org.junit.Assert.assertEquals(Assert.java:118) at org.junit.Assert.assertEquals(Assert.java:144) at org.apache.hive.jdbc.TestActivePassiveHA.testManualFailover(TestActivePassiveHA.java:356) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74){code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19153) Update golden files for few tests
Ashutosh Chauhan created HIVE-19153: --- Summary: Update golden files for few tests Key: HIVE-19153 URL: https://issues.apache.org/jira/browse/HIVE-19153 Project: Hive Issue Type: Test Components: Test Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Some golden file updates which were missed since many tests were failing. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19146) Delete dangling q.out
Ashutosh Chauhan created HIVE-19146: --- Summary: Delete dangling q.out Key: HIVE-19146 URL: https://issues.apache.org/jira/browse/HIVE-19146 Project: Hive Issue Type: Test Components: Test Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Fails TestDanglingQOuts -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19145) Stabilize statsoptimizer.q test
Ashutosh Chauhan created HIVE-19145: --- Summary: Stabilize statsoptimizer.q test Key: HIVE-19145 URL: https://issues.apache.org/jira/browse/HIVE-19145 Project: Hive Issue Type: Test Components: Test Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Uses current_date() which is prone to fail. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19143) Update golden files for negative tests
Ashutosh Chauhan created HIVE-19143: --- Summary: Update golden files for negative tests Key: HIVE-19143 URL: https://issues.apache.org/jira/browse/HIVE-19143 Project: Hive Issue Type: Test Components: Test Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Missed in HIVE-18859 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19128) Update golden files for spark perf tests
Ashutosh Chauhan created HIVE-19128: --- Summary: Update golden files for spark perf tests Key: HIVE-19128 URL: https://issues.apache.org/jira/browse/HIVE-19128 Project: Hive Issue Type: Test Components: Spark Reporter: Ashutosh Chauhan -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-18793) Round udf should support variable as second argument
Ashutosh Chauhan created HIVE-18793: --- Summary: Round udf should support variable as second argument Key: HIVE-18793 URL: https://issues.apache.org/jira/browse/HIVE-18793 Project: Hive Issue Type: Improvement Components: UDF Reporter: Ashutosh Chauhan -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-18792) Allow standard compliant syntax for insert on partitioned tables
Ashutosh Chauhan created HIVE-18792: --- Summary: Allow standard compliant syntax for insert on partitioned tables Key: HIVE-18792 URL: https://issues.apache.org/jira/browse/HIVE-18792 Project: Hive Issue Type: Improvement Components: SQL Reporter: Ashutosh Chauhan Following works: {code} create table t1 (a int, b int, c int); create table t2 (a int, b int, c int) partitioned by (d int); insert into t1 values (1,2,3); insert into t1 (c, b, a) values (1,2,3); insert into t1 (a,b) values (1,2); {code} For partitioned tables it should work similarly but doesn't. All of following fails: {code} insert into t2 values (1,2,3,4); insert into t2 (a, b, c, d) values (1,2,3,4); insert into t2 (c,d) values (1,2); insert into t2 (a,b) values (1,2); {code} All of above should work. Also note following works: {code} insert into t2 partition(d) values (1,2,3,4); insert into t2 partition(d=4) values (1,2,3); {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-18717) Avoid transitive dependency on jetty 6.x
Ashutosh Chauhan created HIVE-18717: --- Summary: Avoid transitive dependency on jetty 6.x Key: HIVE-18717 URL: https://issues.apache.org/jira/browse/HIVE-18717 Project: Hive Issue Type: Bug Components: Build Infrastructure Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Although Hive is using jetty 9.3, transitive dependencies bring in 6.2.x which should be avoided. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-18711) Add percentile_cont and percentile_disc udaf
Ashutosh Chauhan created HIVE-18711: --- Summary: Add percentile_cont and percentile_disc udaf Key: HIVE-18711 URL: https://issues.apache.org/jira/browse/HIVE-18711 Project: Hive Issue Type: Bug Components: UDF Reporter: Ashutosh Chauhan Most common way to implement this is via ordered aggregate which allows users to specify sort specification with group by clause. Some implementations also allow to use these with window functions. Since Hive doesn't have concept of ordered aggregates yet, one possibility is to support these only for window functions where sort specification is also taken from window clause. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-18660) PCR doesn't distinguish between partition and virtual columns
Ashutosh Chauhan created HIVE-18660: --- Summary: PCR doesn't distinguish between partition and virtual columns Key: HIVE-18660 URL: https://issues.apache.org/jira/browse/HIVE-18660 Project: Hive Issue Type: Bug Components: Logical Optimizer Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan As a result transforms a filter {{INPUT__FILE__NAME is not null;}} to \{{false}} causing wrong results. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-18611) Avoid memory allocation of aggregation buffer during stats computation
Ashutosh Chauhan created HIVE-18611: --- Summary: Avoid memory allocation of aggregation buffer during stats computation Key: HIVE-18611 URL: https://issues.apache.org/jira/browse/HIVE-18611 Project: Hive Issue Type: Bug Components: Physical Optimizer, Statistics Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Bloom filter aggregation buffer may result in allocation of upto ~594MB array which is unnecessary. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-18585) Return type for udfs should be determined using Hive inference rules instead of Calcite
Ashutosh Chauhan created HIVE-18585: --- Summary: Return type for udfs should be determined using Hive inference rules instead of Calcite Key: HIVE-18585 URL: https://issues.apache.org/jira/browse/HIVE-18585 Project: Hive Issue Type: Bug Components: Query Planning Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan e.g., Calcite considers date and varchar incompatible types in case system, while Hive doesn't. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-18434) Type is not determined correctly for comparison between decimal column and string constant
Ashutosh Chauhan created HIVE-18434: --- Summary: Type is not determined correctly for comparison between decimal column and string constant Key: HIVE-18434 URL: https://issues.apache.org/jira/browse/HIVE-18434 Project: Hive Issue Type: Bug Components: Types Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-18251) Loosen restriction for some checks
Ashutosh Chauhan created HIVE-18251: --- Summary: Loosen restriction for some checks Key: HIVE-18251 URL: https://issues.apache.org/jira/browse/HIVE-18251 Project: Hive Issue Type: Improvement Components: Query Planning Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-18250) CBO gets turned off with duplicates in RR error
Ashutosh Chauhan created HIVE-18250: --- Summary: CBO gets turned off with duplicates in RR error Key: HIVE-18250 URL: https://issues.apache.org/jira/browse/HIVE-18250 Project: Hive Issue Type: Bug Components: CBO, Query Planning Affects Versions: 2.3.0, 2.2.0, 2.1.0, 2.0.0 Reporter: Ashutosh Chauhan {code} create table t1 (a int); explain select t1.a as a1, min(t1.a) as a from t1 group by t1.a; {code} CBO gets turned off with: {code} WARN [2e80e34e-dc46-49cf-88bf-2c24c0262d41 main] parse.RowResolver: Found duplicate column alias in RR: null.a => {null, a1, _col0: int} adding null.a => {null, null, _col1: int} 2017-12-07T15:27:47,651 ERROR [2e80e34e-dc46-49cf-88bf-2c24c0262d41 main] parse.CalcitePlanner: CBO failed, skipping CBO. org.apache.hadoop.hive.ql.optimizer.calcite.CalciteSemanticException: Cannot add column to RR: null.a => _col1: int due to duplication, see previous warnings at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genSelectLogicalPlan(CalcitePlanner.java:3985) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genLogicalPlan(CalcitePlanner.java:4313) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1392) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1322) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] {code} After that non-CBO path completes the query. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-18109) Don't reserve pool and default as keywords
Ashutosh Chauhan created HIVE-18109: --- Summary: Don't reserve pool and default as keywords Key: HIVE-18109 URL: https://issues.apache.org/jira/browse/HIVE-18109 Project: Hive Issue Type: Bug Affects Versions: 3.0.0 Reporter: Ashutosh Chauhan Assignee: Sergey Shelukhin HIVE-17902 broke this -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-18100) Some tests time out
Ashutosh Chauhan created HIVE-18100: --- Summary: Some tests time out Key: HIVE-18100 URL: https://issues.apache.org/jira/browse/HIVE-18100 Project: Hive Issue Type: Test Components: Test Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Some tests had 100s of queries in a single query which times out resulting in Hive QA failures. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-18089) Update golden files for few tests
Ashutosh Chauhan created HIVE-18089: --- Summary: Update golden files for few tests Key: HIVE-18089 URL: https://issues.apache.org/jira/browse/HIVE-18089 Project: Hive Issue Type: Test Components: Test Affects Versions: 3.0.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-18067) Remove extraneous golden files
Ashutosh Chauhan created HIVE-18067: --- Summary: Remove extraneous golden files Key: HIVE-18067 URL: https://issues.apache.org/jira/browse/HIVE-18067 Project: Hive Issue Type: Bug Components: Tests Affects Versions: 3.0.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan TestDanglingQouts makes sure that there are no unneeded files in repo. This is currently failing. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-18022) Remove DataNucleus repo
Ashutosh Chauhan created HIVE-18022: --- Summary: Remove DataNucleus repo Key: HIVE-18022 URL: https://issues.apache.org/jira/browse/HIVE-18022 Project: Hive Issue Type: Task Components: Build Infrastructure Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan DN jars are already present in other repos (maven central, apache) No need to have this as a repo -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-18018) Add in maven build
Ashutosh Chauhan created HIVE-18018: --- Summary: Add in maven build Key: HIVE-18018 URL: https://issues.apache.org/jira/browse/HIVE-18018 Project: Hive Issue Type: Bug Components: Build Infrastructure Affects Versions: 3.0.0 Reporter: Ashutosh Chauhan As discussed on HIVE-18007 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-18010) Update hbase version
Ashutosh Chauhan created HIVE-18010: --- Summary: Update hbase version Key: HIVE-18010 URL: https://issues.apache.org/jira/browse/HIVE-18010 Project: Hive Issue Type: Improvement Components: HBase Handler Affects Versions: 3.0.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-18007) Address maven warnings
Ashutosh Chauhan created HIVE-18007: --- Summary: Address maven warnings Key: HIVE-18007 URL: https://issues.apache.org/jira/browse/HIVE-18007 Project: Hive Issue Type: Bug Components: Build Infrastructure Affects Versions: 3.0.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan {code} [WARNING] Some problems were encountered while building the effective model for org.apache.hive:hive-metastore:jar:3.0.0-SNAPSHOT [WARNING] 'build.plugins.plugin.(groupId:artifactId)' must be unique but found duplicate declaration of plugin org.apache.maven.plugins:maven-jar-plugin @ line 299, column 15 [WARNING] Some problems were encountered while building the effective model for org.apache.hive:hive-standalone-metastore:jar:3.0.0-SNAPSHOT [WARNING] 'build.plugins.plugin.version' for org.antlr:antlr3-maven-plugin is missing. @ line 538, column 15 [WARNING] It is highly recommended to fix these problems because they threaten the stability of your build. [WARNING] For this reason, future Maven versions might no longer support building such malformed projects. {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17857) Upgrade to orc 1.4
Ashutosh Chauhan created HIVE-17857: --- Summary: Upgrade to orc 1.4 Key: HIVE-17857 URL: https://issues.apache.org/jira/browse/HIVE-17857 Project: Hive Issue Type: Task Components: ORC Reporter: Ashutosh Chauhan -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17847) Exclude net.hydromatic:aggdesigner-algorithm jar as compile and runtime dependency
Ashutosh Chauhan created HIVE-17847: --- Summary: Exclude net.hydromatic:aggdesigner-algorithm jar as compile and runtime dependency Key: HIVE-17847 URL: https://issues.apache.org/jira/browse/HIVE-17847 Project: Hive Issue Type: Improvement Components: CBO Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Hive doesn't use this. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17836) Persisting nulls in bit vector field fails for postgres backed metastore
Ashutosh Chauhan created HIVE-17836: --- Summary: Persisting nulls in bit vector field fails for postgres backed metastore Key: HIVE-17836 URL: https://issues.apache.org/jira/browse/HIVE-17836 Project: Hive Issue Type: Bug Components: Metastore Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17807) Execute maven commands in batch mode for ptests
Ashutosh Chauhan created HIVE-17807: --- Summary: Execute maven commands in batch mode for ptests Key: HIVE-17807 URL: https://issues.apache.org/jira/browse/HIVE-17807 Project: Hive Issue Type: Bug Reporter: Vijay Kumar Assignee: Vijay Kumar No need to run in interactive mode in CI environment. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17801) OpenCSVserde should store schema in metastore
Ashutosh Chauhan created HIVE-17801: --- Summary: OpenCSVserde should store schema in metastore Key: HIVE-17801 URL: https://issues.apache.org/jira/browse/HIVE-17801 Project: Hive Issue Type: Bug Components: Metastore, Serializers/Deserializers Reporter: Ashutosh Chauhan Just need to add opencsv serde in config. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17795) Add distribution management tag in pom
Ashutosh Chauhan created HIVE-17795: --- Summary: Add distribution management tag in pom Key: HIVE-17795 URL: https://issues.apache.org/jira/browse/HIVE-17795 Project: Hive Issue Type: Bug Reporter: Raja Aluri Assignee: Raja Aluri -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17779) Add itests profile
Ashutosh Chauhan created HIVE-17779: --- Summary: Add itests profile Key: HIVE-17779 URL: https://issues.apache.org/jira/browse/HIVE-17779 Project: Hive Issue Type: Bug Components: Build Infrastructure Reporter: Vijay Kumar Assignee: Vijay Kumar -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17777) Add maven coordinates in itests/pom.xml
Ashutosh Chauhan created HIVE-1: --- Summary: Add maven coordinates in itests/pom.xml Key: HIVE-1 URL: https://issues.apache.org/jira/browse/HIVE-1 Project: Hive Issue Type: Bug Components: Build Infrastructure, Testing Infrastructure Reporter: Vijay Kumar Assignee: Vijay Kumar -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17720) Bitvectors are not shown in describe statement on beeline
Ashutosh Chauhan created HIVE-17720: --- Summary: Bitvectors are not shown in describe statement on beeline Key: HIVE-17720 URL: https://issues.apache.org/jira/browse/HIVE-17720 Project: Hive Issue Type: Bug Components: Beeline, Diagnosability Affects Versions: 3.0.0 Reporter: Aswathy Chellammal Sreekumar Assignee: Ashutosh Chauhan Describe statement takes different codepath for HS2 where bit vectors weren't displayed. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17670) Remove auth v1
Ashutosh Chauhan created HIVE-17670: --- Summary: Remove auth v1 Key: HIVE-17670 URL: https://issues.apache.org/jira/browse/HIVE-17670 Project: Hive Issue Type: Task Components: Authorization, Security Reporter: Ashutosh Chauhan Never got adoption and is superseded by sql std auth -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17604) Add druid properties to conf white list
Ashutosh Chauhan created HIVE-17604: --- Summary: Add druid properties to conf white list Key: HIVE-17604 URL: https://issues.apache.org/jira/browse/HIVE-17604 Project: Hive Issue Type: Improvement Components: Configuration, Druid integration Affects Versions: 2.2.0, 2.3.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Currently throws: Error: Error while processing statement: Cannot modify hive.druid.select.distribute at runtime. It is not in list of params that are allowed to be modified at runtime (state=42000,code=1) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17521) Improve defaults for few runtime configs
Ashutosh Chauhan created HIVE-17521: --- Summary: Improve defaults for few runtime configs Key: HIVE-17521 URL: https://issues.apache.org/jira/browse/HIVE-17521 Project: Hive Issue Type: Task Components: Configuration Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Priority: Minor -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17511) Error while populating orc cache in llap
Ashutosh Chauhan created HIVE-17511: --- Summary: Error while populating orc cache in llap Key: HIVE-17511 URL: https://issues.apache.org/jira/browse/HIVE-17511 Project: Hive Issue Type: Bug Components: ORC Reporter: Ashutosh Chauhan Observed that while querying an error is thrown while loading cache in llap daemons -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17391) Compaction fails if there is an empty value in tblproperties
Ashutosh Chauhan created HIVE-17391: --- Summary: Compaction fails if there is an empty value in tblproperties Key: HIVE-17391 URL: https://issues.apache.org/jira/browse/HIVE-17391 Project: Hive Issue Type: Bug Components: Metastore, Transactions Affects Versions: 2.2.0, 2.3.0 Reporter: Ashutosh Chauhan create table t1 (a int) tblproperties ('serialization.null.format'=''); alter table t1 compact 'major'; fails -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17285) Fixes for bit vector retrievals and merging
Ashutosh Chauhan created HIVE-17285: --- Summary: Fixes for bit vector retrievals and merging Key: HIVE-17285 URL: https://issues.apache.org/jira/browse/HIVE-17285 Project: Hive Issue Type: Bug Reporter: Ashutosh Chauhan -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17190) Don't store bitvectors for unpartitioned table
Ashutosh Chauhan created HIVE-17190: --- Summary: Don't store bitvectors for unpartitioned table Key: HIVE-17190 URL: https://issues.apache.org/jira/browse/HIVE-17190 Project: Hive Issue Type: Test Components: Metastore, Statistics Affects Versions: 3.0.0 Reporter: Ashutosh Chauhan Since current ones can't be intersected, there is no advantage of storing them for unpartitioned tables. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17185) TestHiveMetaStoreStatsMerge.testStatsMerge is failing
Ashutosh Chauhan created HIVE-17185: --- Summary: TestHiveMetaStoreStatsMerge.testStatsMerge is failing Key: HIVE-17185 URL: https://issues.apache.org/jira/browse/HIVE-17185 Project: Hive Issue Type: Test Components: Metastore, Test Affects Versions: 3.0.0 Reporter: Ashutosh Chauhan Likely because of HIVE-16997 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-16924) Support distinct in presence Gby
Ashutosh Chauhan created HIVE-16924: --- Summary: Support distinct in presence Gby Key: HIVE-16924 URL: https://issues.apache.org/jira/browse/HIVE-16924 Project: Hive Issue Type: New Feature Components: Query Planning Reporter: Carter Shanklin create table e011_01 (c1 int, c2 smallint); insert into e011_01 values (1, 1), (2, 2); These queries should work: select distinct c1, count(*) from e011_01 group by c1; select distinct c1, avg(c2) from e011_01 group by c1; Currently, you get : FAILED: SemanticException 1:52 SELECT DISTINCT and GROUP BY can not be in the same query. Error encountered near token 'c1' -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-16909) Improve ndv estimates when ndvs are aggregated across partitions
Ashutosh Chauhan created HIVE-16909: --- Summary: Improve ndv estimates when ndvs are aggregated across partitions Key: HIVE-16909 URL: https://issues.apache.org/jira/browse/HIVE-16909 Project: Hive Issue Type: Improvement Components: Metastore, Statistics Reporter: Ashutosh Chauhan Tasks involved: 1. Switch over from FM-sketch to HLL bit vectors to compute ndvs. 2. Store these bit vectors in RDBMS metastore. This code already exists for HBase metastore. 3. Combine bit vectors requested for partition list to get better ndv estimate. This can be done initially only for CachedStore to avoid implementation complexity. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-16536) Various improvements in TestPerfCliDriver
Ashutosh Chauhan created HIVE-16536: --- Summary: Various improvements in TestPerfCliDriver Key: HIVE-16536 URL: https://issues.apache.org/jira/browse/HIVE-16536 Project: Hive Issue Type: Improvement Components: Testing Infrastructure Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Goal is to reduce the size of stats file used to import stats in metastore. This will help to run this tests on partitioned tables. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HIVE-16512) Join fails on operator return path on CBO
Ashutosh Chauhan created HIVE-16512: --- Summary: Join fails on operator return path on CBO Key: HIVE-16512 URL: https://issues.apache.org/jira/browse/HIVE-16512 Project: Hive Issue Type: Sub-task Components: CBO, Query Planning Reporter: Ashutosh Chauhan -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HIVE-16511) CBO looses inner casts on constants of complex type
Ashutosh Chauhan created HIVE-16511: --- Summary: CBO looses inner casts on constants of complex type Key: HIVE-16511 URL: https://issues.apache.org/jira/browse/HIVE-16511 Project: Hive Issue Type: Bug Components: CBO, Query Planning Reporter: Ashutosh Chauhan type for map <10, cast(null as int)> becomes map-- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HIVE-16494) udaf percentile_approx() may fail on CBO
Ashutosh Chauhan created HIVE-16494: --- Summary: udaf percentile_approx() may fail on CBO Key: HIVE-16494 URL: https://issues.apache.org/jira/browse/HIVE-16494 Project: Hive Issue Type: Bug Components: CBO, Logical Optimizer, UDF Reporter: Ashutosh Chauhan select percentile_approx(key, array(0.50, 0.70, 0.90, 0.95, 0.99)) from t; fails with error : The second argument must be a constant. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HIVE-16492) Create view doesn't work with Calcite Return path
Ashutosh Chauhan created HIVE-16492: --- Summary: Create view doesn't work with Calcite Return path Key: HIVE-16492 URL: https://issues.apache.org/jira/browse/HIVE-16492 Project: Hive Issue Type: Sub-task Components: CBO, Logical Optimizer Reporter: Ashutosh Chauhan Support for AST path was added in HIVE-15769 but not for return path. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HIVE-16491) CBO cant handle join involving complex types in on condition
Ashutosh Chauhan created HIVE-16491: --- Summary: CBO cant handle join involving complex types in on condition Key: HIVE-16491 URL: https://issues.apache.org/jira/browse/HIVE-16491 Project: Hive Issue Type: Bug Components: CBO, Logical Optimizer Affects Versions: 2.3.0 Reporter: Ashutosh Chauhan Chokes on query like: {code} select * from test2b join test2a on test2b.a = test2a.a[1]; {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HIVE-16474) Upgrade Druid version to 0.10
Ashutosh Chauhan created HIVE-16474: --- Summary: Upgrade Druid version to 0.10 Key: HIVE-16474 URL: https://issues.apache.org/jira/browse/HIVE-16474 Project: Hive Issue Type: Task Components: Druid integration Reporter: Ashutosh Chauhan Druid 0.10 is out. We shall upgrade to it to take advantage of improvements it brings. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HIVE-16434) Add support for parsing additional timestamp formats
Ashutosh Chauhan created HIVE-16434: --- Summary: Add support for parsing additional timestamp formats Key: HIVE-16434 URL: https://issues.apache.org/jira/browse/HIVE-16434 Project: Hive Issue Type: Bug Components: File Formats, Query Planning Reporter: Ashutosh Chauhan Will be useful to handle additional timestamp formats. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HIVE-16411) Revert HIVE-15199
Ashutosh Chauhan created HIVE-16411: --- Summary: Revert HIVE-15199 Key: HIVE-16411 URL: https://issues.apache.org/jira/browse/HIVE-16411 Project: Hive Issue Type: Task Reporter: Ashutosh Chauhan Priority: Blocker No longer required after HIVE-16402 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HIVE-16374) Remove serializer & deserializer interfaces
Ashutosh Chauhan created HIVE-16374: --- Summary: Remove serializer & deserializer interfaces Key: HIVE-16374 URL: https://issues.apache.org/jira/browse/HIVE-16374 Project: Hive Issue Type: Task Components: Serializers/Deserializers Reporter: Ashutosh Chauhan Priority: Blocker These interfaces are deprecated in favor of their Abstract class equivalents. We can remove them in 3.0 -- This message was sent by Atlassian JIRA (v6.3.15#6346)