[jira] [Commented] (HIVE-12650) Increase default value of hive.spark.client.server.connect.timeout to exceeds spark.yarn.am.waitTime
[ https://issues.apache.org/jira/browse/HIVE-12650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15148177#comment-15148177 ] JoneZhang commented on HIVE-12650: -- Hi all, I'm sorrry reply you so late. Yes hive.spark.client.server.connect.timeout and spark.yarn.am.waitTime does not have any relations. hive.spark.client.server.connect.timeout is the timeout between RPC server and client handshake.When no container is available, hive cient will exit after hive.spark.client.server.connect.timeout. spark.yarn.am.waitTime is the time the Spark AM waits for the SparkContext to be created after the AM has been launched. There are two types of error log 1.Client closed before SASL negotiation finished was happened on resubmitted. See https://issues.apache.org/jira/browse/HIVE-12649. 2.Connection refused: /hiveclientip:port was happend when am tries to connect back to Hive. Container: container_1448873753366_113453_01_01 on 10.247.169.134_8041 LogType: stderr LogLength: 3302 Log Contents: Please use CMSClassUnloadingEnabled in place of CMSPermGenSweepingEnabled in the future Please use CMSClassUnloadingEnabled in place of CMSPermGenSweepingEnabled in the future 15/12/09 02:11:48 INFO yarn.ApplicationMaster: Registered signal handlers for [TERM, HUP, INT] 15/12/09 02:11:48 INFO yarn.ApplicationMaster: ApplicationAttemptId: appattempt_1448873753366_113453_01 15/12/09 02:11:49 INFO spark.SecurityManager: Changing view acls to: mqq 15/12/09 02:11:49 INFO spark.SecurityManager: Changing modify acls to: mqq 15/12/09 02:11:49 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(mqq); users with modify permissions: Set(mqq) 15/12/09 02:11:49 INFO yarn.ApplicationMaster: Starting the user application in a separate Thread 15/12/09 02:11:49 INFO yarn.ApplicationMaster: Waiting for spark context initialization 15/12/09 02:11:49 INFO yarn.ApplicationMaster: Waiting for spark context initialization ... 15/12/09 02:11:49 INFO client.RemoteDriver: Connecting to: 10.179.12.140:58013 15/12/09 02:11:49 ERROR yarn.ApplicationMaster: User class threw exception: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused: /10.179.12.140:58013 java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused: /10.179.12.140:58013 at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:37) at org.apache.hive.spark.client.RemoteDriver.(RemoteDriver.java:156) at org.apache.hive.spark.client.RemoteDriver.main(RemoteDriver.java:556) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:483) Caused by: java.net.ConnectException: Connection refused: /10.179.12.140:58013 at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:208) at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:287) at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:528) at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116) at java.lang.Thread.run(Thread.java:745) 15/12/09 02:11:49 INFO yarn.ApplicationMaster: Final app status: FAILED, exitCode: 15, (reason: User class threw exception: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused: /10.179.12.140:58013) 15/12/09 02:11:59 ERROR yarn.ApplicationMaster: SparkContext did not initialize after waiting for 15 ms. Please check earlier log output for errors. Failing the application. 15/12/09 02:11:59 INFO util.Utils: Shutdown hook called > Increase default value of hive.spark.client.server.connect.timeout to exceeds > spark.yarn.am.waitTime > > > Key: HIVE-12650 > URL: https://issues.apache.org/jira/browse/HIVE-12650 > Project: Hive > Issue Type: Bug >Aff
[jira] [Commented] (HIVE-12649) Hive on Spark will resubmitted application when not enough resouces to launch yarn application master
[ https://issues.apache.org/jira/browse/HIVE-12649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15148175#comment-15148175 ] JoneZhang commented on HIVE-12649: -- The applications submited by estimate reducer number will fast fail when the reources is free after awhile. The error log is like below Container: container_1448873753366_121022_02_01 on 10.239.243.69_8041 === LogType: stderr LogLength: 4284 Log Contents: Please use CMSClassUnloadingEnabled in place of CMSPermGenSweepingEnabled in the future Please use CMSClassUnloadingEnabled in place of CMSPermGenSweepingEnabled in the future 15/12/09 16:29:31 INFO yarn.ApplicationMaster: Registered signal handlers for [TERM, HUP, INT] 15/12/09 16:29:32 INFO yarn.ApplicationMaster: ApplicationAttemptId: appattempt_1448873753366_121022_02 15/12/09 16:29:33 INFO spark.SecurityManager: Changing view acls to: mqq 15/12/09 16:29:33 INFO spark.SecurityManager: Changing modify acls to: mqq 15/12/09 16:29:33 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(mqq); users with modify permissions: Set(mqq) 15/12/09 16:29:33 INFO yarn.ApplicationMaster: Starting the user application in a separate Thread 15/12/09 16:29:33 INFO yarn.ApplicationMaster: Waiting for spark context initialization 15/12/09 16:29:33 INFO yarn.ApplicationMaster: Waiting for spark context initialization ... 15/12/09 16:29:33 INFO client.RemoteDriver: Connecting to: 10.179.12.140:38842 15/12/09 16:29:33 WARN rpc.Rpc: Invalid log level null, reverting to default. 15/12/09 16:29:33 ERROR yarn.ApplicationMaster: User class threw exception: java.util.concurrent.ExecutionException: javax.security.sasl.SaslException: Client closed before SASL negotiation finished. java.util.concurrent.ExecutionException: javax.security.sasl.SaslException: Client closed before SASL negotiation finished. at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:37) at org.apache.hive.spark.client.RemoteDriver.(RemoteDriver.java:156) at org.apache.hive.spark.client.RemoteDriver.main(RemoteDriver.java:556) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:483) Caused by: javax.security.sasl.SaslException: Client closed before SASL negotiation finished. at org.apache.hive.spark.client.rpc.Rpc$SaslClientHandler.dispose(Rpc.java:449) at org.apache.hive.spark.client.rpc.SaslHandler.channelInactive(SaslHandler.java:90) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:233) at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:219) at io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:75) at org.apache.hive.spark.client.rpc.KryoMessageCodec.channelInactive(KryoMessageCodec.java:127) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:233) at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:219) at io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:75) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:233) at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:219) at io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:769) at io.netty.channel.AbstractChannel$AbstractUnsafe$5.run(AbstractChannel.java:567) at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:380) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357) at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116) at java.lang.Thread.run(Thread.java:745) 15/12/09 16:29:33 INFO yarn.ApplicationMaster: Final app status: FAILED, exitCode: 15, (reason: User class threw exception: java.util.concurrent.ExecutionException: javax.security.sasl.SaslException: Client closed before SASL negotiation finished.) 15/12/09 16:29:43 ERROR yarn.ApplicationMaster: SparkContext did not initialize after waiting for 15 ms. Please check earlier log output for errors. Failing th
[jira] [Commented] (HIVE-13043) Reload function has no impact to function registry
[ https://issues.apache.org/jira/browse/HIVE-13043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15148167#comment-15148167 ] Hive QA commented on HIVE-13043: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12787957/HIVE-13043.3.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 21 failed/errored test(s), 9774 tests executed *Failed tests:* {noformat} TestMiniTezCliDriver-auto_join30.q-vector_data_types.q-filter_join_breaktask.q-and-12-more - did not produce a TEST-*.xml file TestMiniTezCliDriver-vector_interval_2.q-mapreduce1.q-vectorization_14.q-and-12-more - did not produce a TEST-*.xml file org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_coltype_literals org.apache.hadoop.hive.cli.TestMiniTezCliDriver.org.apache.hadoop.hive.cli.TestMiniTezCliDriver org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_bucket3 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_cte_mat_2 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_explainuser_1 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_load_dyn_part2 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_metadata_only_queries org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_optimize_nullscan org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_analyze org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge10 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_schema_evol_text_nonvec_fetchwork_table org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_script_env_var2 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_union8 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_distinct_2 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_orderby_5 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_outer_join3 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorization_7 org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import org.apache.hive.jdbc.TestSSL.testSSLVersion {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6996/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6996/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6996/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 21 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12787957 - PreCommit-HIVE-TRUNK-Build > Reload function has no impact to function registry > -- > > Key: HIVE-13043 > URL: https://issues.apache.org/jira/browse/HIVE-13043 > Project: Hive > Issue Type: Bug >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Attachments: HIVE-13043.1.patch, HIVE-13043.2.patch, > HIVE-13043.3.patch > > > With HIVE-2573, users should run "reload function" to refresh cached function > registry. However, "reload function" has no impact at all. We need to fix > this. Otherwise, HS2 needs to be restarted to see new global functions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12612) beeline always exits with 0 status when reading query from standard input
[ https://issues.apache.org/jira/browse/HIVE-12612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15148103#comment-15148103 ] Ruslan Dautkhanov commented on HIVE-12612: -- I could not reproduce your test case. {code:shell} $ echo "show databases;" | bl -f /dev/stdin scan complete in 2ms Connecting to jdbc:hive2://xx.yy.com:1/default;principal=hive/_HOST@HADOOP Connected to: Apache Hive (version 1.1.0-cdh5.5.1) Driver: Hive JDBC (version 1.1.0-cdh5.5.0) Transaction isolation: TRANSACTION_REPEATABLE_READ /dev/stdin (No such device or address) {code} > beeline always exits with 0 status when reading query from standard input > - > > Key: HIVE-12612 > URL: https://issues.apache.org/jira/browse/HIVE-12612 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 1.1.0 > Environment: CDH5.5.0 >Reporter: Paulo Sequeira >Priority: Minor > > Similar to what was reported on HIVE-6978, but now it only happens when the > query is read from the standard input. For example, the following fails as > expected: > {code} > bash$ if beeline -u "jdbc:hive2://..." -e "boo;" ; then echo "Ok?!" ; else > echo "Failed!" ; fi > Connecting to jdbc:hive2://... > Connected to: Apache Hive (version 1.1.0-cdh5.5.0) > Driver: Hive JDBC (version 1.1.0-cdh5.5.0) > Transaction isolation: TRANSACTION_REPEATABLE_READ > Error: Error while compiling statement: FAILED: ParseException line 1:0 > cannot recognize input near 'boo' '' '' (state=42000,code=4) > Closing: 0: jdbc:hive2://... > Failed! > {code} > But the following does not: > {code} > bash$ if echo "boo;"|beeline -u "jdbc:hive2://..." ; then echo "Ok?!" ; else > echo "Failed!" ; fi > Connecting to jdbc:hive2://... > Connected to: Apache Hive (version 1.1.0-cdh5.5.0) > Driver: Hive JDBC (version 1.1.0-cdh5.5.0) > Transaction isolation: TRANSACTION_REPEATABLE_READ > Beeline version 1.1.0-cdh5.5.0 by Apache Hive > 0: jdbc:hive2://...:8> Error: Error while compiling statement: FAILED: > ParseException line 1:0 cannot recognize input near 'boo' '' '' > (state=42000,code=4) > 0: jdbc:hive2://...:8> Closing: 0: jdbc:hive2://... > Ok?! > {code} > This was misleading our batch scripts to always believe that the execution of > the queries succeded, when sometimes that was not the case. > h2. Workaround > We found we can work around the issue by always using the -e or the -f > parameters, and even reading the standard input through the /dev/stdin device > (this was useful because a lot of the scripts fed the queries from here > documents), like this: > {code:title=some-script.sh} > #!/bin/sh > set -o nounset -o errexit -o pipefail > # As beeline is failing to report an error status if reading the query > # to be executed from STDIN, check whether no -f or -e option is used > # and, in that case, pretend it has to read the query from a regular > # file using -f to read from /dev/stdin > function beeline_workaround_exit_status () { > for arg in "$@" > do if [ "$arg" = "-f" -o "$arg" = "-e" ] >then beeline -u "..." "$@" > return >fi > done > beeline -u "..." "$@" -f /dev/stdin > } > beeline_workaround_exit_status < boo; > EOF > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12543) Disable Hive ConstantPropagate optimizer when CBO has optimized the plan
[ https://issues.apache.org/jira/browse/HIVE-12543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15148061#comment-15148061 ] Hive QA commented on HIVE-12543: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12787968/HIVE-12543.07.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 9803 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_coltype_literals org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import org.apache.hive.jdbc.TestSSL.testSSLVersion {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6995/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6995/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6995/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 3 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12787968 - PreCommit-HIVE-TRUNK-Build > Disable Hive ConstantPropagate optimizer when CBO has optimized the plan > > > Key: HIVE-12543 > URL: https://issues.apache.org/jira/browse/HIVE-12543 > Project: Hive > Issue Type: Sub-task > Components: CBO, Logical Optimizer >Affects Versions: 2.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-12543.01.patch, HIVE-12543.02.patch, > HIVE-12543.03.patch, HIVE-12543.04.patch, HIVE-12543.05.patch, > HIVE-12543.06.patch, HIVE-12543.07.patch, HIVE-12543.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12064) prevent transactional=false
[ https://issues.apache.org/jira/browse/HIVE-12064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15148060#comment-15148060 ] Wei Zheng commented on HIVE-12064: -- Test failures are not related. [~alangates] Can you take a look? Thanks. > prevent transactional=false > --- > > Key: HIVE-12064 > URL: https://issues.apache.org/jira/browse/HIVE-12064 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Wei Zheng >Priority: Critical > Attachments: HIVE-12064.2.patch, HIVE-12064.3.patch, > HIVE-12064.4.patch, HIVE-12064.patch > > > currently a tblproperty transactional=true must be set to make a table behave > in ACID compliant way. > This is misleading in that it seems like changing it to transactional=false > makes the table non-acid but on disk layout of acid table is different than > plain tables. So changing this property may cause wrong data to be returned. > Should prevent transactional=false. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
issues@hive.apache.org
[ https://issues.apache.org/jira/browse/HIVE-8339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zengzhaozheng reassigned HIVE-8339: --- Assignee: zengzhaozheng (was: Valera Chevtaev) > Job status not found after 100% succeded map&reduce > --- > > Key: HIVE-8339 > URL: https://issues.apache.org/jira/browse/HIVE-8339 > Project: Hive > Issue Type: Bug >Affects Versions: 0.13.1 > Environment: Hadoop 2.4.0, Hive 0.13.1. > Amazon EMR cluster of 9 i2.4xlarge nodes. > 800+GB of data in HDFS. >Reporter: Valera Chevtaev >Assignee: zengzhaozheng > > According to the logs it seems that the jobs 100% succeed for both map and > reduce but then wasn't able to get the status of the job from job history > server. > Hive logs: > 2014-10-03 07:57:26,593 INFO [main]: exec.Task > (SessionState.java:printInfo(536)) - 2014-10-03 07:57:26,593 Stage-1 map = > 100%, reduce = 99%, Cumulative CPU 872541.02 sec > 2014-10-03 07:57:47,447 INFO [main]: exec.Task > (SessionState.java:printInfo(536)) - 2014-10-03 07:57:47,446 Stage-1 map = > 100%, reduce = 100%, Cumulative CPU 872566.55 sec > 2014-10-03 07:57:48,710 INFO [main]: mapred.ClientServiceDelegate > (ClientServiceDelegate.java:getProxy(273)) - Application state is completed. > FinalApplicationStatus=SUCCEEDED. Redirecting to job history server > 2014-10-03 07:57:48,716 ERROR [main]: exec.Task > (SessionState.java:printError(545)) - Ended Job = job_1412263771568_0002 with > exception 'java.io.IOException(Could not find status of > job:job_1412263771568_0002)' > java.io.IOException: Could not find status of job:job_1412263771568_0002 >at > org.apache.hadoop.hive.ql.exec.mr.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:294) >at > org.apache.hadoop.hive.ql.exec.mr.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:547) >at > org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:426) >at > org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:136) >at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153) >at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85) >at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1503) >at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1270) >at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1088) >at org.apache.hadoop.hive.ql.Driver.run(Driver.java:911) >at org.apache.hadoop.hive.ql.Driver.run(Driver.java:901) >at > org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:275) >at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:227) >at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:430) >at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:366) >at > org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:463) >at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:479) >at > org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:759) >at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:697) >at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:636) >at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >at java.lang.reflect.Method.invoke(Method.java:606) >at org.apache.hadoop.util.RunJar.main(RunJar.java:212) > 2014-10-03 07:57:48,763 ERROR [main]: ql.Driver > (SessionState.java:printError(545)) - FAILED: Execution Error, return code 1 > from org.apache.hadoop.hive.ql.exec.mr.MapRedTask -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12878) Support Vectorization for TEXTFILE and other formats
[ https://issues.apache.org/jira/browse/HIVE-12878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15148018#comment-15148018 ] Hive QA commented on HIVE-12878: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12787933/HIVE-12878.03.patch {color:green}SUCCESS:{color} +1 due to 23 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 1210 failed/errored test(s), 9469 tests executed *Failed tests:* {noformat} TestDateIntervalDayTime - did not produce a TEST-*.xml file TestSparkCliDriver-auto_join11.q-vector_groupby_3.q-smb_mapjoin_8.q-and-3-more - did not produce a TEST-*.xml file TestSparkCliDriver-auto_join9.q-bucketmapjoin10.q-skewjoinopt19.q-and-12-more - did not produce a TEST-*.xml file TestSparkCliDriver-auto_join_reordering_values.q-auto_sortmerge_join_7.q-multigroupby_singlemr.q-and-12-more - did not produce a TEST-*.xml file TestSparkCliDriver-avro_decimal_native.q-bucketmapjoin12.q-ppd_outer_join2.q-and-12-more - did not produce a TEST-*.xml file TestSparkCliDriver-bucketsortoptimize_insert_7.q-enforce_order.q-join36.q-and-12-more - did not produce a TEST-*.xml file TestSparkCliDriver-escape_distributeby1.q-union_remove_7.q-skewjoin_union_remove_2.q-and-12-more - did not produce a TEST-*.xml file TestSparkCliDriver-groupby2_noskew_multi_distinct.q-skewjoin_noskew.q-vector_data_types.q-and-12-more - did not produce a TEST-*.xml file TestSparkCliDriver-groupby4.q-timestamp_null.q-auto_join23.q-and-12-more - did not produce a TEST-*.xml file TestSparkCliDriver-groupby_complex_types.q-vectorization_10.q-join4.q-and-12-more - did not produce a TEST-*.xml file TestSparkCliDriver-groupby_grouping_id2.q-bucketmapjoin4.q-groupby7.q-and-12-more - did not produce a TEST-*.xml file TestSparkCliDriver-join_cond_pushdown_3.q-groupby7_noskew.q-auto_join13.q-and-12-more - did not produce a TEST-*.xml file TestSparkCliDriver-join_rc.q-insert1.q-vectorized_rcfile_columnar.q-and-12-more - did not produce a TEST-*.xml file TestSparkCliDriver-load_dyn_part12.q-nullgroup4_multi_distinct.q-union14.q-and-12-more - did not produce a TEST-*.xml file TestSparkCliDriver-load_dyn_part5.q-skewjoinopt8.q-groupby1_noskew.q-and-12-more - did not produce a TEST-*.xml file TestSparkCliDriver-ppd_gby_join.q-stats2.q-groupby_rollup1.q-and-12-more - did not produce a TEST-*.xml file TestSparkCliDriver-ppd_join4.q-join9.q-ppd_join3.q-and-12-more - did not produce a TEST-*.xml file TestSparkCliDriver-skewjoinopt15.q-bucketmapjoin3.q-auto_join10.q-and-12-more - did not produce a TEST-*.xml file TestSparkCliDriver-smb_mapjoin_15.q-auto_sortmerge_join_13.q-auto_join18_multi_distinct.q-and-12-more - did not produce a TEST-*.xml file TestSparkCliDriver-smb_mapjoin_4.q-auto_join19.q-mapreduce1.q-and-12-more - did not produce a TEST-*.xml file TestSparkCliDriver-stats13.q-groupby6_map.q-join_casesensitive.q-and-12-more - did not produce a TEST-*.xml file TestSparkCliDriver-tez_joins_explain.q-input17.q-union29.q-and-12-more - did not produce a TEST-*.xml file TestSparkCliDriver-timestamp_lazy.q-bucketsortoptimize_insert_4.q-date_udf.q-and-12-more - did not produce a TEST-*.xml file TestSparkCliDriver-vector_distinct_2.q-load_dyn_part2.q-udf_percentile.q-and-12-more - did not produce a TEST-*.xml file org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alias_casted_column org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_partition_coltype org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_deep_filters org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_filter org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_groupby org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_groupby2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_select org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_table org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ansi_sql_arithmetic org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join0 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join14 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join15 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join17 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join18 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join18_multi_distinct org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join19 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join20 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join24 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join26 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join27 org.apache.hadoop
[jira] [Updated] (HIVE-13063) Create UDFs for CHAR and REPLACE
[ https://issues.apache.org/jira/browse/HIVE-13063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alejandro Fernandez updated HIVE-13063: --- Attachment: HIVE-13063.patch > Create UDFs for CHAR and REPLACE > - > > Key: HIVE-13063 > URL: https://issues.apache.org/jira/browse/HIVE-13063 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 1.2.0 >Reporter: Alejandro Fernandez >Assignee: Alejandro Fernandez > Fix For: 2.1.0 > > Attachments: HIVE-13063.patch > > > Create UDFS for these functions. > CHAR: convert n where n : [0, 256) into the ascii equivalent as a varchar. If > n is less than 0 or greater than 255, return the empty string. If n is 0, > return null. > REPLACE: replace all substrings of 'str' that match 'search' with 'rep'. > Example. SELECT REPLACE('Hack and Hue', 'H', 'BL'); > Equals 'BLack and BLue'" -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9457) Fix obsolete parameter name in HiveConf description of hive.hashtable.initialCapacity
[ https://issues.apache.org/jira/browse/HIVE-9457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shannon Ladymon updated HIVE-9457: -- Attachment: HIVE-9457.patch Patch attached. > Fix obsolete parameter name in HiveConf description of > hive.hashtable.initialCapacity > - > > Key: HIVE-9457 > URL: https://issues.apache.org/jira/browse/HIVE-9457 > Project: Hive > Issue Type: Bug > Components: Documentation >Affects Versions: 0.14.0 >Reporter: Lefty Leverenz >Assignee: Shannon Ladymon >Priority: Minor > Attachments: HIVE-9457.patch > > > The description of *hive.hashtable.initialCapacity* in HiveConf.java refers > to a parameter that existed in an early patch for HIVE-7616 > ("hive.hashtable.stats.key.estimate.adjustment") but was renamed in later > patches. So change *hive.hashtable.stats.key.estimate.adjustment* to > *hive.hashtable.key.count.adjustment* in this parameter definition in > HiveConf.java: > {code} > HIVEHASHTABLETHRESHOLD("hive.hashtable.initialCapacity", 10, "Initial > capacity of " + > "mapjoin hashtable if statistics are absent, or if > hive.hashtable.stats.key.estimate.adjustment is set to 0"), > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9457) Fix obsolete parameter name in HiveConf description of hive.hashtable.initialCapacity
[ https://issues.apache.org/jira/browse/HIVE-9457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lefty Leverenz updated HIVE-9457: - Assignee: Shannon Ladymon > Fix obsolete parameter name in HiveConf description of > hive.hashtable.initialCapacity > - > > Key: HIVE-9457 > URL: https://issues.apache.org/jira/browse/HIVE-9457 > Project: Hive > Issue Type: Bug > Components: Documentation >Affects Versions: 0.14.0 >Reporter: Lefty Leverenz >Assignee: Shannon Ladymon >Priority: Minor > > The description of *hive.hashtable.initialCapacity* in HiveConf.java refers > to a parameter that existed in an early patch for HIVE-7616 > ("hive.hashtable.stats.key.estimate.adjustment") but was renamed in later > patches. So change *hive.hashtable.stats.key.estimate.adjustment* to > *hive.hashtable.key.count.adjustment* in this parameter definition in > HiveConf.java: > {code} > HIVEHASHTABLETHRESHOLD("hive.hashtable.initialCapacity", 10, "Initial > capacity of " + > "mapjoin hashtable if statistics are absent, or if > hive.hashtable.stats.key.estimate.adjustment is set to 0"), > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12981) ThriftCLIService uses incompatible getShortName() implementation
[ https://issues.apache.org/jira/browse/HIVE-12981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15147864#comment-15147864 ] Hive QA commented on HIVE-12981: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12787871/HIVE-12981.01.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 9803 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_mult_tables org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_coltype_literals org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import org.apache.hive.jdbc.TestSSL.testSSLVersion {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6993/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6993/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6993/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 4 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12787871 - PreCommit-HIVE-TRUNK-Build > ThriftCLIService uses incompatible getShortName() implementation > > > Key: HIVE-12981 > URL: https://issues.apache.org/jira/browse/HIVE-12981 > Project: Hive > Issue Type: Bug > Components: Authentication, Authorization, CLI, Security >Affects Versions: 1.2.1, 2.1.0 >Reporter: Bolke de Bruin >Assignee: Thejas M Nair >Priority: Critical > Labels: kerberos > Attachments: 0001-HIVE-12981-Use-KerberosName.patch, > HIVE-12981-branch-1.2.patch, HIVE-12981.01.patch, HIVE-12981.patch > > > ThriftCLIService has a local implementation getShortName() that assumes a > short name is always the part before "@" and "/". This is not always the case > as Kerberos Rules (from Hadoop's KerberosName) might actually transform a > name to something else. > Considering a pending change to getShortName() (#HADOOP-12751) and the normal > use of KerberosName in other parts of Hive it only seems logical to use the > standard implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13043) Reload function has no impact to function registry
[ https://issues.apache.org/jira/browse/HIVE-13043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15147856#comment-15147856 ] Jimmy Xiang commented on HIVE-13043: UPloaded patch 3 to fix TestOperation2Privilege.checkHiveOperationTypeMatch > Reload function has no impact to function registry > -- > > Key: HIVE-13043 > URL: https://issues.apache.org/jira/browse/HIVE-13043 > Project: Hive > Issue Type: Bug >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Attachments: HIVE-13043.1.patch, HIVE-13043.2.patch, > HIVE-13043.3.patch > > > With HIVE-2573, users should run "reload function" to refresh cached function > registry. However, "reload function" has no impact at all. We need to fix > this. Otherwise, HS2 needs to be restarted to see new global functions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13019) Optimizer COLLECT_LIST/COLLECT_SET
[ https://issues.apache.org/jira/browse/HIVE-13019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15147807#comment-15147807 ] Ruslan Dautkhanov commented on HIVE-13019: -- Thank you Gopal. Oracle has "RELY NOVALIDATE" option for constraints.. would it be easier for Hive to start with something like that for PK/FK constraints? So CBO has more information for optimizations. It does not have to actually check if that constraint is relationship is actually true; it can just "rely" on that constraint.. Is this already plan for Hive 2.0? Thanks again. > Optimizer COLLECT_LIST/COLLECT_SET > --- > > Key: HIVE-13019 > URL: https://issues.apache.org/jira/browse/HIVE-13019 > Project: Hive > Issue Type: Improvement > Components: CBO, Logical Optimizer >Reporter: Dustin Cote >Priority: Minor > > Currently when using a COLLECT_SET/COLLECT_LIST that involves data from a > single table, the aggregation is done after any JOIN operation that is > present in the query. For example: > {code} > insert into table nested_customers_orders > select c.*, collect_list(named_struct("oid", o.oid, "order_date": o.date...)) > from customers c inner join orders o on (c.cid = o.oid) > group by o.oid, o.date,... > {code} > If we can tell the optimizer to perform the COLLECT_LIST first (where > possible) we can see some performance gains in this pattern of query. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12244) Refactoring code for avoiding of comparison of Strings and do comparison on Path
[ https://issues.apache.org/jira/browse/HIVE-12244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15147806#comment-15147806 ] Hive QA commented on HIVE-12244: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12787865/HIVE-12244.7.patch {color:green}SUCCESS:{color} +1 due to 9 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 588 failed/errored test(s), 9698 tests executed *Failed tests:* {noformat} TestCliDriver-avro_decimal_native.q-alter_file_format.q-groupby3_map_skew.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-gby_star.q-udf_regexp_replace.q-load_dyn_part2.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-orc_createas1.q-nestedvirtual.q-union35.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-orc_merge6.q-input44.q-smb_mapjoin_4.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-part_inherit_tbl_props_with_star.q-truncate_table.q-list_bucket_query_oneskew_2.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-skewjoin_mapjoin11.q-udf_least.q-plan_json.q-and-12-more - did not produce a TEST-*.xml file TestMiniTezCliDriver-vectorization_10.q-schema_evol_orc_acidvec_mapwork_part.q-vector_non_string_partition.q-and-12-more - did not produce a TEST-*.xml file org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_acid_globallimit org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_acid_vectorization org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_acid_vectorization_partition org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_acid_vectorization_project org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_add_part_exist org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_concatenate_indexed_table org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_index org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_merge_2_orc org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_merge_orc org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_partition_change_col org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_rename_partition org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_table_invalidate_column_stats org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_filter org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_groupby org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_limit org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_part org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_select org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_table org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_union org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_9 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_delete org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_delete_own_table org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_index org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_insert org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_parts org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_show_grant org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_update org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_update_own_table org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join10 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join11 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join12 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join13 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join14 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join15 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join16 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join17 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join19 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join20 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join21 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join22 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join23 org.apache.hadoop.hive.cli.Tes
[jira] [Updated] (HIVE-12673) Orcfiledump throws NPE when no files are available
[ https://issues.apache.org/jira/browse/HIVE-12673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-12673: - Attachment: HIVE-12673.3.patch Not sure why this patch was not picked by Precommit QA. Retrying again. > Orcfiledump throws NPE when no files are available > -- > > Key: HIVE-12673 > URL: https://issues.apache.org/jira/browse/HIVE-12673 > Project: Hive > Issue Type: Bug >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan > Attachments: HIVE-12673.1.patch, HIVE-12673.2.patch, > HIVE-12673.2.patch, HIVE-12673.3.patch, HIVE-12673.3.rebased.patch > > > {noformat} > Exception in thread "main" java.lang.NullPointerException > at org.codehaus.jettison.json.JSONTokener.more(JSONTokener.java:106) > at org.codehaus.jettison.json.JSONTokener.next(JSONTokener.java:116) > at > org.codehaus.jettison.json.JSONTokener.nextClean(JSONTokener.java:170) > at org.codehaus.jettison.json.JSONObject.(JSONObject.java:185) > at org.codehaus.jettison.json.JSONObject.(JSONObject.java:293) > at > org.apache.hadoop.hive.ql.io.orc.JsonFileDump.printJsonMetaData(JsonFileDump.java:197) > at org.apache.hadoop.hive.ql.io.orc.FileDump.main(FileDump.java:107) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:497) > at org.apache.hadoop.util.RunJar.run(RunJar.java:221) > at org.apache.hadoop.util.RunJar.main(RunJar.java:136) > {noformat} > hive --orcfiledump -j -p /tmp/orc/inventory/inv_date_sk=2452654 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12981) ThriftCLIService uses incompatible getShortName() implementation
[ https://issues.apache.org/jira/browse/HIVE-12981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15147774#comment-15147774 ] Bolke de Bruin commented on HIVE-12981: --- [~sershe] So is there anything left to do for this patch? testCliDriver_ivyDownload is I guess also not due to this patch. > ThriftCLIService uses incompatible getShortName() implementation > > > Key: HIVE-12981 > URL: https://issues.apache.org/jira/browse/HIVE-12981 > Project: Hive > Issue Type: Bug > Components: Authentication, Authorization, CLI, Security >Affects Versions: 1.2.1, 2.1.0 >Reporter: Bolke de Bruin >Assignee: Thejas M Nair >Priority: Critical > Labels: kerberos > Attachments: 0001-HIVE-12981-Use-KerberosName.patch, > HIVE-12981-branch-1.2.patch, HIVE-12981.01.patch, HIVE-12981.patch > > > ThriftCLIService has a local implementation getShortName() that assumes a > short name is always the part before "@" and "/". This is not always the case > as Kerberos Rules (from Hadoop's KerberosName) might actually transform a > name to something else. > Considering a pending change to getShortName() (#HADOOP-12751) and the normal > use of KerberosName in other parts of Hive it only seems logical to use the > standard implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12994) Implement support for NULLS FIRST/NULLS LAST
[ https://issues.apache.org/jira/browse/HIVE-12994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-12994: --- Component/s: Metastore > Implement support for NULLS FIRST/NULLS LAST > > > Key: HIVE-12994 > URL: https://issues.apache.org/jira/browse/HIVE-12994 > Project: Hive > Issue Type: New Feature > Components: CBO, Metastore, Parser, Serializers/Deserializers >Affects Versions: 2.1.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-12994.01.patch, HIVE-12994.02.patch, > HIVE-12994.03.patch, HIVE-12994.04.patch, HIVE-12994.05.patch, > HIVE-12994.06.patch, HIVE-12994.06.patch, HIVE-12994.07.patch, > HIVE-12994.patch > > > From SQL:2003, the NULLS FIRST and NULLS LAST options can be used to > determine whether nulls appear before or after non-null data values when the > ORDER BY clause is used. > SQL standard does not specify the behavior by default. Currently in Hive, > null values sort as if lower than any non-null value; that is, NULLS FIRST is > the default for ASC order, and NULLS LAST for DESC order. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13062) Hive metastore test failing
[ https://issues.apache.org/jira/browse/HIVE-13062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15147743#comment-15147743 ] Jesus Camacho Rodriguez commented on HIVE-13062: [~sershe], could you review it? HIVE-12994 was triggering PreCommit-HIVE-METASTORE-Test and I was hitting this error. Thanks > Hive metastore test failing > --- > > Key: HIVE-13062 > URL: https://issues.apache.org/jira/browse/HIVE-13062 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 2.0.0, 2.1.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-13062.patch > > > The testutils/metastore/metastore-upgrade-test.sh script takes the list of > scripts to execute for the DB upgrade from the folder itself > ({{metastore/scripts/upgrade/DB_NAME}}). Currently it fails as it executes > upgrade-1.2.0-to-1.3.0 followed by upgrade-1.2.0-to-2.0.0. > Instead, it should use the content of the {{upgrade.order.DB_NAME}} file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13062) Hive metastore test failing
[ https://issues.apache.org/jira/browse/HIVE-13062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-13062: --- Description: The testutils/metastore/metastore-upgrade-test.sh script takes the list of scripts to execute for the DB upgrade from the folder itself ({{metastore/scripts/upgrade/DB_NAME}}). Currently it fails as it executes upgrade-1.2.0-to-1.3.0 followed by upgrade-1.2.0-to-2.0.0. Instead, it should use the content of the {{upgrade.order.DB_NAME}} file. was: The testutils/metastore/metastore-upgrade-test.sh script takes the list of scripts to execute for the DB upgrade from the folder itself (metastore/scripts/upgrade/DB_NAME). Currently it fails as it executes upgrade-1.2.0-to-1.3.0 followed by upgrade-1.2.0-to-2.0.0. Instead, it should use the content of the upgrade.order.DB_NAME file. > Hive metastore test failing > --- > > Key: HIVE-13062 > URL: https://issues.apache.org/jira/browse/HIVE-13062 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 2.0.0, 2.1.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-13062.patch > > > The testutils/metastore/metastore-upgrade-test.sh script takes the list of > scripts to execute for the DB upgrade from the folder itself > ({{metastore/scripts/upgrade/DB_NAME}}). Currently it fails as it executes > upgrade-1.2.0-to-1.3.0 followed by upgrade-1.2.0-to-2.0.0. > Instead, it should use the content of the {{upgrade.order.DB_NAME}} file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12994) Implement support for NULLS FIRST/NULLS LAST
[ https://issues.apache.org/jira/browse/HIVE-12994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15147740#comment-15147740 ] Hive QA commented on HIVE-12994: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12787976/HIVE-12994.07.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:green}SUCCESS:{color} +1 due to 4 tests passed Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-METASTORE-Test/117/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-METASTORE-Test/117/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-METASTORE-Test-117/ Messages: {noformat} LXC derby found. LXC derby is not started. Starting container... Container started. Preparing derby container... Container prepared. Calling /hive/testutils/metastore/dbs/derby/prepare.sh ... Server prepared. Calling /hive/testutils/metastore/dbs/derby/execute.sh ... Tests executed. LXC mysql found. LXC mysql is not started. Starting container... Container started. Preparing mysql container... Container prepared. Calling /hive/testutils/metastore/dbs/mysql/prepare.sh ... Server prepared. Calling /hive/testutils/metastore/dbs/mysql/execute.sh ... Tests executed. LXC oracle found. LXC oracle is not started. Starting container... Container started. Preparing oracle container... Container prepared. Calling /hive/testutils/metastore/dbs/oracle/prepare.sh ... Server prepared. Calling /hive/testutils/metastore/dbs/oracle/execute.sh ... Tests executed. LXC postgres found. LXC postgres is not started. Starting container... Container started. Preparing postgres container... Container prepared. Calling /hive/testutils/metastore/dbs/postgres/prepare.sh ... Server prepared. Calling /hive/testutils/metastore/dbs/postgres/execute.sh ... Tests executed. {noformat} This message is automatically generated. ATTACHMENT ID: 12787976 - PreCommit-HIVE-METASTORE-Test > Implement support for NULLS FIRST/NULLS LAST > > > Key: HIVE-12994 > URL: https://issues.apache.org/jira/browse/HIVE-12994 > Project: Hive > Issue Type: New Feature > Components: CBO, Parser, Serializers/Deserializers >Affects Versions: 2.1.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-12994.01.patch, HIVE-12994.02.patch, > HIVE-12994.03.patch, HIVE-12994.04.patch, HIVE-12994.05.patch, > HIVE-12994.06.patch, HIVE-12994.06.patch, HIVE-12994.07.patch, > HIVE-12994.patch > > > From SQL:2003, the NULLS FIRST and NULLS LAST options can be used to > determine whether nulls appear before or after non-null data values when the > ORDER BY clause is used. > SQL standard does not specify the behavior by default. Currently in Hive, > null values sort as if lower than any non-null value; that is, NULLS FIRST is > the default for ASC order, and NULLS LAST for DESC order. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13061) Advance Hadoop Architecture (AHA) - Advance Hive (Umbrella) JIRA -- Enhanced Updates & others
[ https://issues.apache.org/jira/browse/HIVE-13061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15147734#comment-15147734 ] Dinesh S. Atreya commented on HIVE-13061: - Please review JIRA https://issues.apache.org/jira/browse/HDFS-9607 and provide your inputs/perspectives including additional requirements as that JIRA will serve as a basis for Advancing Hive as proposed in this JIRA. Kindly review ORC JIRA https://issues.apache.org/jira/browse/ORC-42 and provide your thoughts as well. > Advance Hadoop Architecture (AHA) - Advance Hive (Umbrella) JIRA -- Enhanced > Updates & others > - > > Key: HIVE-13061 > URL: https://issues.apache.org/jira/browse/HIVE-13061 > Project: Hive > Issue Type: New Feature >Reporter: Dinesh S. Atreya > > Link to Umbrella JIRA https://issues.apache.org/jira/browse/HADOOP-12620 > See > https://issues.apache.org/jira/browse/HADOOP-12620?focusedCommentId=15046300&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15046300 > for more details. > This JIRA is an umbrella (parent/master) JIRA for advancing Hive through > native Hive update capability given > https://issues.apache.org/jira/browse/HDFS-9607 and > https://issues.apache.org/jira/browse/ORC-42. > Provide seamless and streamlined enhancements to Hive capabilities that are > currently implemented through https://issues.apache.org/jira/browse/HIVE-5317 > and other related JIRAs. > Serve as a counterpart Umbrella JIRA to ORC JIRA > https://issues.apache.org/jira/browse/ORC-42, so that corresponding > capabilities can be supported through SQL interfaces in Hive. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13047) Disabling Web UI leads to NullPointerException
[ https://issues.apache.org/jira/browse/HIVE-13047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15147733#comment-15147733 ] Sergey Shelukhin commented on HIVE-13047: - Failures are unrelated > Disabling Web UI leads to NullPointerException > -- > > Key: HIVE-13047 > URL: https://issues.apache.org/jira/browse/HIVE-13047 > Project: Hive > Issue Type: Bug >Reporter: Mohit Sabharwal >Assignee: Mohit Sabharwal > Attachments: HIVE-13047.patch > > > Disabling the Web UI or it's historical query display feature can lead to > NullPointerException since {{historicSqlOperations}} is unintialized. > For ex: If hive.server2.webui.port is set to 0 > {code} > Caused by: java.lang.NullPointerException: null > at > org.apache.hive.service.cli.operation.OperationManager.removeOperation(OperationManager.java:212) > at > org.apache.hive.service.cli.operation.OperationManager.closeOperation(OperationManager.java:240) > at > org.apache.hive.service.cli.session.HiveSessionImpl.closeOperation(HiveSessionImpl.java:727) > at > org.apache.hive.service.cli.CLIService.closeOperation(CLIService.java:408) > at > org.apache.hive.service.cli.thrift.ThriftCLIService.CloseOperation(ThriftCLIService.java:664) > at > org.apache.hive.service.cli.thrift.TCLIService$Processor$CloseOperation.getResult(TCLIService.java:1513) > at > org.apache.hive.service.cli.thrift.TCLIService$Processor$CloseOperation.getResult(TCLIService.java:1498) > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) > at > org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56) > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12994) Implement support for NULLS FIRST/NULLS LAST
[ https://issues.apache.org/jira/browse/HIVE-12994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-12994: --- Attachment: (was: HIVE-12994.07.patch) > Implement support for NULLS FIRST/NULLS LAST > > > Key: HIVE-12994 > URL: https://issues.apache.org/jira/browse/HIVE-12994 > Project: Hive > Issue Type: New Feature > Components: CBO, Parser, Serializers/Deserializers >Affects Versions: 2.1.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-12994.01.patch, HIVE-12994.02.patch, > HIVE-12994.03.patch, HIVE-12994.04.patch, HIVE-12994.05.patch, > HIVE-12994.06.patch, HIVE-12994.06.patch, HIVE-12994.07.patch, > HIVE-12994.patch > > > From SQL:2003, the NULLS FIRST and NULLS LAST options can be used to > determine whether nulls appear before or after non-null data values when the > ORDER BY clause is used. > SQL standard does not specify the behavior by default. Currently in Hive, > null values sort as if lower than any non-null value; that is, NULLS FIRST is > the default for ASC order, and NULLS LAST for DESC order. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12994) Implement support for NULLS FIRST/NULLS LAST
[ https://issues.apache.org/jira/browse/HIVE-12994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-12994: --- Attachment: HIVE-12994.07.patch > Implement support for NULLS FIRST/NULLS LAST > > > Key: HIVE-12994 > URL: https://issues.apache.org/jira/browse/HIVE-12994 > Project: Hive > Issue Type: New Feature > Components: CBO, Parser, Serializers/Deserializers >Affects Versions: 2.1.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-12994.01.patch, HIVE-12994.02.patch, > HIVE-12994.03.patch, HIVE-12994.04.patch, HIVE-12994.05.patch, > HIVE-12994.06.patch, HIVE-12994.06.patch, HIVE-12994.07.patch, > HIVE-12994.patch > > > From SQL:2003, the NULLS FIRST and NULLS LAST options can be used to > determine whether nulls appear before or after non-null data values when the > ORDER BY clause is used. > SQL standard does not specify the behavior by default. Currently in Hive, > null values sort as if lower than any non-null value; that is, NULLS FIRST is > the default for ASC order, and NULLS LAST for DESC order. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13062) Hive metastore test failing
[ https://issues.apache.org/jira/browse/HIVE-13062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-13062: --- Attachment: HIVE-13062.patch > Hive metastore test failing > --- > > Key: HIVE-13062 > URL: https://issues.apache.org/jira/browse/HIVE-13062 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 2.0.0, 2.1.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-13062.patch > > > The testutils/metastore/metastore-upgrade-test.sh script takes the list of > scripts to execute for the DB upgrade from the folder itself > (metastore/scripts/upgrade/DB_NAME). Currently it fails as it executes > upgrade-1.2.0-to-1.3.0 followed by upgrade-1.2.0-to-2.0.0. > Instead, it should use the content of the upgrade.order.DB_NAME file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13061) Advance Hadoop Architecture (AHA) - Advance Hive (Umbrella) JIRA -- Enhanced Updates & others
[ https://issues.apache.org/jira/browse/HIVE-13061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15147721#comment-15147721 ] Dinesh S. Atreya commented on HIVE-13061: - Adding related JIRAS * HIVE-9675 * HIVE-6905 * HIVE-8244 > Advance Hadoop Architecture (AHA) - Advance Hive (Umbrella) JIRA -- Enhanced > Updates & others > - > > Key: HIVE-13061 > URL: https://issues.apache.org/jira/browse/HIVE-13061 > Project: Hive > Issue Type: New Feature >Reporter: Dinesh S. Atreya > > Link to Umbrella JIRA https://issues.apache.org/jira/browse/HADOOP-12620 > See > https://issues.apache.org/jira/browse/HADOOP-12620?focusedCommentId=15046300&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15046300 > for more details. > This JIRA is an umbrella (parent/master) JIRA for advancing Hive through > native Hive update capability given > https://issues.apache.org/jira/browse/HDFS-9607 and > https://issues.apache.org/jira/browse/ORC-42. > Provide seamless and streamlined enhancements to Hive capabilities that are > currently implemented through https://issues.apache.org/jira/browse/HIVE-5317 > and other related JIRAs. > Serve as a counterpart Umbrella JIRA to ORC JIRA > https://issues.apache.org/jira/browse/ORC-42, so that corresponding > capabilities can be supported through SQL interfaces in Hive. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13061) Advance Hadoop Architecture (AHA) - Advance Hive (Umbrella) JIRA -- Enhanced Updates & others
[ https://issues.apache.org/jira/browse/HIVE-13061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15147717#comment-15147717 ] Dinesh S. Atreya commented on HIVE-13061: - This JIRA is dependent on both https://issues.apache.org/jira/browse/HDFS-9607 and https://issues.apache.org/jira/browse/ORC-42 > Advance Hadoop Architecture (AHA) - Advance Hive (Umbrella) JIRA -- Enhanced > Updates & others > - > > Key: HIVE-13061 > URL: https://issues.apache.org/jira/browse/HIVE-13061 > Project: Hive > Issue Type: New Feature >Reporter: Dinesh S. Atreya > > Link to Umbrella JIRA https://issues.apache.org/jira/browse/HADOOP-12620 > See > https://issues.apache.org/jira/browse/HADOOP-12620?focusedCommentId=15046300&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15046300 > for more details. > This JIRA is an umbrella (parent/master) JIRA for advancing Hive through > native Hive update capability given > https://issues.apache.org/jira/browse/HDFS-9607 and > https://issues.apache.org/jira/browse/ORC-42. > Provide seamless and streamlined enhancements to Hive capabilities that are > currently implemented through https://issues.apache.org/jira/browse/HIVE-5317 > and other related JIRAs. > Serve as a counterpart Umbrella JIRA to ORC JIRA > https://issues.apache.org/jira/browse/ORC-42, so that corresponding > capabilities can be supported through SQL interfaces in Hive. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13061) Advance Hadoop Architecture (AHA) - Advance Hive (Umbrella) JIRA -- Enhanced Updates & others
[ https://issues.apache.org/jira/browse/HIVE-13061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dinesh S. Atreya updated HIVE-13061: Description: Link to Umbrella JIRA https://issues.apache.org/jira/browse/HADOOP-12620 See https://issues.apache.org/jira/browse/HADOOP-12620?focusedCommentId=15046300&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15046300 for more details. This JIRA is an umbrella (parent/master) JIRA for advancing Hive through native Hive update capability given https://issues.apache.org/jira/browse/HDFS-9607 and https://issues.apache.org/jira/browse/ORC-42. Provide seamless and streamlined enhancements to Hive capabilities that are currently implemented through https://issues.apache.org/jira/browse/HIVE-5317 and other related JIRAs. Serve as a counterpart Umbrella JIRA to ORC JIRA https://issues.apache.org/jira/browse/ORC-42, so that corresponding capabilities can be supported through SQL interfaces in Hive. was: Link to Umbrella JIRA https://issues.apache.org/jira/browse/HADOOP-12620 See https://issues.apache.org/jira/browse/HADOOP-12620?focusedCommentId=15046300&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15046300 for more details. This JIRA is an umbrella (parent/master) JIRA for advancing Hive through native Hive update capability given https://issues.apache.org/jira/browse/HDFS-9607 and https://issues.apache.org/jira/browse/ORC-42. Provide seamless and streamlined enhancements to Hive capabilities that are currently implemented through https://issues.apache.org/jira/browse/HIVE-5317 and other related JIRAs. Serve as a counterpart Umbrella JIRA to ORC JIRA https://issues.apache.org/jira/browse/ORC-42, so that corresponding capabilities can be supported through SQL interfaces in Hive. > Advance Hadoop Architecture (AHA) - Advance Hive (Umbrella) JIRA -- Enhanced > Updates & others > - > > Key: HIVE-13061 > URL: https://issues.apache.org/jira/browse/HIVE-13061 > Project: Hive > Issue Type: New Feature >Reporter: Dinesh S. Atreya > > Link to Umbrella JIRA https://issues.apache.org/jira/browse/HADOOP-12620 > See > https://issues.apache.org/jira/browse/HADOOP-12620?focusedCommentId=15046300&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15046300 > for more details. > This JIRA is an umbrella (parent/master) JIRA for advancing Hive through > native Hive update capability given > https://issues.apache.org/jira/browse/HDFS-9607 and > https://issues.apache.org/jira/browse/ORC-42. > Provide seamless and streamlined enhancements to Hive capabilities that are > currently implemented through https://issues.apache.org/jira/browse/HIVE-5317 > and other related JIRAs. > Serve as a counterpart Umbrella JIRA to ORC JIRA > https://issues.apache.org/jira/browse/ORC-42, so that corresponding > capabilities can be supported through SQL interfaces in Hive. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12857) LLAP: modify the decider to allow using LLAP with whitelisted UDFs
[ https://issues.apache.org/jira/browse/HIVE-12857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15147697#comment-15147697 ] Sergey Shelukhin commented on HIVE-12857: - Test failures are due to some other commit; not related. > LLAP: modify the decider to allow using LLAP with whitelisted UDFs > -- > > Key: HIVE-12857 > URL: https://issues.apache.org/jira/browse/HIVE-12857 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-12857.01.patch, HIVE-12857.02.patch, > HIVE-12857.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12981) ThriftCLIService uses incompatible getShortName() implementation
[ https://issues.apache.org/jira/browse/HIVE-12981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15147696#comment-15147696 ] Sergey Shelukhin commented on HIVE-12981: - CTE tests, partition_coltype_literals, and the last two fail elsewhere (CTE should already be fixed). > ThriftCLIService uses incompatible getShortName() implementation > > > Key: HIVE-12981 > URL: https://issues.apache.org/jira/browse/HIVE-12981 > Project: Hive > Issue Type: Bug > Components: Authentication, Authorization, CLI, Security >Affects Versions: 1.2.1, 2.1.0 >Reporter: Bolke de Bruin >Assignee: Thejas M Nair >Priority: Critical > Labels: kerberos > Attachments: 0001-HIVE-12981-Use-KerberosName.patch, > HIVE-12981-branch-1.2.patch, HIVE-12981.01.patch, HIVE-12981.patch > > > ThriftCLIService has a local implementation getShortName() that assumes a > short name is always the part before "@" and "/". This is not always the case > as Kerberos Rules (from Hadoop's KerberosName) might actually transform a > name to something else. > Considering a pending change to getShortName() (#HADOOP-12751) and the normal > use of KerberosName in other parts of Hive it only seems logical to use the > standard implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13058) Add session and operation_log directory deletion messages
[ https://issues.apache.org/jira/browse/HIVE-13058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15147675#comment-15147675 ] Hive QA commented on HIVE-13058: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12787855/HIVE-13058.1.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 22 failed/errored test(s), 9789 tests executed *Failed tests:* {noformat} TestSparkCliDriver-timestamp_lazy.q-bucketsortoptimize_insert_4.q-date_udf.q-and-12-more - did not produce a TEST-*.xml file org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_coltype_literals org.apache.hadoop.hive.cli.TestMiniTezCliDriver.org.apache.hadoop.hive.cli.TestMiniTezCliDriver org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_auto_join_filters org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_auto_join_nulls org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_delete_tmp_table org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_empty_join org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_mapreduce1 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_schema_evol_text_mapwork_table org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_temp_table org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_vector_dynpart_hashjoin_2 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_transform1 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_decimal_round_2 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_decimal_udf org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_interval_2 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_string_concat org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorization_14 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorized_distinct_gby org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testSimpleTable org.apache.hadoop.hive.ql.TestTxnCommands2.testInitiatorWithMultipleFailedCompactions org.apache.hive.jdbc.TestSSL.testSSLVersion {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6991/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6991/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6991/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 22 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12787855 - PreCommit-HIVE-TRUNK-Build > Add session and operation_log directory deletion messages > - > > Key: HIVE-13058 > URL: https://issues.apache.org/jira/browse/HIVE-13058 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Shinichi Yamashita >Assignee: Shinichi Yamashita >Priority: Minor > Attachments: HIVE-13058.1.patch > > > In the current master, when connecting to HS2, we can confirm that some > directories were created. For example the following messages of HS2 log. > {code} > 2016-02-14T13:57:52,412 INFO [HiveServer2-Handler-Pool: Thread-607]: > session.SessionState (SessionState.java:createPath(693)) - Created HDFS > directory: /tmp/hive/root/bfe28a51-5804-4cbb-952c-26709aed4652 > 2016-02-14T13:57:52,416 INFO [HiveServer2-Handler-Pool: Thread-607]: > session.SessionState (SessionState.java:createPath(693)) - Created local > directory: /tmp/hive/bfe28a51-5804-4cbb-952c-26709aed4652 > 2016-02-14T13:57:52,419 INFO [HiveServer2-Handler-Pool: Thread-607]: > session.SessionState (SessionState.java:createPath(693)) - Created HDFS > directory: /tmp/hive/root/bfe28a51-5804-4cbb-952c-26709aed4652/_tmp_space.db > 2016-02-14T13:57:52,421 INFO [HiveServer2-Handler-Pool: Thread-607]: > session.HiveSessionImpl (HiveSessionImpl.java:setOperationLogSessionDir(287)) > - Operation log session directory is created: > /tmp/hive/operation_logs/bfe28a51-5804-4cbb-952c-26709aed4652 > {code} > But when closing from HS2, we cannot confirm that these directories were > deleted. So I change it so that some messages about these directories > deletion may be output as follows. > {code} > 2016-02-14T13:57:56,877 INFO > [bfe28a51-5804-4cbb-952c-26709aed4652bfe28a51-5804-4cbb-952c-26709aed
[jira] [Commented] (HIVE-11752) Pre-materializing complex CTE queries
[ https://issues.apache.org/jira/browse/HIVE-11752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15147670#comment-15147670 ] Jesus Camacho Rodriguez commented on HIVE-11752: [~sershe], I pushed an ADDENDUM a few hours ago to fix this issue. > Pre-materializing complex CTE queries > - > > Key: HIVE-11752 > URL: https://issues.apache.org/jira/browse/HIVE-11752 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 2.1.0 >Reporter: Navis >Assignee: Jesus Camacho Rodriguez >Priority: Minor > Labels: TODOC2.1 > Fix For: 2.1.0 > > Attachments: HIVE-11752.03.patch, HIVE-11752.04.patch, > HIVE-11752.05.patch, HIVE-11752.1.patch.txt, HIVE-11752.2.patch.txt > > > Currently, hive regards CTE clauses as a simple alias to the query block, > which makes redundant works if it's used multiple times in a query. This > introduces a reference threshold for pre-materializing the CTE clause as a > volatile table (which is not exists in any form of metastore and just > accessible from QB). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11752) Pre-materializing complex CTE queries
[ https://issues.apache.org/jira/browse/HIVE-11752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15147668#comment-15147668 ] Sergey Shelukhin commented on HIVE-11752: - CTE tests are failing in all the JIRAs > Pre-materializing complex CTE queries > - > > Key: HIVE-11752 > URL: https://issues.apache.org/jira/browse/HIVE-11752 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 2.1.0 >Reporter: Navis >Assignee: Jesus Camacho Rodriguez >Priority: Minor > Labels: TODOC2.1 > Fix For: 2.1.0 > > Attachments: HIVE-11752.03.patch, HIVE-11752.04.patch, > HIVE-11752.05.patch, HIVE-11752.1.patch.txt, HIVE-11752.2.patch.txt > > > Currently, hive regards CTE clauses as a simple alias to the query block, > which makes redundant works if it's used multiple times in a query. This > introduces a reference threshold for pre-materializing the CTE clause as a > volatile table (which is not exists in any form of metastore and just > accessible from QB). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12981) ThriftCLIService uses incompatible getShortName() implementation
[ https://issues.apache.org/jira/browse/HIVE-12981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15147666#comment-15147666 ] Sergey Shelukhin commented on HIVE-12981: - Cancel and resubmit is not actually necessary, it's ok to submit another patch without changing the status. > ThriftCLIService uses incompatible getShortName() implementation > > > Key: HIVE-12981 > URL: https://issues.apache.org/jira/browse/HIVE-12981 > Project: Hive > Issue Type: Bug > Components: Authentication, Authorization, CLI, Security >Affects Versions: 1.2.1, 2.1.0 >Reporter: Bolke de Bruin >Assignee: Thejas M Nair >Priority: Critical > Labels: kerberos > Attachments: 0001-HIVE-12981-Use-KerberosName.patch, > HIVE-12981-branch-1.2.patch, HIVE-12981.01.patch, HIVE-12981.patch > > > ThriftCLIService has a local implementation getShortName() that assumes a > short name is always the part before "@" and "/". This is not always the case > as Kerberos Rules (from Hadoop's KerberosName) might actually transform a > name to something else. > Considering a pending change to getShortName() (#HADOOP-12751) and the normal > use of KerberosName in other parts of Hive it only seems logical to use the > standard implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13015) Bundle Log4j2 jars with hive-exec
[ https://issues.apache.org/jira/browse/HIVE-13015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15147664#comment-15147664 ] Sergey Shelukhin commented on HIVE-13015: - Looks like it broke a lot of tests... > Bundle Log4j2 jars with hive-exec > - > > Key: HIVE-13015 > URL: https://issues.apache.org/jira/browse/HIVE-13015 > Project: Hive > Issue Type: Bug >Affects Versions: 2.1.0 >Reporter: Prasanth Jayachandran >Assignee: Gopal V > Attachments: HIVE-13015.1.patch, HIVE-13015.1.patch > > > In some of the recent test runs, we are seeing multiple bindings for SLF4j > that causes issues with LOG4j2 logger. > {code} > SLF4J: Found binding in > [jar:file:/grid/0/hadoop/yarn/local/usercache/hrt_qa/appcache/application_1454694331819_0001/container_e06_1454694331819_0001_01_02/app/install/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] > {code} > We have added explicit exclusions for slf4j-log4j12 but some library is > pulling it transitively and it's getting packaged with hive libs. Also hive > currently uses version 1.7.5 for slf4j. We should add dependency convergence > for sl4fj and also remove packaging of slf4j-log4j12.*.jar -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12994) Implement support for NULLS FIRST/NULLS LAST
[ https://issues.apache.org/jira/browse/HIVE-12994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15147635#comment-15147635 ] Hive QA commented on HIVE-12994: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12787969/HIVE-12994.07.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 41 tests executed *Failed tests:* {noformat} Test failed: mysql/upgrade-1.2.0-to-2.0.0.mysql.sql {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-METASTORE-Test/116/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-METASTORE-Test/116/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-METASTORE-Test-116/ Messages: {noformat} LXC derby found. LXC derby is not started. Starting container... Container started. Preparing derby container... Container prepared. Calling /hive/testutils/metastore/dbs/derby/prepare.sh ... Server prepared. Calling /hive/testutils/metastore/dbs/derby/execute.sh ... Tests executed. LXC mysql found. LXC mysql is not started. Starting container... Container started. Preparing mysql container... Container prepared. Calling /hive/testutils/metastore/dbs/mysql/prepare.sh ... Server prepared. Calling /hive/testutils/metastore/dbs/mysql/execute.sh ... Test failed: mysql/upgrade-1.2.0-to-2.0.0.mysql.sql Tests executed. LXC oracle found. LXC oracle is not started. Starting container... Container started. Preparing oracle container... Container prepared. Calling /hive/testutils/metastore/dbs/oracle/prepare.sh ... Server prepared. Calling /hive/testutils/metastore/dbs/oracle/execute.sh ... Tests executed. LXC postgres found. LXC postgres is not started. Starting container... Container started. Preparing postgres container... Container prepared. Calling /hive/testutils/metastore/dbs/postgres/prepare.sh ... Server prepared. Calling /hive/testutils/metastore/dbs/postgres/execute.sh ... Tests executed. {noformat} This message is automatically generated. ATTACHMENT ID: 12787969 - PreCommit-HIVE-METASTORE-Test > Implement support for NULLS FIRST/NULLS LAST > > > Key: HIVE-12994 > URL: https://issues.apache.org/jira/browse/HIVE-12994 > Project: Hive > Issue Type: New Feature > Components: CBO, Parser, Serializers/Deserializers >Affects Versions: 2.1.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-12994.01.patch, HIVE-12994.02.patch, > HIVE-12994.03.patch, HIVE-12994.04.patch, HIVE-12994.05.patch, > HIVE-12994.06.patch, HIVE-12994.06.patch, HIVE-12994.07.patch, > HIVE-12994.patch > > > From SQL:2003, the NULLS FIRST and NULLS LAST options can be used to > determine whether nulls appear before or after non-null data values when the > ORDER BY clause is used. > SQL standard does not specify the behavior by default. Currently in Hive, > null values sort as if lower than any non-null value; that is, NULLS FIRST is > the default for ASC order, and NULLS LAST for DESC order. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12994) Implement support for NULLS FIRST/NULLS LAST
[ https://issues.apache.org/jira/browse/HIVE-12994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-12994: --- Attachment: HIVE-12994.07.patch > Implement support for NULLS FIRST/NULLS LAST > > > Key: HIVE-12994 > URL: https://issues.apache.org/jira/browse/HIVE-12994 > Project: Hive > Issue Type: New Feature > Components: CBO, Parser, Serializers/Deserializers >Affects Versions: 2.1.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-12994.01.patch, HIVE-12994.02.patch, > HIVE-12994.03.patch, HIVE-12994.04.patch, HIVE-12994.05.patch, > HIVE-12994.06.patch, HIVE-12994.06.patch, HIVE-12994.07.patch, > HIVE-12994.patch > > > From SQL:2003, the NULLS FIRST and NULLS LAST options can be used to > determine whether nulls appear before or after non-null data values when the > ORDER BY clause is used. > SQL standard does not specify the behavior by default. Currently in Hive, > null values sort as if lower than any non-null value; that is, NULLS FIRST is > the default for ASC order, and NULLS LAST for DESC order. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12543) Disable Hive ConstantPropagate optimizer when CBO has optimized the plan
[ https://issues.apache.org/jira/browse/HIVE-12543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15147609#comment-15147609 ] Jesus Camacho Rodriguez commented on HIVE-12543: [~ashutoshc], new patch addresses the following: - Dynamic partition pruning not being triggered anymore when static partition pruning has been performed. - testNegativeCliDriver_case_with_row_sequence fail : stateful functions are not allowed within CASE clauses. Previously, these were identified when we were running ConstantPropagate. With this patch, constant folding is done by Calcite, thus we need to identify them in the parser. - Sorted dynamic partition optimizer not being triggered : constant columns recognition in SortedDynPartitionOptimizer needed to be improved. Thus, we only need to create the follow-up JIRA for join merging. > Disable Hive ConstantPropagate optimizer when CBO has optimized the plan > > > Key: HIVE-12543 > URL: https://issues.apache.org/jira/browse/HIVE-12543 > Project: Hive > Issue Type: Sub-task > Components: CBO, Logical Optimizer >Affects Versions: 2.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-12543.01.patch, HIVE-12543.02.patch, > HIVE-12543.03.patch, HIVE-12543.04.patch, HIVE-12543.05.patch, > HIVE-12543.06.patch, HIVE-12543.07.patch, HIVE-12543.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12543) Disable Hive ConstantPropagate optimizer when CBO has optimized the plan
[ https://issues.apache.org/jira/browse/HIVE-12543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-12543: --- Attachment: (was: HIVE-12543.07.patch) > Disable Hive ConstantPropagate optimizer when CBO has optimized the plan > > > Key: HIVE-12543 > URL: https://issues.apache.org/jira/browse/HIVE-12543 > Project: Hive > Issue Type: Sub-task > Components: CBO, Logical Optimizer >Affects Versions: 2.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-12543.01.patch, HIVE-12543.02.patch, > HIVE-12543.03.patch, HIVE-12543.04.patch, HIVE-12543.05.patch, > HIVE-12543.06.patch, HIVE-12543.07.patch, HIVE-12543.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12543) Disable Hive ConstantPropagate optimizer when CBO has optimized the plan
[ https://issues.apache.org/jira/browse/HIVE-12543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-12543: --- Attachment: HIVE-12543.07.patch > Disable Hive ConstantPropagate optimizer when CBO has optimized the plan > > > Key: HIVE-12543 > URL: https://issues.apache.org/jira/browse/HIVE-12543 > Project: Hive > Issue Type: Sub-task > Components: CBO, Logical Optimizer >Affects Versions: 2.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-12543.01.patch, HIVE-12543.02.patch, > HIVE-12543.03.patch, HIVE-12543.04.patch, HIVE-12543.05.patch, > HIVE-12543.06.patch, HIVE-12543.07.patch, HIVE-12543.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13043) Reload function has no impact to function registry
[ https://issues.apache.org/jira/browse/HIVE-13043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-13043: --- Attachment: HIVE-13043.3.patch > Reload function has no impact to function registry > -- > > Key: HIVE-13043 > URL: https://issues.apache.org/jira/browse/HIVE-13043 > Project: Hive > Issue Type: Bug >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Attachments: HIVE-13043.1.patch, HIVE-13043.2.patch, > HIVE-13043.3.patch > > > With HIVE-2573, users should run "reload function" to refresh cached function > registry. However, "reload function" has no impact at all. We need to fix > this. Otherwise, HS2 needs to be restarted to see new global functions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13043) Reload function has no impact to function registry
[ https://issues.apache.org/jira/browse/HIVE-13043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15147469#comment-15147469 ] Hive QA commented on HIVE-13043: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12787826/HIVE-13043.2.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 9788 tests executed *Failed tests:* {noformat} TestSparkCliDriver-timestamp_lazy.q-bucketsortoptimize_insert_4.q-date_udf.q-and-12-more - did not produce a TEST-*.xml file org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_coltype_literals org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import org.apache.hadoop.hive.ql.parse.authorization.plugin.sqlstd.TestOperation2Privilege.checkHiveOperationTypeMatch org.apache.hive.jdbc.TestSSL.testSSLVersion org.apache.hive.service.TestHS2ImpersonationWithRemoteMS.org.apache.hive.service.TestHS2ImpersonationWithRemoteMS {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6990/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6990/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6990/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 6 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12787826 - PreCommit-HIVE-TRUNK-Build > Reload function has no impact to function registry > -- > > Key: HIVE-13043 > URL: https://issues.apache.org/jira/browse/HIVE-13043 > Project: Hive > Issue Type: Bug >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Attachments: HIVE-13043.1.patch, HIVE-13043.2.patch > > > With HIVE-2573, users should run "reload function" to refresh cached function > registry. However, "reload function" has no impact at all. We need to fix > this. Otherwise, HS2 needs to be restarted to see new global functions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12543) Disable Hive ConstantPropagate optimizer when CBO has optimized the plan
[ https://issues.apache.org/jira/browse/HIVE-12543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-12543: --- Attachment: HIVE-12543.07.patch > Disable Hive ConstantPropagate optimizer when CBO has optimized the plan > > > Key: HIVE-12543 > URL: https://issues.apache.org/jira/browse/HIVE-12543 > Project: Hive > Issue Type: Sub-task > Components: CBO, Logical Optimizer >Affects Versions: 2.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-12543.01.patch, HIVE-12543.02.patch, > HIVE-12543.03.patch, HIVE-12543.04.patch, HIVE-12543.05.patch, > HIVE-12543.06.patch, HIVE-12543.07.patch, HIVE-12543.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12878) Support Vectorization for TEXTFILE and other formats
[ https://issues.apache.org/jira/browse/HIVE-12878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-12878: Attachment: HIVE-12878.03.patch > Support Vectorization for TEXTFILE and other formats > > > Key: HIVE-12878 > URL: https://issues.apache.org/jira/browse/HIVE-12878 > Project: Hive > Issue Type: New Feature > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-12878.01.patch, HIVE-12878.02.patch, > HIVE-12878.03.patch > > > Support vectorizing when the input format is TEXTFILE and other formats for > better Map Vertex performance. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13000) Hive returns useless parsing error
[ https://issues.apache.org/jira/browse/HIVE-13000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15147328#comment-15147328 ] Alina Abramova commented on HIVE-13000: --- Have we any mention about this ticket? > Hive returns useless parsing error > --- > > Key: HIVE-13000 > URL: https://issues.apache.org/jira/browse/HIVE-13000 > Project: Hive > Issue Type: Bug >Affects Versions: 0.13.0, 1.0.0, 1.2.1 >Reporter: Alina Abramova >Assignee: Alina Abramova >Priority: Minor > Attachments: HIVE-13000.1.patch, HIVE-13000.2.patch, > HIVE-13000.3.patch > > > When I run query like these I receive unclear exception > hive> SELECT record FROM ctest GROUP BY record.instance_id; > FAILED: SemanticException Error in parsing > It will be clearer if it would be like: > hive> SELECT record FROM ctest GROUP BY record.instance_id; > FAILED: SemanticException Expression not in GROUP BY key record -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13057) Remove duplicate copies of TableDesc property values in PartitionDesc
[ https://issues.apache.org/jira/browse/HIVE-13057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15147281#comment-15147281 ] Hive QA commented on HIVE-13057: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12787776/HIVE-13057.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 15 failed/errored test(s), 9803 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_mat_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_mat_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_mat_3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_mat_4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_mat_5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_coltype_literals org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_5 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_1 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_2 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_3 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_4 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_5 org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import org.apache.hive.jdbc.TestSSL.testSSLVersion {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6989/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6989/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6989/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 15 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12787776 - PreCommit-HIVE-TRUNK-Build > Remove duplicate copies of TableDesc property values in PartitionDesc > - > > Key: HIVE-13057 > URL: https://issues.apache.org/jira/browse/HIVE-13057 > Project: Hive > Issue Type: Bug >Reporter: Mohit Sabharwal >Assignee: Mohit Sabharwal > Attachments: HIVE-13057.patch > > > For a partitioned table, each PartitionDesc has a copy of corresponding > TableDesc. > While TableDesc is mutable and hence cannot be interned, it's property values > can be. > For a simple select on a table with 100K partitions, this cut total number of > String instances by ~65%. > Most replicated strings were location, serde, input/output format, column, > types, table name, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13015) Bundle Log4j2 jars with hive-exec
[ https://issues.apache.org/jira/browse/HIVE-13015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15147185#comment-15147185 ] Hive QA commented on HIVE-13015: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12787756/HIVE-13015.1.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 601 failed/errored test(s), 9803 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_mat_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_mat_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_mat_3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_mat_4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_mat_5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_coltype_literals org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_5 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_1 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_2 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_3 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_4 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_5 org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_add_part_multiple org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_alter_merge_orc org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_alter_merge_stats_orc org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_annotate_stats_join org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join0 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join1 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join10 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join11 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join12 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join13 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join14 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join15 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join16 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join17 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join18 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join18_multi_distinct org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join19 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join2 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join20 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join21 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join22 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join23 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join24 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join26 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join27 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join28 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join29 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join3 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join30 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join31 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join32 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join4 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join5 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join6 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join7 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join8 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join9 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join_filters org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join_nulls org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join_reordering_values org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join_stats org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join_stats2 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join_without_localtask org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_smb_mapjoin_14 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_sortmerge_join_1 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_sortmerge_join_10 org.apache.hadoo
[jira] [Commented] (HIVE-13055) Add unit tests for HIVE-11512
[ https://issues.apache.org/jira/browse/HIVE-13055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15147080#comment-15147080 ] Hive QA commented on HIVE-13055: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12787730/HIVE-13055.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 16 failed/errored test(s), 9796 tests executed *Failed tests:* {noformat} TestSparkCliDriver-timestamp_lazy.q-bucketsortoptimize_insert_4.q-date_udf.q-and-12-more - did not produce a TEST-*.xml file org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_mat_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_mat_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_mat_3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_mat_4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_mat_5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_coltype_literals org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_5 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_1 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_2 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_3 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_4 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_5 org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import org.apache.hive.jdbc.TestSSL.testSSLVersion {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6987/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6987/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6987/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 16 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12787730 - PreCommit-HIVE-TRUNK-Build > Add unit tests for HIVE-11512 > - > > Key: HIVE-13055 > URL: https://issues.apache.org/jira/browse/HIVE-13055 > Project: Hive > Issue Type: Test > Components: HiveServer2 >Affects Versions: 1.3.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam > Attachments: HIVE-13055.patch > > > HIVE-11866 adds the ability to test LDAP authenticator with a LDAP Server. We > need more test coverage for feature in HIVE-11512. -- This message was sent by Atlassian JIRA (v6.3.4#6332)