[jira] [Commented] (HIVE-12650) Increase default value of hive.spark.client.server.connect.timeout to exceeds spark.yarn.am.waitTime
[ https://issues.apache.org/jira/browse/HIVE-12650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148177#comment-15148177 ] JoneZhang commented on HIVE-12650: -- Hi all, I'm sorrry reply you so late. Yes hive.spark.client.server.connect.timeout and spark.yarn.am.waitTime does not have any relations. hive.spark.client.server.connect.timeout is the timeout between RPC server and client handshake.When no container is available, hive cient will exit after hive.spark.client.server.connect.timeout. spark.yarn.am.waitTime is the time the Spark AM waits for the SparkContext to be created after the AM has been launched. There are two types of error log 1.Client closed before SASL negotiation finished was happened on resubmitted. See https://issues.apache.org/jira/browse/HIVE-12649. 2.Connection refused: /hiveclientip:port was happend when am tries to connect back to Hive. Container: container_1448873753366_113453_01_01 on 10.247.169.134_8041 LogType: stderr LogLength: 3302 Log Contents: Please use CMSClassUnloadingEnabled in place of CMSPermGenSweepingEnabled in the future Please use CMSClassUnloadingEnabled in place of CMSPermGenSweepingEnabled in the future 15/12/09 02:11:48 INFO yarn.ApplicationMaster: Registered signal handlers for [TERM, HUP, INT] 15/12/09 02:11:48 INFO yarn.ApplicationMaster: ApplicationAttemptId: appattempt_1448873753366_113453_01 15/12/09 02:11:49 INFO spark.SecurityManager: Changing view acls to: mqq 15/12/09 02:11:49 INFO spark.SecurityManager: Changing modify acls to: mqq 15/12/09 02:11:49 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(mqq); users with modify permissions: Set(mqq) 15/12/09 02:11:49 INFO yarn.ApplicationMaster: Starting the user application in a separate Thread 15/12/09 02:11:49 INFO yarn.ApplicationMaster: Waiting for spark context initialization 15/12/09 02:11:49 INFO yarn.ApplicationMaster: Waiting for spark context initialization ... 15/12/09 02:11:49 INFO client.RemoteDriver: Connecting to: 10.179.12.140:58013 15/12/09 02:11:49 ERROR yarn.ApplicationMaster: User class threw exception: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused: /10.179.12.140:58013 java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused: /10.179.12.140:58013 at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:37) at org.apache.hive.spark.client.RemoteDriver.(RemoteDriver.java:156) at org.apache.hive.spark.client.RemoteDriver.main(RemoteDriver.java:556) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:483) Caused by: java.net.ConnectException: Connection refused: /10.179.12.140:58013 at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:208) at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:287) at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:528) at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116) at java.lang.Thread.run(Thread.java:745) 15/12/09 02:11:49 INFO yarn.ApplicationMaster: Final app status: FAILED, exitCode: 15, (reason: User class threw exception: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused: /10.179.12.140:58013) 15/12/09 02:11:59 ERROR yarn.ApplicationMaster: SparkContext did not initialize after waiting for 15 ms. Please check earlier log output for errors. Failing the application. 15/12/09 02:11:59 INFO util.Utils: Shutdown hook called > Increase default value of hive.spark.client.server.connect.timeout to exceeds > spark.yarn.am.waitTime > > > Key: HIVE-12650 > URL: https://issues.apache.org/jira/browse/HIVE-12650 > Project: Hive > Issue Type: Bug >Affects Versions:
[jira] [Commented] (HIVE-12649) Hive on Spark will resubmitted application when not enough resouces to launch yarn application master
[ https://issues.apache.org/jira/browse/HIVE-12649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148175#comment-15148175 ] JoneZhang commented on HIVE-12649: -- The applications submited by estimate reducer number will fast fail when the reources is free after awhile. The error log is like below Container: container_1448873753366_121022_02_01 on 10.239.243.69_8041 === LogType: stderr LogLength: 4284 Log Contents: Please use CMSClassUnloadingEnabled in place of CMSPermGenSweepingEnabled in the future Please use CMSClassUnloadingEnabled in place of CMSPermGenSweepingEnabled in the future 15/12/09 16:29:31 INFO yarn.ApplicationMaster: Registered signal handlers for [TERM, HUP, INT] 15/12/09 16:29:32 INFO yarn.ApplicationMaster: ApplicationAttemptId: appattempt_1448873753366_121022_02 15/12/09 16:29:33 INFO spark.SecurityManager: Changing view acls to: mqq 15/12/09 16:29:33 INFO spark.SecurityManager: Changing modify acls to: mqq 15/12/09 16:29:33 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(mqq); users with modify permissions: Set(mqq) 15/12/09 16:29:33 INFO yarn.ApplicationMaster: Starting the user application in a separate Thread 15/12/09 16:29:33 INFO yarn.ApplicationMaster: Waiting for spark context initialization 15/12/09 16:29:33 INFO yarn.ApplicationMaster: Waiting for spark context initialization ... 15/12/09 16:29:33 INFO client.RemoteDriver: Connecting to: 10.179.12.140:38842 15/12/09 16:29:33 WARN rpc.Rpc: Invalid log level null, reverting to default. 15/12/09 16:29:33 ERROR yarn.ApplicationMaster: User class threw exception: java.util.concurrent.ExecutionException: javax.security.sasl.SaslException: Client closed before SASL negotiation finished. java.util.concurrent.ExecutionException: javax.security.sasl.SaslException: Client closed before SASL negotiation finished. at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:37) at org.apache.hive.spark.client.RemoteDriver.(RemoteDriver.java:156) at org.apache.hive.spark.client.RemoteDriver.main(RemoteDriver.java:556) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:483) Caused by: javax.security.sasl.SaslException: Client closed before SASL negotiation finished. at org.apache.hive.spark.client.rpc.Rpc$SaslClientHandler.dispose(Rpc.java:449) at org.apache.hive.spark.client.rpc.SaslHandler.channelInactive(SaslHandler.java:90) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:233) at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:219) at io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:75) at org.apache.hive.spark.client.rpc.KryoMessageCodec.channelInactive(KryoMessageCodec.java:127) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:233) at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:219) at io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:75) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:233) at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:219) at io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:769) at io.netty.channel.AbstractChannel$AbstractUnsafe$5.run(AbstractChannel.java:567) at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:380) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357) at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116) at java.lang.Thread.run(Thread.java:745) 15/12/09 16:29:33 INFO yarn.ApplicationMaster: Final app status: FAILED, exitCode: 15, (reason: User class threw exception: java.util.concurrent.ExecutionException: javax.security.sasl.SaslException: Client closed before SASL negotiation finished.) 15/12/09 16:29:43 ERROR yarn.ApplicationMaster: SparkContext did not initialize after waiting for 15 ms. Please check earlier log output for errors. Failing the application.
[jira] [Commented] (HIVE-13043) Reload function has no impact to function registry
[
https://issues.apache.org/jira/browse/HIVE-13043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148167#comment-15148167
]
Hive QA commented on HIVE-13043:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12787957/HIVE-13043.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 21 failed/errored test(s), 9774 tests
executed
*Failed tests:*
{noformat}
TestMiniTezCliDriver-auto_join30.q-vector_data_types.q-filter_join_breaktask.q-and-12-more
- did not produce a TEST-*.xml file
TestMiniTezCliDriver-vector_interval_2.q-mapreduce1.q-vectorization_14.q-and-12-more
- did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_coltype_literals
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.org.apache.hadoop.hive.cli.TestMiniTezCliDriver
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_bucket3
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_cte_mat_2
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_explainuser_1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_load_dyn_part2
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_metadata_only_queries
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_optimize_nullscan
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_analyze
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge10
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_schema_evol_text_nonvec_fetchwork_table
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_script_env_var2
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_union8
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_distinct_2
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_orderby_5
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_outer_join3
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorization_7
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6996/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6996/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6996/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 21 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12787957 - PreCommit-HIVE-TRUNK-Build
> Reload function has no impact to function registry
> --
>
> Key: HIVE-13043
> URL: https://issues.apache.org/jira/browse/HIVE-13043
> Project: Hive
> Issue Type: Bug
>Reporter: Jimmy Xiang
>Assignee: Jimmy Xiang
> Attachments: HIVE-13043.1.patch, HIVE-13043.2.patch,
> HIVE-13043.3.patch
>
>
> With HIVE-2573, users should run "reload function" to refresh cached function
> registry. However, "reload function" has no impact at all. We need to fix
> this. Otherwise, HS2 needs to be restarted to see new global functions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12612) beeline always exits with 0 status when reading query from standard input
[
https://issues.apache.org/jira/browse/HIVE-12612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148103#comment-15148103
]
Ruslan Dautkhanov commented on HIVE-12612:
--
I could not reproduce your test case.
{code:shell}
$ echo "show databases;" | bl -f /dev/stdin
scan complete in 2ms
Connecting to jdbc:hive2://xx.yy.com:1/default;principal=hive/_HOST@HADOOP
Connected to: Apache Hive (version 1.1.0-cdh5.5.1)
Driver: Hive JDBC (version 1.1.0-cdh5.5.0)
Transaction isolation: TRANSACTION_REPEATABLE_READ
/dev/stdin (No such device or address)
{code}
> beeline always exits with 0 status when reading query from standard input
> -
>
> Key: HIVE-12612
> URL: https://issues.apache.org/jira/browse/HIVE-12612
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 1.1.0
> Environment: CDH5.5.0
>Reporter: Paulo Sequeira
>Priority: Minor
>
> Similar to what was reported on HIVE-6978, but now it only happens when the
> query is read from the standard input. For example, the following fails as
> expected:
> {code}
> bash$ if beeline -u "jdbc:hive2://..." -e "boo;" ; then echo "Ok?!" ; else
> echo "Failed!" ; fi
> Connecting to jdbc:hive2://...
> Connected to: Apache Hive (version 1.1.0-cdh5.5.0)
> Driver: Hive JDBC (version 1.1.0-cdh5.5.0)
> Transaction isolation: TRANSACTION_REPEATABLE_READ
> Error: Error while compiling statement: FAILED: ParseException line 1:0
> cannot recognize input near 'boo' '' '' (state=42000,code=4)
> Closing: 0: jdbc:hive2://...
> Failed!
> {code}
> But the following does not:
> {code}
> bash$ if echo "boo;"|beeline -u "jdbc:hive2://..." ; then echo "Ok?!" ; else
> echo "Failed!" ; fi
> Connecting to jdbc:hive2://...
> Connected to: Apache Hive (version 1.1.0-cdh5.5.0)
> Driver: Hive JDBC (version 1.1.0-cdh5.5.0)
> Transaction isolation: TRANSACTION_REPEATABLE_READ
> Beeline version 1.1.0-cdh5.5.0 by Apache Hive
> 0: jdbc:hive2://...:8> Error: Error while compiling statement: FAILED:
> ParseException line 1:0 cannot recognize input near 'boo' '' ''
> (state=42000,code=4)
> 0: jdbc:hive2://...:8> Closing: 0: jdbc:hive2://...
> Ok?!
> {code}
> This was misleading our batch scripts to always believe that the execution of
> the queries succeded, when sometimes that was not the case.
> h2. Workaround
> We found we can work around the issue by always using the -e or the -f
> parameters, and even reading the standard input through the /dev/stdin device
> (this was useful because a lot of the scripts fed the queries from here
> documents), like this:
> {code:title=some-script.sh}
> #!/bin/sh
> set -o nounset -o errexit -o pipefail
> # As beeline is failing to report an error status if reading the query
> # to be executed from STDIN, check whether no -f or -e option is used
> # and, in that case, pretend it has to read the query from a regular
> # file using -f to read from /dev/stdin
> function beeline_workaround_exit_status () {
> for arg in "$@"
> do if [ "$arg" = "-f" -o "$arg" = "-e" ]
>then beeline -u "..." "$@"
> return
>fi
> done
> beeline -u "..." "$@" -f /dev/stdin
> }
> beeline_workaround_exit_status < boo;
> EOF
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12543) Disable Hive ConstantPropagate optimizer when CBO has optimized the plan
[
https://issues.apache.org/jira/browse/HIVE-12543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148061#comment-15148061
]
Hive QA commented on HIVE-12543:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12787968/HIVE-12543.07.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 9803 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_coltype_literals
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6995/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6995/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6995/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12787968 - PreCommit-HIVE-TRUNK-Build
> Disable Hive ConstantPropagate optimizer when CBO has optimized the plan
>
>
> Key: HIVE-12543
> URL: https://issues.apache.org/jira/browse/HIVE-12543
> Project: Hive
> Issue Type: Sub-task
> Components: CBO, Logical Optimizer
>Affects Versions: 2.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-12543.01.patch, HIVE-12543.02.patch,
> HIVE-12543.03.patch, HIVE-12543.04.patch, HIVE-12543.05.patch,
> HIVE-12543.06.patch, HIVE-12543.07.patch, HIVE-12543.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12064) prevent transactional=false
[ https://issues.apache.org/jira/browse/HIVE-12064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148060#comment-15148060 ] Wei Zheng commented on HIVE-12064: -- Test failures are not related. [~alangates] Can you take a look? Thanks. > prevent transactional=false > --- > > Key: HIVE-12064 > URL: https://issues.apache.org/jira/browse/HIVE-12064 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Wei Zheng >Priority: Critical > Attachments: HIVE-12064.2.patch, HIVE-12064.3.patch, > HIVE-12064.4.patch, HIVE-12064.patch > > > currently a tblproperty transactional=true must be set to make a table behave > in ACID compliant way. > This is misleading in that it seems like changing it to transactional=false > makes the table non-acid but on disk layout of acid table is different than > plain tables. So changing this property may cause wrong data to be returned. > Should prevent transactional=false. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HIVE-8339) Job status not found after 100% succeded map
[ https://issues.apache.org/jira/browse/HIVE-8339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zengzhaozheng reassigned HIVE-8339: --- Assignee: zengzhaozheng (was: Valera Chevtaev) > Job status not found after 100% succeded map > --- > > Key: HIVE-8339 > URL: https://issues.apache.org/jira/browse/HIVE-8339 > Project: Hive > Issue Type: Bug >Affects Versions: 0.13.1 > Environment: Hadoop 2.4.0, Hive 0.13.1. > Amazon EMR cluster of 9 i2.4xlarge nodes. > 800+GB of data in HDFS. >Reporter: Valera Chevtaev >Assignee: zengzhaozheng > > According to the logs it seems that the jobs 100% succeed for both map and > reduce but then wasn't able to get the status of the job from job history > server. > Hive logs: > 2014-10-03 07:57:26,593 INFO [main]: exec.Task > (SessionState.java:printInfo(536)) - 2014-10-03 07:57:26,593 Stage-1 map = > 100%, reduce = 99%, Cumulative CPU 872541.02 sec > 2014-10-03 07:57:47,447 INFO [main]: exec.Task > (SessionState.java:printInfo(536)) - 2014-10-03 07:57:47,446 Stage-1 map = > 100%, reduce = 100%, Cumulative CPU 872566.55 sec > 2014-10-03 07:57:48,710 INFO [main]: mapred.ClientServiceDelegate > (ClientServiceDelegate.java:getProxy(273)) - Application state is completed. > FinalApplicationStatus=SUCCEEDED. Redirecting to job history server > 2014-10-03 07:57:48,716 ERROR [main]: exec.Task > (SessionState.java:printError(545)) - Ended Job = job_1412263771568_0002 with > exception 'java.io.IOException(Could not find status of > job:job_1412263771568_0002)' > java.io.IOException: Could not find status of job:job_1412263771568_0002 >at > org.apache.hadoop.hive.ql.exec.mr.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:294) >at > org.apache.hadoop.hive.ql.exec.mr.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:547) >at > org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:426) >at > org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:136) >at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153) >at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85) >at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1503) >at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1270) >at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1088) >at org.apache.hadoop.hive.ql.Driver.run(Driver.java:911) >at org.apache.hadoop.hive.ql.Driver.run(Driver.java:901) >at > org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:275) >at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:227) >at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:430) >at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:366) >at > org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:463) >at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:479) >at > org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:759) >at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:697) >at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:636) >at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >at java.lang.reflect.Method.invoke(Method.java:606) >at org.apache.hadoop.util.RunJar.main(RunJar.java:212) > 2014-10-03 07:57:48,763 ERROR [main]: ql.Driver > (SessionState.java:printError(545)) - FAILED: Execution Error, return code 1 > from org.apache.hadoop.hive.ql.exec.mr.MapRedTask -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12878) Support Vectorization for TEXTFILE and other formats
[
https://issues.apache.org/jira/browse/HIVE-12878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148018#comment-15148018
]
Hive QA commented on HIVE-12878:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12787933/HIVE-12878.03.patch
{color:green}SUCCESS:{color} +1 due to 23 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1210 failed/errored test(s), 9469 tests
executed
*Failed tests:*
{noformat}
TestDateIntervalDayTime - did not produce a TEST-*.xml file
TestSparkCliDriver-auto_join11.q-vector_groupby_3.q-smb_mapjoin_8.q-and-3-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-auto_join9.q-bucketmapjoin10.q-skewjoinopt19.q-and-12-more -
did not produce a TEST-*.xml file
TestSparkCliDriver-auto_join_reordering_values.q-auto_sortmerge_join_7.q-multigroupby_singlemr.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-avro_decimal_native.q-bucketmapjoin12.q-ppd_outer_join2.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-bucketsortoptimize_insert_7.q-enforce_order.q-join36.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-escape_distributeby1.q-union_remove_7.q-skewjoin_union_remove_2.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-groupby2_noskew_multi_distinct.q-skewjoin_noskew.q-vector_data_types.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-groupby4.q-timestamp_null.q-auto_join23.q-and-12-more - did
not produce a TEST-*.xml file
TestSparkCliDriver-groupby_complex_types.q-vectorization_10.q-join4.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-groupby_grouping_id2.q-bucketmapjoin4.q-groupby7.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-join_cond_pushdown_3.q-groupby7_noskew.q-auto_join13.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-join_rc.q-insert1.q-vectorized_rcfile_columnar.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-load_dyn_part12.q-nullgroup4_multi_distinct.q-union14.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-load_dyn_part5.q-skewjoinopt8.q-groupby1_noskew.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-ppd_gby_join.q-stats2.q-groupby_rollup1.q-and-12-more - did
not produce a TEST-*.xml file
TestSparkCliDriver-ppd_join4.q-join9.q-ppd_join3.q-and-12-more - did not
produce a TEST-*.xml file
TestSparkCliDriver-skewjoinopt15.q-bucketmapjoin3.q-auto_join10.q-and-12-more -
did not produce a TEST-*.xml file
TestSparkCliDriver-smb_mapjoin_15.q-auto_sortmerge_join_13.q-auto_join18_multi_distinct.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-smb_mapjoin_4.q-auto_join19.q-mapreduce1.q-and-12-more - did
not produce a TEST-*.xml file
TestSparkCliDriver-stats13.q-groupby6_map.q-join_casesensitive.q-and-12-more -
did not produce a TEST-*.xml file
TestSparkCliDriver-tez_joins_explain.q-input17.q-union29.q-and-12-more - did
not produce a TEST-*.xml file
TestSparkCliDriver-timestamp_lazy.q-bucketsortoptimize_insert_4.q-date_udf.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-vector_distinct_2.q-load_dyn_part2.q-udf_percentile.q-and-12-more
- did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alias_casted_column
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_partition_coltype
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_deep_filters
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_filter
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_groupby
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_groupby2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_select
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_table
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ansi_sql_arithmetic
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join0
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join14
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join15
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join17
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join18
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join18_multi_distinct
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join19
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join20
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join24
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join26
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join27
[jira] [Updated] (HIVE-13063) Create UDFs for CHAR and REPLACE
[
https://issues.apache.org/jira/browse/HIVE-13063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alejandro Fernandez updated HIVE-13063:
---
Attachment: HIVE-13063.patch
> Create UDFs for CHAR and REPLACE
> -
>
> Key: HIVE-13063
> URL: https://issues.apache.org/jira/browse/HIVE-13063
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 1.2.0
>Reporter: Alejandro Fernandez
>Assignee: Alejandro Fernandez
> Fix For: 2.1.0
>
> Attachments: HIVE-13063.patch
>
>
> Create UDFS for these functions.
> CHAR: convert n where n : [0, 256) into the ascii equivalent as a varchar. If
> n is less than 0 or greater than 255, return the empty string. If n is 0,
> return null.
> REPLACE: replace all substrings of 'str' that match 'search' with 'rep'.
> Example. SELECT REPLACE('Hack and Hue', 'H', 'BL');
> Equals 'BLack and BLue'"
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9457) Fix obsolete parameter name in HiveConf description of hive.hashtable.initialCapacity
[
https://issues.apache.org/jira/browse/HIVE-9457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shannon Ladymon updated HIVE-9457:
--
Attachment: HIVE-9457.patch
Patch attached.
> Fix obsolete parameter name in HiveConf description of
> hive.hashtable.initialCapacity
> -
>
> Key: HIVE-9457
> URL: https://issues.apache.org/jira/browse/HIVE-9457
> Project: Hive
> Issue Type: Bug
> Components: Documentation
>Affects Versions: 0.14.0
>Reporter: Lefty Leverenz
>Assignee: Shannon Ladymon
>Priority: Minor
> Attachments: HIVE-9457.patch
>
>
> The description of *hive.hashtable.initialCapacity* in HiveConf.java refers
> to a parameter that existed in an early patch for HIVE-7616
> ("hive.hashtable.stats.key.estimate.adjustment") but was renamed in later
> patches. So change *hive.hashtable.stats.key.estimate.adjustment* to
> *hive.hashtable.key.count.adjustment* in this parameter definition in
> HiveConf.java:
> {code}
> HIVEHASHTABLETHRESHOLD("hive.hashtable.initialCapacity", 10, "Initial
> capacity of " +
> "mapjoin hashtable if statistics are absent, or if
> hive.hashtable.stats.key.estimate.adjustment is set to 0"),
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9457) Fix obsolete parameter name in HiveConf description of hive.hashtable.initialCapacity
[
https://issues.apache.org/jira/browse/HIVE-9457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lefty Leverenz updated HIVE-9457:
-
Assignee: Shannon Ladymon
> Fix obsolete parameter name in HiveConf description of
> hive.hashtable.initialCapacity
> -
>
> Key: HIVE-9457
> URL: https://issues.apache.org/jira/browse/HIVE-9457
> Project: Hive
> Issue Type: Bug
> Components: Documentation
>Affects Versions: 0.14.0
>Reporter: Lefty Leverenz
>Assignee: Shannon Ladymon
>Priority: Minor
>
> The description of *hive.hashtable.initialCapacity* in HiveConf.java refers
> to a parameter that existed in an early patch for HIVE-7616
> ("hive.hashtable.stats.key.estimate.adjustment") but was renamed in later
> patches. So change *hive.hashtable.stats.key.estimate.adjustment* to
> *hive.hashtable.key.count.adjustment* in this parameter definition in
> HiveConf.java:
> {code}
> HIVEHASHTABLETHRESHOLD("hive.hashtable.initialCapacity", 10, "Initial
> capacity of " +
> "mapjoin hashtable if statistics are absent, or if
> hive.hashtable.stats.key.estimate.adjustment is set to 0"),
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13043) Reload function has no impact to function registry
[ https://issues.apache.org/jira/browse/HIVE-13043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147856#comment-15147856 ] Jimmy Xiang commented on HIVE-13043: UPloaded patch 3 to fix TestOperation2Privilege.checkHiveOperationTypeMatch > Reload function has no impact to function registry > -- > > Key: HIVE-13043 > URL: https://issues.apache.org/jira/browse/HIVE-13043 > Project: Hive > Issue Type: Bug >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Attachments: HIVE-13043.1.patch, HIVE-13043.2.patch, > HIVE-13043.3.patch > > > With HIVE-2573, users should run "reload function" to refresh cached function > registry. However, "reload function" has no impact at all. We need to fix > this. Otherwise, HS2 needs to be restarted to see new global functions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13019) Optimizer COLLECT_LIST/COLLECT_SET
[
https://issues.apache.org/jira/browse/HIVE-13019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147807#comment-15147807
]
Ruslan Dautkhanov commented on HIVE-13019:
--
Thank you Gopal. Oracle has "RELY NOVALIDATE" option for constraints.. would it
be easier for Hive to start with something like that for PK/FK constraints? So
CBO has more information for optimizations. It does not have to actually check
if that constraint is relationship is actually true; it can just "rely" on that
constraint.. Is this already plan for Hive 2.0? Thanks again.
> Optimizer COLLECT_LIST/COLLECT_SET
> ---
>
> Key: HIVE-13019
> URL: https://issues.apache.org/jira/browse/HIVE-13019
> Project: Hive
> Issue Type: Improvement
> Components: CBO, Logical Optimizer
>Reporter: Dustin Cote
>Priority: Minor
>
> Currently when using a COLLECT_SET/COLLECT_LIST that involves data from a
> single table, the aggregation is done after any JOIN operation that is
> present in the query. For example:
> {code}
> insert into table nested_customers_orders
> select c.*, collect_list(named_struct("oid", o.oid, "order_date": o.date...))
> from customers c inner join orders o on (c.cid = o.oid)
> group by o.oid, o.date,...
> {code}
> If we can tell the optimizer to perform the COLLECT_LIST first (where
> possible) we can see some performance gains in this pattern of query.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12244) Refactoring code for avoiding of comparison of Strings and do comparison on Path
[
https://issues.apache.org/jira/browse/HIVE-12244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147806#comment-15147806
]
Hive QA commented on HIVE-12244:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12787865/HIVE-12244.7.patch
{color:green}SUCCESS:{color} +1 due to 9 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 588 failed/errored test(s), 9698 tests
executed
*Failed tests:*
{noformat}
TestCliDriver-avro_decimal_native.q-alter_file_format.q-groupby3_map_skew.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-gby_star.q-udf_regexp_replace.q-load_dyn_part2.q-and-12-more -
did not produce a TEST-*.xml file
TestCliDriver-orc_createas1.q-nestedvirtual.q-union35.q-and-12-more - did not
produce a TEST-*.xml file
TestCliDriver-orc_merge6.q-input44.q-smb_mapjoin_4.q-and-12-more - did not
produce a TEST-*.xml file
TestCliDriver-part_inherit_tbl_props_with_star.q-truncate_table.q-list_bucket_query_oneskew_2.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-skewjoin_mapjoin11.q-udf_least.q-plan_json.q-and-12-more - did
not produce a TEST-*.xml file
TestMiniTezCliDriver-vectorization_10.q-schema_evol_orc_acidvec_mapwork_part.q-vector_non_string_partition.q-and-12-more
- did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_acid_globallimit
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_acid_vectorization
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_acid_vectorization_partition
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_acid_vectorization_project
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_add_part_exist
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_concatenate_indexed_table
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_index
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_merge_2_orc
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_merge_orc
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_partition_change_col
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_rename_partition
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_table_invalidate_column_stats
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_filter
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_groupby
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_limit
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_part
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_select
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_table
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_union
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_9
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_delete
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_delete_own_table
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_index
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_insert
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_parts
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_show_grant
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_update
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_update_own_table
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join10
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join11
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join14
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join15
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join16
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join17
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join19
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join20
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join21
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join22
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join23
[jira] [Updated] (HIVE-12673) Orcfiledump throws NPE when no files are available
[
https://issues.apache.org/jira/browse/HIVE-12673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran updated HIVE-12673:
-
Attachment: HIVE-12673.3.patch
Not sure why this patch was not picked by Precommit QA. Retrying again.
> Orcfiledump throws NPE when no files are available
> --
>
> Key: HIVE-12673
> URL: https://issues.apache.org/jira/browse/HIVE-12673
> Project: Hive
> Issue Type: Bug
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
> Attachments: HIVE-12673.1.patch, HIVE-12673.2.patch,
> HIVE-12673.2.patch, HIVE-12673.3.patch, HIVE-12673.3.rebased.patch
>
>
> {noformat}
> Exception in thread "main" java.lang.NullPointerException
> at org.codehaus.jettison.json.JSONTokener.more(JSONTokener.java:106)
> at org.codehaus.jettison.json.JSONTokener.next(JSONTokener.java:116)
> at
> org.codehaus.jettison.json.JSONTokener.nextClean(JSONTokener.java:170)
> at org.codehaus.jettison.json.JSONObject.(JSONObject.java:185)
> at org.codehaus.jettison.json.JSONObject.(JSONObject.java:293)
> at
> org.apache.hadoop.hive.ql.io.orc.JsonFileDump.printJsonMetaData(JsonFileDump.java:197)
> at org.apache.hadoop.hive.ql.io.orc.FileDump.main(FileDump.java:107)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> {noformat}
> hive --orcfiledump -j -p /tmp/orc/inventory/inv_date_sk=2452654
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12981) ThriftCLIService uses incompatible getShortName() implementation
[ https://issues.apache.org/jira/browse/HIVE-12981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147774#comment-15147774 ] Bolke de Bruin commented on HIVE-12981: --- [~sershe] So is there anything left to do for this patch? testCliDriver_ivyDownload is I guess also not due to this patch. > ThriftCLIService uses incompatible getShortName() implementation > > > Key: HIVE-12981 > URL: https://issues.apache.org/jira/browse/HIVE-12981 > Project: Hive > Issue Type: Bug > Components: Authentication, Authorization, CLI, Security >Affects Versions: 1.2.1, 2.1.0 >Reporter: Bolke de Bruin >Assignee: Thejas M Nair >Priority: Critical > Labels: kerberos > Attachments: 0001-HIVE-12981-Use-KerberosName.patch, > HIVE-12981-branch-1.2.patch, HIVE-12981.01.patch, HIVE-12981.patch > > > ThriftCLIService has a local implementation getShortName() that assumes a > short name is always the part before "@" and "/". This is not always the case > as Kerberos Rules (from Hadoop's KerberosName) might actually transform a > name to something else. > Considering a pending change to getShortName() (#HADOOP-12751) and the normal > use of KerberosName in other parts of Hive it only seems logical to use the > standard implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12994) Implement support for NULLS FIRST/NULLS LAST
[ https://issues.apache.org/jira/browse/HIVE-12994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-12994: --- Component/s: Metastore > Implement support for NULLS FIRST/NULLS LAST > > > Key: HIVE-12994 > URL: https://issues.apache.org/jira/browse/HIVE-12994 > Project: Hive > Issue Type: New Feature > Components: CBO, Metastore, Parser, Serializers/Deserializers >Affects Versions: 2.1.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-12994.01.patch, HIVE-12994.02.patch, > HIVE-12994.03.patch, HIVE-12994.04.patch, HIVE-12994.05.patch, > HIVE-12994.06.patch, HIVE-12994.06.patch, HIVE-12994.07.patch, > HIVE-12994.patch > > > From SQL:2003, the NULLS FIRST and NULLS LAST options can be used to > determine whether nulls appear before or after non-null data values when the > ORDER BY clause is used. > SQL standard does not specify the behavior by default. Currently in Hive, > null values sort as if lower than any non-null value; that is, NULLS FIRST is > the default for ASC order, and NULLS LAST for DESC order. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13062) Hive metastore test failing
[
https://issues.apache.org/jira/browse/HIVE-13062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147743#comment-15147743
]
Jesus Camacho Rodriguez commented on HIVE-13062:
[~sershe], could you review it? HIVE-12994 was triggering
PreCommit-HIVE-METASTORE-Test and I was hitting this error. Thanks
> Hive metastore test failing
> ---
>
> Key: HIVE-13062
> URL: https://issues.apache.org/jira/browse/HIVE-13062
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 2.0.0, 2.1.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-13062.patch
>
>
> The testutils/metastore/metastore-upgrade-test.sh script takes the list of
> scripts to execute for the DB upgrade from the folder itself
> ({{metastore/scripts/upgrade/DB_NAME}}). Currently it fails as it executes
> upgrade-1.2.0-to-1.3.0 followed by upgrade-1.2.0-to-2.0.0.
> Instead, it should use the content of the {{upgrade.order.DB_NAME}} file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13062) Hive metastore test failing
[
https://issues.apache.org/jira/browse/HIVE-13062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-13062:
---
Description:
The testutils/metastore/metastore-upgrade-test.sh script takes the list of
scripts to execute for the DB upgrade from the folder itself
({{metastore/scripts/upgrade/DB_NAME}}). Currently it fails as it executes
upgrade-1.2.0-to-1.3.0 followed by upgrade-1.2.0-to-2.0.0.
Instead, it should use the content of the {{upgrade.order.DB_NAME}} file.
was:
The testutils/metastore/metastore-upgrade-test.sh script takes the list of
scripts to execute for the DB upgrade from the folder itself
(metastore/scripts/upgrade/DB_NAME). Currently it fails as it executes
upgrade-1.2.0-to-1.3.0 followed by upgrade-1.2.0-to-2.0.0.
Instead, it should use the content of the upgrade.order.DB_NAME file.
> Hive metastore test failing
> ---
>
> Key: HIVE-13062
> URL: https://issues.apache.org/jira/browse/HIVE-13062
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 2.0.0, 2.1.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-13062.patch
>
>
> The testutils/metastore/metastore-upgrade-test.sh script takes the list of
> scripts to execute for the DB upgrade from the folder itself
> ({{metastore/scripts/upgrade/DB_NAME}}). Currently it fails as it executes
> upgrade-1.2.0-to-1.3.0 followed by upgrade-1.2.0-to-2.0.0.
> Instead, it should use the content of the {{upgrade.order.DB_NAME}} file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12994) Implement support for NULLS FIRST/NULLS LAST
[
https://issues.apache.org/jira/browse/HIVE-12994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147740#comment-15147740
]
Hive QA commented on HIVE-12994:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12787976/HIVE-12994.07.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 4 tests passed
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-METASTORE-Test/117/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-METASTORE-Test/117/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-METASTORE-Test-117/
Messages:
{noformat}
LXC derby found.
LXC derby is not started. Starting container...
Container started.
Preparing derby container...
Container prepared.
Calling /hive/testutils/metastore/dbs/derby/prepare.sh ...
Server prepared.
Calling /hive/testutils/metastore/dbs/derby/execute.sh ...
Tests executed.
LXC mysql found.
LXC mysql is not started. Starting container...
Container started.
Preparing mysql container...
Container prepared.
Calling /hive/testutils/metastore/dbs/mysql/prepare.sh ...
Server prepared.
Calling /hive/testutils/metastore/dbs/mysql/execute.sh ...
Tests executed.
LXC oracle found.
LXC oracle is not started. Starting container...
Container started.
Preparing oracle container...
Container prepared.
Calling /hive/testutils/metastore/dbs/oracle/prepare.sh ...
Server prepared.
Calling /hive/testutils/metastore/dbs/oracle/execute.sh ...
Tests executed.
LXC postgres found.
LXC postgres is not started. Starting container...
Container started.
Preparing postgres container...
Container prepared.
Calling /hive/testutils/metastore/dbs/postgres/prepare.sh ...
Server prepared.
Calling /hive/testutils/metastore/dbs/postgres/execute.sh ...
Tests executed.
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12787976 - PreCommit-HIVE-METASTORE-Test
> Implement support for NULLS FIRST/NULLS LAST
>
>
> Key: HIVE-12994
> URL: https://issues.apache.org/jira/browse/HIVE-12994
> Project: Hive
> Issue Type: New Feature
> Components: CBO, Parser, Serializers/Deserializers
>Affects Versions: 2.1.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-12994.01.patch, HIVE-12994.02.patch,
> HIVE-12994.03.patch, HIVE-12994.04.patch, HIVE-12994.05.patch,
> HIVE-12994.06.patch, HIVE-12994.06.patch, HIVE-12994.07.patch,
> HIVE-12994.patch
>
>
> From SQL:2003, the NULLS FIRST and NULLS LAST options can be used to
> determine whether nulls appear before or after non-null data values when the
> ORDER BY clause is used.
> SQL standard does not specify the behavior by default. Currently in Hive,
> null values sort as if lower than any non-null value; that is, NULLS FIRST is
> the default for ASC order, and NULLS LAST for DESC order.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13061) Advance Hadoop Architecture (AHA) - Advance Hive (Umbrella) JIRA -- Enhanced Updates & others
[ https://issues.apache.org/jira/browse/HIVE-13061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147734#comment-15147734 ] Dinesh S. Atreya commented on HIVE-13061: - Please review JIRA https://issues.apache.org/jira/browse/HDFS-9607 and provide your inputs/perspectives including additional requirements as that JIRA will serve as a basis for Advancing Hive as proposed in this JIRA. Kindly review ORC JIRA https://issues.apache.org/jira/browse/ORC-42 and provide your thoughts as well. > Advance Hadoop Architecture (AHA) - Advance Hive (Umbrella) JIRA -- Enhanced > Updates & others > - > > Key: HIVE-13061 > URL: https://issues.apache.org/jira/browse/HIVE-13061 > Project: Hive > Issue Type: New Feature >Reporter: Dinesh S. Atreya > > Link to Umbrella JIRA https://issues.apache.org/jira/browse/HADOOP-12620 > See > https://issues.apache.org/jira/browse/HADOOP-12620?focusedCommentId=15046300=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15046300 > for more details. > This JIRA is an umbrella (parent/master) JIRA for advancing Hive through > native Hive update capability given > https://issues.apache.org/jira/browse/HDFS-9607 and > https://issues.apache.org/jira/browse/ORC-42. > Provide seamless and streamlined enhancements to Hive capabilities that are > currently implemented through https://issues.apache.org/jira/browse/HIVE-5317 > and other related JIRAs. > Serve as a counterpart Umbrella JIRA to ORC JIRA > https://issues.apache.org/jira/browse/ORC-42, so that corresponding > capabilities can be supported through SQL interfaces in Hive. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13047) Disabling Web UI leads to NullPointerException
[
https://issues.apache.org/jira/browse/HIVE-13047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147733#comment-15147733
]
Sergey Shelukhin commented on HIVE-13047:
-
Failures are unrelated
> Disabling Web UI leads to NullPointerException
> --
>
> Key: HIVE-13047
> URL: https://issues.apache.org/jira/browse/HIVE-13047
> Project: Hive
> Issue Type: Bug
>Reporter: Mohit Sabharwal
>Assignee: Mohit Sabharwal
> Attachments: HIVE-13047.patch
>
>
> Disabling the Web UI or it's historical query display feature can lead to
> NullPointerException since {{historicSqlOperations}} is unintialized.
> For ex: If hive.server2.webui.port is set to 0
> {code}
> Caused by: java.lang.NullPointerException: null
> at
> org.apache.hive.service.cli.operation.OperationManager.removeOperation(OperationManager.java:212)
> at
> org.apache.hive.service.cli.operation.OperationManager.closeOperation(OperationManager.java:240)
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.closeOperation(HiveSessionImpl.java:727)
> at
> org.apache.hive.service.cli.CLIService.closeOperation(CLIService.java:408)
> at
> org.apache.hive.service.cli.thrift.ThriftCLIService.CloseOperation(ThriftCLIService.java:664)
> at
> org.apache.hive.service.cli.thrift.TCLIService$Processor$CloseOperation.getResult(TCLIService.java:1513)
> at
> org.apache.hive.service.cli.thrift.TCLIService$Processor$CloseOperation.getResult(TCLIService.java:1498)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-12994) Implement support for NULLS FIRST/NULLS LAST
[ https://issues.apache.org/jira/browse/HIVE-12994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-12994: --- Attachment: (was: HIVE-12994.07.patch) > Implement support for NULLS FIRST/NULLS LAST > > > Key: HIVE-12994 > URL: https://issues.apache.org/jira/browse/HIVE-12994 > Project: Hive > Issue Type: New Feature > Components: CBO, Parser, Serializers/Deserializers >Affects Versions: 2.1.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-12994.01.patch, HIVE-12994.02.patch, > HIVE-12994.03.patch, HIVE-12994.04.patch, HIVE-12994.05.patch, > HIVE-12994.06.patch, HIVE-12994.06.patch, HIVE-12994.07.patch, > HIVE-12994.patch > > > From SQL:2003, the NULLS FIRST and NULLS LAST options can be used to > determine whether nulls appear before or after non-null data values when the > ORDER BY clause is used. > SQL standard does not specify the behavior by default. Currently in Hive, > null values sort as if lower than any non-null value; that is, NULLS FIRST is > the default for ASC order, and NULLS LAST for DESC order. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12994) Implement support for NULLS FIRST/NULLS LAST
[ https://issues.apache.org/jira/browse/HIVE-12994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-12994: --- Attachment: HIVE-12994.07.patch > Implement support for NULLS FIRST/NULLS LAST > > > Key: HIVE-12994 > URL: https://issues.apache.org/jira/browse/HIVE-12994 > Project: Hive > Issue Type: New Feature > Components: CBO, Parser, Serializers/Deserializers >Affects Versions: 2.1.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-12994.01.patch, HIVE-12994.02.patch, > HIVE-12994.03.patch, HIVE-12994.04.patch, HIVE-12994.05.patch, > HIVE-12994.06.patch, HIVE-12994.06.patch, HIVE-12994.07.patch, > HIVE-12994.patch > > > From SQL:2003, the NULLS FIRST and NULLS LAST options can be used to > determine whether nulls appear before or after non-null data values when the > ORDER BY clause is used. > SQL standard does not specify the behavior by default. Currently in Hive, > null values sort as if lower than any non-null value; that is, NULLS FIRST is > the default for ASC order, and NULLS LAST for DESC order. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13062) Hive metastore test failing
[ https://issues.apache.org/jira/browse/HIVE-13062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-13062: --- Attachment: HIVE-13062.patch > Hive metastore test failing > --- > > Key: HIVE-13062 > URL: https://issues.apache.org/jira/browse/HIVE-13062 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 2.0.0, 2.1.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-13062.patch > > > The testutils/metastore/metastore-upgrade-test.sh script takes the list of > scripts to execute for the DB upgrade from the folder itself > (metastore/scripts/upgrade/DB_NAME). Currently it fails as it executes > upgrade-1.2.0-to-1.3.0 followed by upgrade-1.2.0-to-2.0.0. > Instead, it should use the content of the upgrade.order.DB_NAME file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13061) Advance Hadoop Architecture (AHA) - Advance Hive (Umbrella) JIRA -- Enhanced Updates & others
[ https://issues.apache.org/jira/browse/HIVE-13061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147721#comment-15147721 ] Dinesh S. Atreya commented on HIVE-13061: - Adding related JIRAS * HIVE-9675 * HIVE-6905 * HIVE-8244 > Advance Hadoop Architecture (AHA) - Advance Hive (Umbrella) JIRA -- Enhanced > Updates & others > - > > Key: HIVE-13061 > URL: https://issues.apache.org/jira/browse/HIVE-13061 > Project: Hive > Issue Type: New Feature >Reporter: Dinesh S. Atreya > > Link to Umbrella JIRA https://issues.apache.org/jira/browse/HADOOP-12620 > See > https://issues.apache.org/jira/browse/HADOOP-12620?focusedCommentId=15046300=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15046300 > for more details. > This JIRA is an umbrella (parent/master) JIRA for advancing Hive through > native Hive update capability given > https://issues.apache.org/jira/browse/HDFS-9607 and > https://issues.apache.org/jira/browse/ORC-42. > Provide seamless and streamlined enhancements to Hive capabilities that are > currently implemented through https://issues.apache.org/jira/browse/HIVE-5317 > and other related JIRAs. > Serve as a counterpart Umbrella JIRA to ORC JIRA > https://issues.apache.org/jira/browse/ORC-42, so that corresponding > capabilities can be supported through SQL interfaces in Hive. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13061) Advance Hadoop Architecture (AHA) - Advance Hive (Umbrella) JIRA -- Enhanced Updates & others
[ https://issues.apache.org/jira/browse/HIVE-13061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147717#comment-15147717 ] Dinesh S. Atreya commented on HIVE-13061: - This JIRA is dependent on both https://issues.apache.org/jira/browse/HDFS-9607 and https://issues.apache.org/jira/browse/ORC-42 > Advance Hadoop Architecture (AHA) - Advance Hive (Umbrella) JIRA -- Enhanced > Updates & others > - > > Key: HIVE-13061 > URL: https://issues.apache.org/jira/browse/HIVE-13061 > Project: Hive > Issue Type: New Feature >Reporter: Dinesh S. Atreya > > Link to Umbrella JIRA https://issues.apache.org/jira/browse/HADOOP-12620 > See > https://issues.apache.org/jira/browse/HADOOP-12620?focusedCommentId=15046300=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15046300 > for more details. > This JIRA is an umbrella (parent/master) JIRA for advancing Hive through > native Hive update capability given > https://issues.apache.org/jira/browse/HDFS-9607 and > https://issues.apache.org/jira/browse/ORC-42. > Provide seamless and streamlined enhancements to Hive capabilities that are > currently implemented through https://issues.apache.org/jira/browse/HIVE-5317 > and other related JIRAs. > Serve as a counterpart Umbrella JIRA to ORC JIRA > https://issues.apache.org/jira/browse/ORC-42, so that corresponding > capabilities can be supported through SQL interfaces in Hive. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13061) Advance Hadoop Architecture (AHA) - Advance Hive (Umbrella) JIRA -- Enhanced Updates & others
[ https://issues.apache.org/jira/browse/HIVE-13061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dinesh S. Atreya updated HIVE-13061: Description: Link to Umbrella JIRA https://issues.apache.org/jira/browse/HADOOP-12620 See https://issues.apache.org/jira/browse/HADOOP-12620?focusedCommentId=15046300=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15046300 for more details. This JIRA is an umbrella (parent/master) JIRA for advancing Hive through native Hive update capability given https://issues.apache.org/jira/browse/HDFS-9607 and https://issues.apache.org/jira/browse/ORC-42. Provide seamless and streamlined enhancements to Hive capabilities that are currently implemented through https://issues.apache.org/jira/browse/HIVE-5317 and other related JIRAs. Serve as a counterpart Umbrella JIRA to ORC JIRA https://issues.apache.org/jira/browse/ORC-42, so that corresponding capabilities can be supported through SQL interfaces in Hive. was: Link to Umbrella JIRA https://issues.apache.org/jira/browse/HADOOP-12620 See https://issues.apache.org/jira/browse/HADOOP-12620?focusedCommentId=15046300=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15046300 for more details. This JIRA is an umbrella (parent/master) JIRA for advancing Hive through native Hive update capability given https://issues.apache.org/jira/browse/HDFS-9607 and https://issues.apache.org/jira/browse/ORC-42. Provide seamless and streamlined enhancements to Hive capabilities that are currently implemented through https://issues.apache.org/jira/browse/HIVE-5317 and other related JIRAs. Serve as a counterpart Umbrella JIRA to ORC JIRA https://issues.apache.org/jira/browse/ORC-42, so that corresponding capabilities can be supported through SQL interfaces in Hive. > Advance Hadoop Architecture (AHA) - Advance Hive (Umbrella) JIRA -- Enhanced > Updates & others > - > > Key: HIVE-13061 > URL: https://issues.apache.org/jira/browse/HIVE-13061 > Project: Hive > Issue Type: New Feature >Reporter: Dinesh S. Atreya > > Link to Umbrella JIRA https://issues.apache.org/jira/browse/HADOOP-12620 > See > https://issues.apache.org/jira/browse/HADOOP-12620?focusedCommentId=15046300=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15046300 > for more details. > This JIRA is an umbrella (parent/master) JIRA for advancing Hive through > native Hive update capability given > https://issues.apache.org/jira/browse/HDFS-9607 and > https://issues.apache.org/jira/browse/ORC-42. > Provide seamless and streamlined enhancements to Hive capabilities that are > currently implemented through https://issues.apache.org/jira/browse/HIVE-5317 > and other related JIRAs. > Serve as a counterpart Umbrella JIRA to ORC JIRA > https://issues.apache.org/jira/browse/ORC-42, so that corresponding > capabilities can be supported through SQL interfaces in Hive. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12857) LLAP: modify the decider to allow using LLAP with whitelisted UDFs
[ https://issues.apache.org/jira/browse/HIVE-12857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147697#comment-15147697 ] Sergey Shelukhin commented on HIVE-12857: - Test failures are due to some other commit; not related. > LLAP: modify the decider to allow using LLAP with whitelisted UDFs > -- > > Key: HIVE-12857 > URL: https://issues.apache.org/jira/browse/HIVE-12857 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-12857.01.patch, HIVE-12857.02.patch, > HIVE-12857.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12981) ThriftCLIService uses incompatible getShortName() implementation
[ https://issues.apache.org/jira/browse/HIVE-12981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147696#comment-15147696 ] Sergey Shelukhin commented on HIVE-12981: - CTE tests, partition_coltype_literals, and the last two fail elsewhere (CTE should already be fixed). > ThriftCLIService uses incompatible getShortName() implementation > > > Key: HIVE-12981 > URL: https://issues.apache.org/jira/browse/HIVE-12981 > Project: Hive > Issue Type: Bug > Components: Authentication, Authorization, CLI, Security >Affects Versions: 1.2.1, 2.1.0 >Reporter: Bolke de Bruin >Assignee: Thejas M Nair >Priority: Critical > Labels: kerberos > Attachments: 0001-HIVE-12981-Use-KerberosName.patch, > HIVE-12981-branch-1.2.patch, HIVE-12981.01.patch, HIVE-12981.patch > > > ThriftCLIService has a local implementation getShortName() that assumes a > short name is always the part before "@" and "/". This is not always the case > as Kerberos Rules (from Hadoop's KerberosName) might actually transform a > name to something else. > Considering a pending change to getShortName() (#HADOOP-12751) and the normal > use of KerberosName in other parts of Hive it only seems logical to use the > standard implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13058) Add session and operation_log directory deletion messages
[
https://issues.apache.org/jira/browse/HIVE-13058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147675#comment-15147675
]
Hive QA commented on HIVE-13058:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12787855/HIVE-13058.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 22 failed/errored test(s), 9789 tests
executed
*Failed tests:*
{noformat}
TestSparkCliDriver-timestamp_lazy.q-bucketsortoptimize_insert_4.q-date_udf.q-and-12-more
- did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_coltype_literals
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.org.apache.hadoop.hive.cli.TestMiniTezCliDriver
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_auto_join_filters
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_auto_join_nulls
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_delete_tmp_table
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_empty_join
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_mapreduce1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_schema_evol_text_mapwork_table
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_temp_table
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_vector_dynpart_hashjoin_2
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_transform1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_decimal_round_2
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_decimal_udf
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_interval_2
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_string_concat
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorization_14
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorized_distinct_gby
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testSimpleTable
org.apache.hadoop.hive.ql.TestTxnCommands2.testInitiatorWithMultipleFailedCompactions
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6991/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6991/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6991/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 22 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12787855 - PreCommit-HIVE-TRUNK-Build
> Add session and operation_log directory deletion messages
> -
>
> Key: HIVE-13058
> URL: https://issues.apache.org/jira/browse/HIVE-13058
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Reporter: Shinichi Yamashita
>Assignee: Shinichi Yamashita
>Priority: Minor
> Attachments: HIVE-13058.1.patch
>
>
> In the current master, when connecting to HS2, we can confirm that some
> directories were created. For example the following messages of HS2 log.
> {code}
> 2016-02-14T13:57:52,412 INFO [HiveServer2-Handler-Pool: Thread-607]:
> session.SessionState (SessionState.java:createPath(693)) - Created HDFS
> directory: /tmp/hive/root/bfe28a51-5804-4cbb-952c-26709aed4652
> 2016-02-14T13:57:52,416 INFO [HiveServer2-Handler-Pool: Thread-607]:
> session.SessionState (SessionState.java:createPath(693)) - Created local
> directory: /tmp/hive/bfe28a51-5804-4cbb-952c-26709aed4652
> 2016-02-14T13:57:52,419 INFO [HiveServer2-Handler-Pool: Thread-607]:
> session.SessionState (SessionState.java:createPath(693)) - Created HDFS
> directory: /tmp/hive/root/bfe28a51-5804-4cbb-952c-26709aed4652/_tmp_space.db
> 2016-02-14T13:57:52,421 INFO [HiveServer2-Handler-Pool: Thread-607]:
> session.HiveSessionImpl (HiveSessionImpl.java:setOperationLogSessionDir(287))
> - Operation log session directory is created:
> /tmp/hive/operation_logs/bfe28a51-5804-4cbb-952c-26709aed4652
> {code}
> But when closing from HS2, we cannot confirm that these directories were
> deleted. So I change it so that some messages about these directories
> deletion may be output as follows.
> {code}
> 2016-02-14T13:57:56,877 INFO
>
[jira] [Commented] (HIVE-11752) Pre-materializing complex CTE queries
[ https://issues.apache.org/jira/browse/HIVE-11752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147670#comment-15147670 ] Jesus Camacho Rodriguez commented on HIVE-11752: [~sershe], I pushed an ADDENDUM a few hours ago to fix this issue. > Pre-materializing complex CTE queries > - > > Key: HIVE-11752 > URL: https://issues.apache.org/jira/browse/HIVE-11752 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 2.1.0 >Reporter: Navis >Assignee: Jesus Camacho Rodriguez >Priority: Minor > Labels: TODOC2.1 > Fix For: 2.1.0 > > Attachments: HIVE-11752.03.patch, HIVE-11752.04.patch, > HIVE-11752.05.patch, HIVE-11752.1.patch.txt, HIVE-11752.2.patch.txt > > > Currently, hive regards CTE clauses as a simple alias to the query block, > which makes redundant works if it's used multiple times in a query. This > introduces a reference threshold for pre-materializing the CTE clause as a > volatile table (which is not exists in any form of metastore and just > accessible from QB). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11752) Pre-materializing complex CTE queries
[ https://issues.apache.org/jira/browse/HIVE-11752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147668#comment-15147668 ] Sergey Shelukhin commented on HIVE-11752: - CTE tests are failing in all the JIRAs > Pre-materializing complex CTE queries > - > > Key: HIVE-11752 > URL: https://issues.apache.org/jira/browse/HIVE-11752 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 2.1.0 >Reporter: Navis >Assignee: Jesus Camacho Rodriguez >Priority: Minor > Labels: TODOC2.1 > Fix For: 2.1.0 > > Attachments: HIVE-11752.03.patch, HIVE-11752.04.patch, > HIVE-11752.05.patch, HIVE-11752.1.patch.txt, HIVE-11752.2.patch.txt > > > Currently, hive regards CTE clauses as a simple alias to the query block, > which makes redundant works if it's used multiple times in a query. This > introduces a reference threshold for pre-materializing the CTE clause as a > volatile table (which is not exists in any form of metastore and just > accessible from QB). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12981) ThriftCLIService uses incompatible getShortName() implementation
[ https://issues.apache.org/jira/browse/HIVE-12981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147666#comment-15147666 ] Sergey Shelukhin commented on HIVE-12981: - Cancel and resubmit is not actually necessary, it's ok to submit another patch without changing the status. > ThriftCLIService uses incompatible getShortName() implementation > > > Key: HIVE-12981 > URL: https://issues.apache.org/jira/browse/HIVE-12981 > Project: Hive > Issue Type: Bug > Components: Authentication, Authorization, CLI, Security >Affects Versions: 1.2.1, 2.1.0 >Reporter: Bolke de Bruin >Assignee: Thejas M Nair >Priority: Critical > Labels: kerberos > Attachments: 0001-HIVE-12981-Use-KerberosName.patch, > HIVE-12981-branch-1.2.patch, HIVE-12981.01.patch, HIVE-12981.patch > > > ThriftCLIService has a local implementation getShortName() that assumes a > short name is always the part before "@" and "/". This is not always the case > as Kerberos Rules (from Hadoop's KerberosName) might actually transform a > name to something else. > Considering a pending change to getShortName() (#HADOOP-12751) and the normal > use of KerberosName in other parts of Hive it only seems logical to use the > standard implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13015) Bundle Log4j2 jars with hive-exec
[
https://issues.apache.org/jira/browse/HIVE-13015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147664#comment-15147664
]
Sergey Shelukhin commented on HIVE-13015:
-
Looks like it broke a lot of tests...
> Bundle Log4j2 jars with hive-exec
> -
>
> Key: HIVE-13015
> URL: https://issues.apache.org/jira/browse/HIVE-13015
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: Prasanth Jayachandran
>Assignee: Gopal V
> Attachments: HIVE-13015.1.patch, HIVE-13015.1.patch
>
>
> In some of the recent test runs, we are seeing multiple bindings for SLF4j
> that causes issues with LOG4j2 logger.
> {code}
> SLF4J: Found binding in
> [jar:file:/grid/0/hadoop/yarn/local/usercache/hrt_qa/appcache/application_1454694331819_0001/container_e06_1454694331819_0001_01_02/app/install/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> {code}
> We have added explicit exclusions for slf4j-log4j12 but some library is
> pulling it transitively and it's getting packaged with hive libs. Also hive
> currently uses version 1.7.5 for slf4j. We should add dependency convergence
> for sl4fj and also remove packaging of slf4j-log4j12.*.jar
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-12994) Implement support for NULLS FIRST/NULLS LAST
[ https://issues.apache.org/jira/browse/HIVE-12994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-12994: --- Attachment: HIVE-12994.07.patch > Implement support for NULLS FIRST/NULLS LAST > > > Key: HIVE-12994 > URL: https://issues.apache.org/jira/browse/HIVE-12994 > Project: Hive > Issue Type: New Feature > Components: CBO, Parser, Serializers/Deserializers >Affects Versions: 2.1.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-12994.01.patch, HIVE-12994.02.patch, > HIVE-12994.03.patch, HIVE-12994.04.patch, HIVE-12994.05.patch, > HIVE-12994.06.patch, HIVE-12994.06.patch, HIVE-12994.07.patch, > HIVE-12994.patch > > > From SQL:2003, the NULLS FIRST and NULLS LAST options can be used to > determine whether nulls appear before or after non-null data values when the > ORDER BY clause is used. > SQL standard does not specify the behavior by default. Currently in Hive, > null values sort as if lower than any non-null value; that is, NULLS FIRST is > the default for ASC order, and NULLS LAST for DESC order. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12543) Disable Hive ConstantPropagate optimizer when CBO has optimized the plan
[ https://issues.apache.org/jira/browse/HIVE-12543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147609#comment-15147609 ] Jesus Camacho Rodriguez commented on HIVE-12543: [~ashutoshc], new patch addresses the following: - Dynamic partition pruning not being triggered anymore when static partition pruning has been performed. - testNegativeCliDriver_case_with_row_sequence fail : stateful functions are not allowed within CASE clauses. Previously, these were identified when we were running ConstantPropagate. With this patch, constant folding is done by Calcite, thus we need to identify them in the parser. - Sorted dynamic partition optimizer not being triggered : constant columns recognition in SortedDynPartitionOptimizer needed to be improved. Thus, we only need to create the follow-up JIRA for join merging. > Disable Hive ConstantPropagate optimizer when CBO has optimized the plan > > > Key: HIVE-12543 > URL: https://issues.apache.org/jira/browse/HIVE-12543 > Project: Hive > Issue Type: Sub-task > Components: CBO, Logical Optimizer >Affects Versions: 2.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-12543.01.patch, HIVE-12543.02.patch, > HIVE-12543.03.patch, HIVE-12543.04.patch, HIVE-12543.05.patch, > HIVE-12543.06.patch, HIVE-12543.07.patch, HIVE-12543.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12543) Disable Hive ConstantPropagate optimizer when CBO has optimized the plan
[ https://issues.apache.org/jira/browse/HIVE-12543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-12543: --- Attachment: (was: HIVE-12543.07.patch) > Disable Hive ConstantPropagate optimizer when CBO has optimized the plan > > > Key: HIVE-12543 > URL: https://issues.apache.org/jira/browse/HIVE-12543 > Project: Hive > Issue Type: Sub-task > Components: CBO, Logical Optimizer >Affects Versions: 2.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-12543.01.patch, HIVE-12543.02.patch, > HIVE-12543.03.patch, HIVE-12543.04.patch, HIVE-12543.05.patch, > HIVE-12543.06.patch, HIVE-12543.07.patch, HIVE-12543.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12543) Disable Hive ConstantPropagate optimizer when CBO has optimized the plan
[ https://issues.apache.org/jira/browse/HIVE-12543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-12543: --- Attachment: HIVE-12543.07.patch > Disable Hive ConstantPropagate optimizer when CBO has optimized the plan > > > Key: HIVE-12543 > URL: https://issues.apache.org/jira/browse/HIVE-12543 > Project: Hive > Issue Type: Sub-task > Components: CBO, Logical Optimizer >Affects Versions: 2.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-12543.01.patch, HIVE-12543.02.patch, > HIVE-12543.03.patch, HIVE-12543.04.patch, HIVE-12543.05.patch, > HIVE-12543.06.patch, HIVE-12543.07.patch, HIVE-12543.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13043) Reload function has no impact to function registry
[ https://issues.apache.org/jira/browse/HIVE-13043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-13043: --- Attachment: HIVE-13043.3.patch > Reload function has no impact to function registry > -- > > Key: HIVE-13043 > URL: https://issues.apache.org/jira/browse/HIVE-13043 > Project: Hive > Issue Type: Bug >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Attachments: HIVE-13043.1.patch, HIVE-13043.2.patch, > HIVE-13043.3.patch > > > With HIVE-2573, users should run "reload function" to refresh cached function > registry. However, "reload function" has no impact at all. We need to fix > this. Otherwise, HS2 needs to be restarted to see new global functions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13043) Reload function has no impact to function registry
[
https://issues.apache.org/jira/browse/HIVE-13043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147469#comment-15147469
]
Hive QA commented on HIVE-13043:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12787826/HIVE-13043.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 9788 tests executed
*Failed tests:*
{noformat}
TestSparkCliDriver-timestamp_lazy.q-bucketsortoptimize_insert_4.q-date_udf.q-and-12-more
- did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_coltype_literals
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hadoop.hive.ql.parse.authorization.plugin.sqlstd.TestOperation2Privilege.checkHiveOperationTypeMatch
org.apache.hive.jdbc.TestSSL.testSSLVersion
org.apache.hive.service.TestHS2ImpersonationWithRemoteMS.org.apache.hive.service.TestHS2ImpersonationWithRemoteMS
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6990/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6990/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6990/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12787826 - PreCommit-HIVE-TRUNK-Build
> Reload function has no impact to function registry
> --
>
> Key: HIVE-13043
> URL: https://issues.apache.org/jira/browse/HIVE-13043
> Project: Hive
> Issue Type: Bug
>Reporter: Jimmy Xiang
>Assignee: Jimmy Xiang
> Attachments: HIVE-13043.1.patch, HIVE-13043.2.patch
>
>
> With HIVE-2573, users should run "reload function" to refresh cached function
> registry. However, "reload function" has no impact at all. We need to fix
> this. Otherwise, HS2 needs to be restarted to see new global functions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-12543) Disable Hive ConstantPropagate optimizer when CBO has optimized the plan
[ https://issues.apache.org/jira/browse/HIVE-12543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-12543: --- Attachment: HIVE-12543.07.patch > Disable Hive ConstantPropagate optimizer when CBO has optimized the plan > > > Key: HIVE-12543 > URL: https://issues.apache.org/jira/browse/HIVE-12543 > Project: Hive > Issue Type: Sub-task > Components: CBO, Logical Optimizer >Affects Versions: 2.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-12543.01.patch, HIVE-12543.02.patch, > HIVE-12543.03.patch, HIVE-12543.04.patch, HIVE-12543.05.patch, > HIVE-12543.06.patch, HIVE-12543.07.patch, HIVE-12543.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12878) Support Vectorization for TEXTFILE and other formats
[ https://issues.apache.org/jira/browse/HIVE-12878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-12878: Attachment: HIVE-12878.03.patch > Support Vectorization for TEXTFILE and other formats > > > Key: HIVE-12878 > URL: https://issues.apache.org/jira/browse/HIVE-12878 > Project: Hive > Issue Type: New Feature > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-12878.01.patch, HIVE-12878.02.patch, > HIVE-12878.03.patch > > > Support vectorizing when the input format is TEXTFILE and other formats for > better Map Vertex performance. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13000) Hive returns useless parsing error
[ https://issues.apache.org/jira/browse/HIVE-13000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147328#comment-15147328 ] Alina Abramova commented on HIVE-13000: --- Have we any mention about this ticket? > Hive returns useless parsing error > --- > > Key: HIVE-13000 > URL: https://issues.apache.org/jira/browse/HIVE-13000 > Project: Hive > Issue Type: Bug >Affects Versions: 0.13.0, 1.0.0, 1.2.1 >Reporter: Alina Abramova >Assignee: Alina Abramova >Priority: Minor > Attachments: HIVE-13000.1.patch, HIVE-13000.2.patch, > HIVE-13000.3.patch > > > When I run query like these I receive unclear exception > hive> SELECT record FROM ctest GROUP BY record.instance_id; > FAILED: SemanticException Error in parsing > It will be clearer if it would be like: > hive> SELECT record FROM ctest GROUP BY record.instance_id; > FAILED: SemanticException Expression not in GROUP BY key record -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13057) Remove duplicate copies of TableDesc property values in PartitionDesc
[
https://issues.apache.org/jira/browse/HIVE-13057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147281#comment-15147281
]
Hive QA commented on HIVE-13057:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12787776/HIVE-13057.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 15 failed/errored test(s), 9803 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_mat_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_mat_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_mat_3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_mat_4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_mat_5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_coltype_literals
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_5
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_2
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_3
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_4
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_5
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6989/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6989/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6989/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 15 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12787776 - PreCommit-HIVE-TRUNK-Build
> Remove duplicate copies of TableDesc property values in PartitionDesc
> -
>
> Key: HIVE-13057
> URL: https://issues.apache.org/jira/browse/HIVE-13057
> Project: Hive
> Issue Type: Bug
>Reporter: Mohit Sabharwal
>Assignee: Mohit Sabharwal
> Attachments: HIVE-13057.patch
>
>
> For a partitioned table, each PartitionDesc has a copy of corresponding
> TableDesc.
> While TableDesc is mutable and hence cannot be interned, it's property values
> can be.
> For a simple select on a table with 100K partitions, this cut total number of
> String instances by ~65%.
> Most replicated strings were location, serde, input/output format, column,
> types, table name, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13015) Bundle Log4j2 jars with hive-exec
[
https://issues.apache.org/jira/browse/HIVE-13015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147185#comment-15147185
]
Hive QA commented on HIVE-13015:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12787756/HIVE-13015.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 601 failed/errored test(s), 9803 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_mat_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_mat_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_mat_3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_mat_4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_mat_5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_coltype_literals
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_5
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_2
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_3
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_4
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_5
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_add_part_multiple
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_alter_merge_orc
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_alter_merge_stats_orc
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_annotate_stats_join
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join0
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join1
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join10
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join11
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join12
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join13
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join14
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join15
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join16
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join17
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join18
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join18_multi_distinct
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join19
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join2
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join20
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join21
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join22
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join23
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join24
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join26
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join27
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join28
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join29
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join3
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join30
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join31
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join32
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join4
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join5
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join6
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join7
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join8
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join9
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join_filters
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join_nulls
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join_reordering_values
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join_stats
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join_stats2
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_join_without_localtask
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_smb_mapjoin_14
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_sortmerge_join_1
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_sortmerge_join_10
[jira] [Commented] (HIVE-13055) Add unit tests for HIVE-11512
[
https://issues.apache.org/jira/browse/HIVE-13055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147080#comment-15147080
]
Hive QA commented on HIVE-13055:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12787730/HIVE-13055.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 16 failed/errored test(s), 9796 tests
executed
*Failed tests:*
{noformat}
TestSparkCliDriver-timestamp_lazy.q-bucketsortoptimize_insert_4.q-date_udf.q-and-12-more
- did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_mat_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_mat_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_mat_3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_mat_4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_mat_5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_coltype_literals
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_5
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_2
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_3
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_4
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_5
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6987/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6987/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6987/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 16 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12787730 - PreCommit-HIVE-TRUNK-Build
> Add unit tests for HIVE-11512
> -
>
> Key: HIVE-13055
> URL: https://issues.apache.org/jira/browse/HIVE-13055
> Project: Hive
> Issue Type: Test
> Components: HiveServer2
>Affects Versions: 1.3.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
> Attachments: HIVE-13055.patch
>
>
> HIVE-11866 adds the ability to test LDAP authenticator with a LDAP Server. We
> need more test coverage for feature in HIVE-11512.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
