[jira] [Commented] (HIVE-14123) Add beeline configuration option to show database in the prompt
[
https://issues.apache.org/jira/browse/HIVE-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358361#comment-15358361
]
Hive QA commented on HIVE-14123:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12815451/HIVE-14123.3.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 10291 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_all
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_join
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testDelayedLocalityNodeCommErrorImmediateAllocation
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/332/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/332/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-332/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12815451 - PreCommit-HIVE-MASTER-Build
> Add beeline configuration option to show database in the prompt
> ---
>
> Key: HIVE-14123
> URL: https://issues.apache.org/jira/browse/HIVE-14123
> Project: Hive
> Issue Type: Improvement
> Components: Beeline, CLI
>Affects Versions: 2.2.0
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Minor
> Attachments: HIVE-14123.2.patch, HIVE-14123.3.patch, HIVE-14123.patch
>
>
> There are several jira issues complaining that, the Beeline does not respect
> hive.cli.print.current.db.
> This is partially true, since in embedded mode, it uses the
> hive.cli.print.current.db to change the prompt, since HIVE-10511.
> In remote mode, I think this function should use a beeline command line
> option instead, like for the showHeader option emphasizing, that this is a
> client side option.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13278) Many redundant 'File not found' messages appeared in container log during query execution with Hive on Spark
[
https://issues.apache.org/jira/browse/HIVE-13278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358317#comment-15358317
]
Rui Li commented on HIVE-13278:
---
Thanks [~stakiar] for the explanation. Then my question is can we avoid this on
higher level? Specifically, why we need to call getReduceWork for map-only job?
One possible reason is we call {{HiveOutputFormatImpl.checkOutputSpecs ->
Utilities.getMapRedWork}}. I think that can be fixed, at least for HoS, because
for HoS we shouldn't try to get both MapWork and ReduceWork from one JobConf.
> Many redundant 'File not found' messages appeared in container log during
> query execution with Hive on Spark
>
>
> Key: HIVE-13278
> URL: https://issues.apache.org/jira/browse/HIVE-13278
> Project: Hive
> Issue Type: Bug
> Environment: Hive on Spark engine
> Found based on :
> Apache Hive 2.0.0
> Apache Spark 1.6.0
>Reporter: Xin Hao
>Assignee: Sahil Takiar
>Priority: Minor
>
> Many redundant 'File not found' messages appeared in container log during
> query execution with Hive on Spark.
> Certainly, it doesn't prevent the query from running successfully. So mark it
> as Minor currently.
> Error message example:
> {noformat}
> 16/03/14 01:45:06 INFO exec.Utilities: File not found: File does not exist:
> /tmp/hive/hadoop/2d378538-f5d3-493c-9276-c62dd6634fb4/hive_2016-03-14_01-44-16_835_623058724409492515-6/-mr-10010/0a6d0cae-1eb3-448c-883b-590b3b198a73/reduce.xml
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:66)
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:56)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsUpdateTimes(FSNamesystem.java:1932)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:1873)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1853)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1825)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:565)
> at
> org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.getBlockLocations(AuthorizationProviderProxyClientProtocol.java:87)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:363)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1060)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2086)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2082)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2080)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14032) INSERT OVERWRITE command failed with case sensitive partition key names
[
https://issues.apache.org/jira/browse/HIVE-14032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358259#comment-15358259
]
Hive QA commented on HIVE-14032:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12815419/HIVE-14032.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10287 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_all
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_join
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testDelayedLocalityNodeCommErrorImmediateAllocation
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/331/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/331/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-331/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12815419 - PreCommit-HIVE-MASTER-Build
> INSERT OVERWRITE command failed with case sensitive partition key names
> ---
>
> Key: HIVE-14032
> URL: https://issues.apache.org/jira/browse/HIVE-14032
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 2.0.1
>Reporter: Chinna Rao Lalam
>Assignee: Chinna Rao Lalam
> Attachments: HIVE-14032.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14139) NPE dropping permanent function
[
https://issues.apache.org/jira/browse/HIVE-14139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358251#comment-15358251
]
Rui Li commented on HIVE-14139:
---
The problem is that we failed to register the function with
{{Hive.registerAllFunctionsOnce -> reloadFunctions}}. When we call
{{FunctionRegistry.registerPermanentFunction}}, we set registerToSession to
false so that the custom jar won't be downloaded and we end up with a
{{ClassNotFoundException}}. Therefore we get the NPE when we try to drop the
function.
[~sershe] - I found this with latest master code. BTW do you know why we choose
not to register the function to session when calling
{{Hive.registerAllFunctionsOnce}}?
> NPE dropping permanent function
> ---
>
> Key: HIVE-14139
> URL: https://issues.apache.org/jira/browse/HIVE-14139
> Project: Hive
> Issue Type: Bug
>Reporter: Rui Li
>Assignee: Rui Li
>
> To reproduce:
> 1. Start a CLI session and create a permanent function.
> 2. Exit current CLI session.
> 3. Start a new CLI session and drop the function.
> Stack trace:
> {noformat}
> FAILED: error during drop function: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.Registry.removePersistentFunctionUnderLock(Registry.java:513)
> at
> org.apache.hadoop.hive.ql.exec.Registry.unregisterFunction(Registry.java:501)
> at
> org.apache.hadoop.hive.ql.exec.FunctionRegistry.unregisterPermanentFunction(FunctionRegistry.java:1532)
> at
> org.apache.hadoop.hive.ql.exec.FunctionTask.dropPermanentFunction(FunctionTask.java:228)
> at
> org.apache.hadoop.hive.ql.exec.FunctionTask.execute(FunctionTask.java:95)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1860)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1564)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1316)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1085)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1073)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:232)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14114) Ensure RecordWriter in streaming API is using the same UserGroupInformation as StreamingConnection
[ https://issues.apache.org/jira/browse/HIVE-14114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-14114: -- Status: Patch Available (was: Open) > Ensure RecordWriter in streaming API is using the same UserGroupInformation > as StreamingConnection > -- > > Key: HIVE-14114 > URL: https://issues.apache.org/jira/browse/HIVE-14114 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-14114.patch > > > currently both DelimitedInputWriter and StrictJsonWriter perform some > Metastore access operations but without using UGI created by the caller for > Metastore operations made by matching StreamingConnection & TransactionBatch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-7224) Set incremental printing to true by default in Beeline
[ https://issues.apache.org/jira/browse/HIVE-7224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358180#comment-15358180 ] Sahil Takiar commented on HIVE-7224: None of these test failures seem related to the given change. All Beeline tests are passing. The new patch I attached is a rebased version of the previous patch. Is there anything else blocking this from being merged? > Set incremental printing to true by default in Beeline > -- > > Key: HIVE-7224 > URL: https://issues.apache.org/jira/browse/HIVE-7224 > Project: Hive > Issue Type: Bug > Components: Clients, JDBC >Affects Versions: 0.13.0, 1.0.0, 1.2.0, 1.1.0 >Reporter: Vaibhav Gumashta >Assignee: Sahil Takiar > Attachments: HIVE-7224.1.patch, HIVE-7224.2.patch, HIVE-7224.2.patch, > HIVE-7224.3.patch > > > See HIVE-7221. > By default beeline tries to buffer the entire output relation before printing > it on stdout. This can cause OOM when the output relation is large. However, > beeline has the option of incremental prints. We should keep that as the > default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-7224) Set incremental printing to true by default in Beeline
[
https://issues.apache.org/jira/browse/HIVE-7224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358108#comment-15358108
]
Hive QA commented on HIVE-7224:
---
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12813155/HIVE-7224.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 23 failed/errored test(s), 10288 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_all
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_join
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.org.apache.hadoop.hive.cli.TestMiniTezCliDriver
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_auto_join1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_auto_sortmerge_join_10
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_cbo_simple_select
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_deleteAnalyze
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_metadataonly1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_schema_evol_text_vec_mapwork_part_all_complex
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_transform1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_complex_join
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_decimal_udf
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_groupby6
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_string_concat
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_varchar_mapjoin1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorization_17
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorization_limit
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorized_math_funcs
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testDelayedLocalityNodeCommErrorImmediateAllocation
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/330/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/330/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-330/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 23 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12813155 - PreCommit-HIVE-MASTER-Build
> Set incremental printing to true by default in Beeline
> --
>
> Key: HIVE-7224
> URL: https://issues.apache.org/jira/browse/HIVE-7224
> Project: Hive
> Issue Type: Bug
> Components: Clients, JDBC
>Affects Versions: 0.13.0, 1.0.0, 1.2.0, 1.1.0
>Reporter: Vaibhav Gumashta
>Assignee: Sahil Takiar
> Attachments: HIVE-7224.1.patch, HIVE-7224.2.patch, HIVE-7224.2.patch,
> HIVE-7224.3.patch
>
>
> See HIVE-7221.
> By default beeline tries to buffer the entire output relation before printing
> it on stdout. This can cause OOM when the output relation is large. However,
> beeline has the option of incremental prints. We should keep that as the
> default.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14090) JDOExceptions thrown by the Metastore have their full stack trace returned to clients
[
https://issues.apache.org/jira/browse/HIVE-14090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358102#comment-15358102
]
Sahil Takiar commented on HIVE-14090:
-
Test failures seem un-related.
> JDOExceptions thrown by the Metastore have their full stack trace returned to
> clients
> -
>
> Key: HIVE-14090
> URL: https://issues.apache.org/jira/browse/HIVE-14090
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.1.0, 2.1.0
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
> Attachments: HIVE-14090.1.patch, HIVE-14090.patch
>
>
> When user try to create any database or table with a name longer than 128
> characters:
> {code}
> create database
> test_longname_looonglooonglooonglooonglooonglooonglooonglooonglooonglooonglooongNametableFAIL;
> {code}
> It dumps the full exception stack-trace in a non-user-friendly message. The
> lends to relatively negative user-experience for Beeline users who hit this
> exception, they are generally not interested in the full stack-trace.
> The formatted stack-trace is below:
> {code}
> Error while processing statement: FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask.
> MetaException(message:javax.jdo.JDOFatalUserException: Attempt to store value
> "test_longname_looonglooonglooonglooonglooonglooonglooonglooonglooonglooonglooongnametablefail2"
> in column "`NAME`" that has maximum length of 128. Please correct your data!
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:528)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:732)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:752)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.createDatabase(ObjectStore.java:569)
> at sun.reflect.GeneratedMethodAccessor31.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)
> at com.sun.proxy.$Proxy10.createDatabase(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_database_core(HiveMetaStore.java:923)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_database(HiveMetaStore.java:962)
> at sun.reflect.GeneratedMethodAccessor30.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:138)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:99)
> at com.sun.proxy.$Proxy12.create_database(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_database.getResult(ThriftHiveMetastore.java:8863)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_database.getResult(ThriftHiveMetastore.java:8847)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:707)
> at
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:702)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
> at
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:702)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745) NestedThrowablesStackTrace: Attempt
> to store value
> "test_longname_looonglooonglooonglooonglooonglooonglooonglooonglooonglooonglooongnametablefail2"
> in column "`NAME`" that has maximum length of 128. Please correct your data!
> org.datanucleus.exceptions.NucleusUserException: Attempt to store value
> "test_longname_looonglooonglooonglooonglooonglooonglooongloo
[jira] [Commented] (HIVE-14137) Hive on Spark throws FileAlreadyExistsException for jobs with multiple empty tables
[
https://issues.apache.org/jira/browse/HIVE-14137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358042#comment-15358042
]
Sahil Takiar commented on HIVE-14137:
-
Updated diff and RB to use a UUID instead of the alias in the constructed
{{Path}}.
> Hive on Spark throws FileAlreadyExistsException for jobs with multiple empty
> tables
> ---
>
> Key: HIVE-14137
> URL: https://issues.apache.org/jira/browse/HIVE-14137
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
> Attachments: HIVE-14137.1.patch, HIVE-14137.2.patch, HIVE-14137.patch
>
>
> The following queries:
> {code}
> -- Setup
> drop table if exists empty1;
> create table empty1 (col1 bigint) stored as parquet tblproperties
> ('parquet.compress'='snappy');
> drop table if exists empty2;
> create table empty2 (col1 bigint, col2 bigint) stored as parquet
> tblproperties ('parquet.compress'='snappy');
> drop table if exists empty3;
> create table empty3 (col1 bigint) stored as parquet tblproperties
> ('parquet.compress'='snappy');
> -- All empty HDFS directories.
> -- Fails with [08S01]: Error while processing statement: FAILED: Execution
> Error, return code 3 from org.apache.hadoop.hive.ql.exec.spark.SparkTask.
> select empty1.col1
> from empty1
> inner join empty2
> on empty2.col1 = empty1.col1
> inner join empty3
> on empty3.col1 = empty2.col2;
> -- Two empty HDFS directories.
> -- Create an empty file in HDFS.
> insert into empty1 select * from empty1 where false;
> -- Same query fails with [08S01]: Error while processing statement: FAILED:
> Execution Error, return code 3 from
> org.apache.hadoop.hive.ql.exec.spark.SparkTask.
> select empty1.col1
> from empty1
> inner join empty2
> on empty2.col1 = empty1.col1
> inner join empty3
> on empty3.col1 = empty2.col2;
> -- One empty HDFS directory.
> -- Create an empty file in HDFS.
> insert into empty2 select * from empty2 where false;
> -- Same query succeeds.
> select empty1.col1
> from empty1
> inner join empty2
> on empty2.col1 = empty1.col1
> inner join empty3
> on empty3.col1 = empty2.col2;
> {code}
> Will result in the following exception:
> {code}
> org.apache.hadoop.fs.FileAlreadyExistsException:
> /tmp/hive/hive/1f3837aa-9407-4780-92b1-42a66d205139/hive_2016-06-24_15-45-23_206_79177714958655528-2/-mr-10004/0/emptyFile
> for client 172.26.14.151 already exists
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:2784)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInt(FSNamesystem.java:2676)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:2561)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.create(NameNodeRpcServer.java:593)
> at
> org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.create(AuthorizationProviderProxyClientProtocol.java:111)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.create(ClientNamenodeProtocolServerSideTranslatorPB.java:393)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2086)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2082)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2080)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at
> org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
> at
> org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)
> at
> org.apache.hadoop.hdfs.DFSOutputStream.newStreamForCreate(DFSOutputStream.java:1902)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1738)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1663)
> at
> org.apache.hadoop.hdfs.
[jira] [Updated] (HIVE-14136) LLAP ZK SecretManager should resolve _HOST in principal
[ https://issues.apache.org/jira/browse/HIVE-14136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-14136: Resolution: Fixed Fix Version/s: 2.1.1 2.2.0 Status: Resolved (was: Patch Available) Committed to relevant branches; thanks for the review! > LLAP ZK SecretManager should resolve _HOST in principal > --- > > Key: HIVE-14136 > URL: https://issues.apache.org/jira/browse/HIVE-14136 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 2.2.0, 2.1.1 > > Attachments: HIVE-14136.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14137) Hive on Spark throws FileAlreadyExistsException for jobs with multiple empty tables
[
https://issues.apache.org/jira/browse/HIVE-14137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar updated HIVE-14137:
Attachment: HIVE-14137.2.patch
> Hive on Spark throws FileAlreadyExistsException for jobs with multiple empty
> tables
> ---
>
> Key: HIVE-14137
> URL: https://issues.apache.org/jira/browse/HIVE-14137
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
> Attachments: HIVE-14137.1.patch, HIVE-14137.2.patch, HIVE-14137.patch
>
>
> The following queries:
> {code}
> -- Setup
> drop table if exists empty1;
> create table empty1 (col1 bigint) stored as parquet tblproperties
> ('parquet.compress'='snappy');
> drop table if exists empty2;
> create table empty2 (col1 bigint, col2 bigint) stored as parquet
> tblproperties ('parquet.compress'='snappy');
> drop table if exists empty3;
> create table empty3 (col1 bigint) stored as parquet tblproperties
> ('parquet.compress'='snappy');
> -- All empty HDFS directories.
> -- Fails with [08S01]: Error while processing statement: FAILED: Execution
> Error, return code 3 from org.apache.hadoop.hive.ql.exec.spark.SparkTask.
> select empty1.col1
> from empty1
> inner join empty2
> on empty2.col1 = empty1.col1
> inner join empty3
> on empty3.col1 = empty2.col2;
> -- Two empty HDFS directories.
> -- Create an empty file in HDFS.
> insert into empty1 select * from empty1 where false;
> -- Same query fails with [08S01]: Error while processing statement: FAILED:
> Execution Error, return code 3 from
> org.apache.hadoop.hive.ql.exec.spark.SparkTask.
> select empty1.col1
> from empty1
> inner join empty2
> on empty2.col1 = empty1.col1
> inner join empty3
> on empty3.col1 = empty2.col2;
> -- One empty HDFS directory.
> -- Create an empty file in HDFS.
> insert into empty2 select * from empty2 where false;
> -- Same query succeeds.
> select empty1.col1
> from empty1
> inner join empty2
> on empty2.col1 = empty1.col1
> inner join empty3
> on empty3.col1 = empty2.col2;
> {code}
> Will result in the following exception:
> {code}
> org.apache.hadoop.fs.FileAlreadyExistsException:
> /tmp/hive/hive/1f3837aa-9407-4780-92b1-42a66d205139/hive_2016-06-24_15-45-23_206_79177714958655528-2/-mr-10004/0/emptyFile
> for client 172.26.14.151 already exists
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:2784)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInt(FSNamesystem.java:2676)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:2561)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.create(NameNodeRpcServer.java:593)
> at
> org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.create(AuthorizationProviderProxyClientProtocol.java:111)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.create(ClientNamenodeProtocolServerSideTranslatorPB.java:393)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2086)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2082)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2080)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at
> org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
> at
> org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)
> at
> org.apache.hadoop.hdfs.DFSOutputStream.newStreamForCreate(DFSOutputStream.java:1902)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1738)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1663)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$6.doCall(DistributedFileSystem.java:405)
> at
> org.apache.hadoop.hdfs.Distribu
[jira] [Commented] (HIVE-14136) LLAP ZK SecretManager should resolve _HOST in principal
[ https://issues.apache.org/jira/browse/HIVE-14136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358004#comment-15358004 ] Siddharth Seth commented on HIVE-14136: --- +1 > LLAP ZK SecretManager should resolve _HOST in principal > --- > > Key: HIVE-14136 > URL: https://issues.apache.org/jira/browse/HIVE-14136 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-14136.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14111) better concurrency handling for TezSessionState - part I
[ https://issues.apache.org/jira/browse/HIVE-14111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357992#comment-15357992 ] Sergey Shelukhin commented on HIVE-14111: - [~sseth] I don't think that's a problem per current code... some other code reuses the object on failure too. So, the session will be closed and nulled out but the same object could be reused and would be reopened later in a different TezTask, right? I added a comment suggesting that we remove that pattern and only use the object once, we can do that in a follow-up > better concurrency handling for TezSessionState - part I > > > Key: HIVE-14111 > URL: https://issues.apache.org/jira/browse/HIVE-14111 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-14111.01.patch, HIVE-14111.02.patch, > HIVE-14111.03.patch, HIVE-14111.patch, sessionPoolNotes.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14035) Enable predicate pushdown to delta files created by ACID Transactions
[ https://issues.apache.org/jira/browse/HIVE-14035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saket Saurabh updated HIVE-14035: - Attachment: HIVE-14035.05.patch Fix failing test cases for Streaming Ingest and Acid stats that now also have 'transactional_properties' as one of the table parameters. > Enable predicate pushdown to delta files created by ACID Transactions > - > > Key: HIVE-14035 > URL: https://issues.apache.org/jira/browse/HIVE-14035 > Project: Hive > Issue Type: New Feature > Components: Transactions >Reporter: Saket Saurabh >Assignee: Saket Saurabh > Attachments: HIVE-14035.02.patch, HIVE-14035.03.patch, > HIVE-14035.04.patch, HIVE-14035.05.patch, HIVE-14035.patch > > > In current Hive version, delta files created by ACID transactions do not > allow predicate pushdown if they contain any update/delete events. This is > done to preserve correctness when following a multi-version approach during > event collapsing, where an update event overwrites an existing insert event. > This JIRA proposes to split an update event into a combination of a delete > event followed by a new insert event, that can enable predicate push down to > all delta files without breaking correctness. To support backward > compatibility for this feature, this JIRA also proposes to add some sort of > versioning to ACID that can allow different versions of ACID transactions to > co-exist together. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14035) Enable predicate pushdown to delta files created by ACID Transactions
[ https://issues.apache.org/jira/browse/HIVE-14035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saket Saurabh updated HIVE-14035: - Status: Patch Available (was: In Progress) > Enable predicate pushdown to delta files created by ACID Transactions > - > > Key: HIVE-14035 > URL: https://issues.apache.org/jira/browse/HIVE-14035 > Project: Hive > Issue Type: New Feature > Components: Transactions >Reporter: Saket Saurabh >Assignee: Saket Saurabh > Attachments: HIVE-14035.02.patch, HIVE-14035.03.patch, > HIVE-14035.04.patch, HIVE-14035.05.patch, HIVE-14035.patch > > > In current Hive version, delta files created by ACID transactions do not > allow predicate pushdown if they contain any update/delete events. This is > done to preserve correctness when following a multi-version approach during > event collapsing, where an update event overwrites an existing insert event. > This JIRA proposes to split an update event into a combination of a delete > event followed by a new insert event, that can enable predicate push down to > all delta files without breaking correctness. To support backward > compatibility for this feature, this JIRA also proposes to add some sort of > versioning to ACID that can allow different versions of ACID transactions to > co-exist together. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14035) Enable predicate pushdown to delta files created by ACID Transactions
[ https://issues.apache.org/jira/browse/HIVE-14035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saket Saurabh updated HIVE-14035: - Status: In Progress (was: Patch Available) > Enable predicate pushdown to delta files created by ACID Transactions > - > > Key: HIVE-14035 > URL: https://issues.apache.org/jira/browse/HIVE-14035 > Project: Hive > Issue Type: New Feature > Components: Transactions >Reporter: Saket Saurabh >Assignee: Saket Saurabh > Attachments: HIVE-14035.02.patch, HIVE-14035.03.patch, > HIVE-14035.04.patch, HIVE-14035.patch > > > In current Hive version, delta files created by ACID transactions do not > allow predicate pushdown if they contain any update/delete events. This is > done to preserve correctness when following a multi-version approach during > event collapsing, where an update event overwrites an existing insert event. > This JIRA proposes to split an update event into a combination of a delete > event followed by a new insert event, that can enable predicate push down to > all delta files without breaking correctness. To support backward > compatibility for this feature, this JIRA also proposes to add some sort of > versioning to ACID that can allow different versions of ACID transactions to > co-exist together. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14111) better concurrency handling for TezSessionState - part I
[ https://issues.apache.org/jira/browse/HIVE-14111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-14111: Attachment: HIVE-14111.03.patch Nice test for the patch in QTestUtil... > better concurrency handling for TezSessionState - part I > > > Key: HIVE-14111 > URL: https://issues.apache.org/jira/browse/HIVE-14111 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-14111.01.patch, HIVE-14111.02.patch, > HIVE-14111.03.patch, HIVE-14111.patch, sessionPoolNotes.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14090) JDOExceptions thrown by the Metastore have their full stack trace returned to clients
[
https://issues.apache.org/jira/browse/HIVE-14090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357979#comment-15357979
]
Hive QA commented on HIVE-14090:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12815279/HIVE-14090.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 10287 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_all
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_join
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testDelayedLocalityNodeCommErrorImmediateAllocation
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/329/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/329/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-329/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12815279 - PreCommit-HIVE-MASTER-Build
> JDOExceptions thrown by the Metastore have their full stack trace returned to
> clients
> -
>
> Key: HIVE-14090
> URL: https://issues.apache.org/jira/browse/HIVE-14090
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.1.0, 2.1.0
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
> Attachments: HIVE-14090.1.patch, HIVE-14090.patch
>
>
> When user try to create any database or table with a name longer than 128
> characters:
> {code}
> create database
> test_longname_looonglooonglooonglooonglooonglooonglooonglooonglooonglooonglooongNametableFAIL;
> {code}
> It dumps the full exception stack-trace in a non-user-friendly message. The
> lends to relatively negative user-experience for Beeline users who hit this
> exception, they are generally not interested in the full stack-trace.
> The formatted stack-trace is below:
> {code}
> Error while processing statement: FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask.
> MetaException(message:javax.jdo.JDOFatalUserException: Attempt to store value
> "test_longname_looonglooonglooonglooonglooonglooonglooonglooonglooonglooonglooongnametablefail2"
> in column "`NAME`" that has maximum length of 128. Please correct your data!
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:528)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:732)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:752)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.createDatabase(ObjectStore.java:569)
> at sun.reflect.GeneratedMethodAccessor31.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)
> at com.sun.proxy.$Proxy10.createDatabase(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_database_core(HiveMetaStore.java:923)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_database(HiveMetaStore.java:962)
> at sun.reflect.GeneratedMethodAccessor30.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:138)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:99)
> at com.sun.proxy.$Proxy12.create_database(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_database.getResult(ThriftHiveMetastore.java:8863)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_database.getResult(ThriftHiveMetastore.java:8847)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TB
[jira] [Commented] (HIVE-14139) NPE dropping permanent function
[
https://issues.apache.org/jira/browse/HIVE-14139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357977#comment-15357977
]
Sergey Shelukhin commented on HIVE-14139:
-
Which version is this, btw?
> NPE dropping permanent function
> ---
>
> Key: HIVE-14139
> URL: https://issues.apache.org/jira/browse/HIVE-14139
> Project: Hive
> Issue Type: Bug
>Reporter: Rui Li
>Assignee: Rui Li
>
> To reproduce:
> 1. Start a CLI session and create a permanent function.
> 2. Exit current CLI session.
> 3. Start a new CLI session and drop the function.
> Stack trace:
> {noformat}
> FAILED: error during drop function: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.Registry.removePersistentFunctionUnderLock(Registry.java:513)
> at
> org.apache.hadoop.hive.ql.exec.Registry.unregisterFunction(Registry.java:501)
> at
> org.apache.hadoop.hive.ql.exec.FunctionRegistry.unregisterPermanentFunction(FunctionRegistry.java:1532)
> at
> org.apache.hadoop.hive.ql.exec.FunctionTask.dropPermanentFunction(FunctionTask.java:228)
> at
> org.apache.hadoop.hive.ql.exec.FunctionTask.execute(FunctionTask.java:95)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1860)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1564)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1316)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1085)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1073)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:232)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13884) Disallow queries in HMS fetching more than a configured number of partitions
[
https://issues.apache.org/jira/browse/HIVE-13884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Peña updated HIVE-13884:
---
Attachment: HIVE-13884.10.patch
Here's another patch the fixes the tests, and it avoids that the configuration
variable is freely modified by the client. Only the HMS server admins should be
able to do so from the HMS hive-site.xml.
> Disallow queries in HMS fetching more than a configured number of partitions
>
>
> Key: HIVE-13884
> URL: https://issues.apache.org/jira/browse/HIVE-13884
> Project: Hive
> Issue Type: Improvement
>Reporter: Mohit Sabharwal
>Assignee: Sergio Peña
> Attachments: HIVE-13884.1.patch, HIVE-13884.10.patch,
> HIVE-13884.2.patch, HIVE-13884.3.patch, HIVE-13884.4.patch,
> HIVE-13884.5.patch, HIVE-13884.6.patch, HIVE-13884.7.patch,
> HIVE-13884.8.patch, HIVE-13884.9.patch
>
>
> Currently the PartitionPruner requests either all partitions or partitions
> based on filter expression. In either scenarios, if the number of partitions
> accessed is large there can be significant memory pressure at the HMS server
> end.
> We already have a config {{hive.limit.query.max.table.partition}} that
> enforces limits on number of partitions that may be scanned per operator. But
> this check happens after the PartitionPruner has already fetched all
> partitions.
> We should add an option at PartitionPruner level to disallow queries that
> attempt to access number of partitions beyond a configurable limit.
> Note that {{hive.mapred.mode=strict}} disallow queries without a partition
> filter in PartitionPruner, but this check accepts any query with a pruning
> condition, even if partitions fetched are large. In multi-tenant
> environments, admins could use more control w.r.t. number of partitions
> allowed based on HMS memory capacity.
> One option is to have PartitionPruner first fetch the partition names
> (instead of partition specs) and throw an exception if number of partitions
> exceeds the configured value. Otherwise, fetch the partition specs.
> Looks like the existing {{listPartitionNames}} call could be used if extended
> to take partition filter expressions like {{getPartitionsByExpr}} call does.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14137) Hive on Spark throws FileAlreadyExistsException for jobs with multiple empty tables
[
https://issues.apache.org/jira/browse/HIVE-14137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357933#comment-15357933
]
Sahil Takiar commented on HIVE-14137:
-
It seems there are a bunch of test failures occurring because alias names may
have special characters in them such as ":", this causes problems since we are
putting the alias name in a {{Path}} object. This eventually causes exceptions
like {{java.net.URISyntaxException: Relative path in absolute URI:
null-subquery1:$hdt$_0-subquery1:$hdt$_0:x}} when trying to perform on ls on
the {{Path}} object.
Instead of putting the alias in the {{Path}} I am just going to place a UUID in
the {{Path}} instead.
> Hive on Spark throws FileAlreadyExistsException for jobs with multiple empty
> tables
> ---
>
> Key: HIVE-14137
> URL: https://issues.apache.org/jira/browse/HIVE-14137
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
> Attachments: HIVE-14137.1.patch, HIVE-14137.patch
>
>
> The following queries:
> {code}
> -- Setup
> drop table if exists empty1;
> create table empty1 (col1 bigint) stored as parquet tblproperties
> ('parquet.compress'='snappy');
> drop table if exists empty2;
> create table empty2 (col1 bigint, col2 bigint) stored as parquet
> tblproperties ('parquet.compress'='snappy');
> drop table if exists empty3;
> create table empty3 (col1 bigint) stored as parquet tblproperties
> ('parquet.compress'='snappy');
> -- All empty HDFS directories.
> -- Fails with [08S01]: Error while processing statement: FAILED: Execution
> Error, return code 3 from org.apache.hadoop.hive.ql.exec.spark.SparkTask.
> select empty1.col1
> from empty1
> inner join empty2
> on empty2.col1 = empty1.col1
> inner join empty3
> on empty3.col1 = empty2.col2;
> -- Two empty HDFS directories.
> -- Create an empty file in HDFS.
> insert into empty1 select * from empty1 where false;
> -- Same query fails with [08S01]: Error while processing statement: FAILED:
> Execution Error, return code 3 from
> org.apache.hadoop.hive.ql.exec.spark.SparkTask.
> select empty1.col1
> from empty1
> inner join empty2
> on empty2.col1 = empty1.col1
> inner join empty3
> on empty3.col1 = empty2.col2;
> -- One empty HDFS directory.
> -- Create an empty file in HDFS.
> insert into empty2 select * from empty2 where false;
> -- Same query succeeds.
> select empty1.col1
> from empty1
> inner join empty2
> on empty2.col1 = empty1.col1
> inner join empty3
> on empty3.col1 = empty2.col2;
> {code}
> Will result in the following exception:
> {code}
> org.apache.hadoop.fs.FileAlreadyExistsException:
> /tmp/hive/hive/1f3837aa-9407-4780-92b1-42a66d205139/hive_2016-06-24_15-45-23_206_79177714958655528-2/-mr-10004/0/emptyFile
> for client 172.26.14.151 already exists
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:2784)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInt(FSNamesystem.java:2676)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:2561)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.create(NameNodeRpcServer.java:593)
> at
> org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.create(AuthorizationProviderProxyClientProtocol.java:111)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.create(ClientNamenodeProtocolServerSideTranslatorPB.java:393)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2086)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2082)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2080)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at
> org.apache.hadoop.ipc.RemoteException.instantiateException(
[jira] [Updated] (HIVE-14141) Fix for HIVE-14062 breaks indirect urls in beeline
[
https://issues.apache.org/jira/browse/HIVE-14141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vihang Karajgaonkar updated HIVE-14141:
---
Affects Version/s: 2.1.0
Status: Patch Available (was: Open)
> Fix for HIVE-14062 breaks indirect urls in beeline
> --
>
> Key: HIVE-14141
> URL: https://issues.apache.org/jira/browse/HIVE-14141
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 2.1.0
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Minor
> Attachments: HIVE-14141.1.patch
>
>
> Looks like the patch for HIVE-14062 breaks indirect urls which uses
> environment variables to get the url in beeline
> In order to reproduce this issue:
> {noformat}
> $ export BEELINE_URL_DEFAULT="jdbc:hive2://localhost:1"
> $ beeline -u default
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14111) better concurrency handling for TezSessionState - part I
[ https://issues.apache.org/jira/browse/HIVE-14111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357921#comment-15357921 ] Siddharth Seth commented on HIVE-14111: --- [~sershe] - I didn't meant to add complexity to the patch. The one I attached contained comments from me walking through the code - some related to this jira, other comments in general. We could check in the comments (which are added in your second patch). It would be better to create jiras to track them though - especially ones which you think are valid (I think some of the comments in the file I uploaded were questions, and some may not have been valid). bq. What is session.destroy? Rather what object is this; do you mean TezClient::stop? In TezTask.submit -> TezSessionPoolManager.getInstance().destroySession(sessionState); I think that's the main bit which needs to be looked at from your original patch, to get past the core problem here of a session getting lost in case of an error. > better concurrency handling for TezSessionState - part I > > > Key: HIVE-14111 > URL: https://issues.apache.org/jira/browse/HIVE-14111 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-14111.01.patch, HIVE-14111.02.patch, > HIVE-14111.patch, sessionPoolNotes.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14141) Fix for HIVE-14062 breaks indirect urls in beeline
[
https://issues.apache.org/jira/browse/HIVE-14141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vihang Karajgaonkar updated HIVE-14141:
---
Attachment: HIVE-14141.1.patch
> Fix for HIVE-14062 breaks indirect urls in beeline
> --
>
> Key: HIVE-14141
> URL: https://issues.apache.org/jira/browse/HIVE-14141
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Minor
> Attachments: HIVE-14141.1.patch
>
>
> Looks like the patch for HIVE-14062 breaks indirect urls which uses
> environment variables to get the url in beeline
> In order to reproduce this issue:
> {noformat}
> $ export BEELINE_URL_DEFAULT="jdbc:hive2://localhost:1"
> $ beeline -u default
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14140) LLAP: package codec jars
[ https://issues.apache.org/jira/browse/HIVE-14140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-14140: Status: Patch Available (was: Open) > LLAP: package codec jars > > > Key: HIVE-14140 > URL: https://issues.apache.org/jira/browse/HIVE-14140 > Project: Hive > Issue Type: Bug >Reporter: Takahiko Saito >Assignee: Sergey Shelukhin > Attachments: HIVE-14140.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14140) LLAP: package codec jars
[ https://issues.apache.org/jira/browse/HIVE-14140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-14140: Reporter: Takahiko Saito (was: Sergey Shelukhin) > LLAP: package codec jars > > > Key: HIVE-14140 > URL: https://issues.apache.org/jira/browse/HIVE-14140 > Project: Hive > Issue Type: Bug >Reporter: Takahiko Saito >Assignee: Sergey Shelukhin > Attachments: HIVE-14140.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14140) LLAP: package codec jars
[ https://issues.apache.org/jira/browse/HIVE-14140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-14140: Attachment: HIVE-14140.patch [~gopalv] can you take a look? > LLAP: package codec jars > > > Key: HIVE-14140 > URL: https://issues.apache.org/jira/browse/HIVE-14140 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-14140.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11527) bypass HiveServer2 thrift interface for query results
[
https://issues.apache.org/jira/browse/HIVE-11527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357810#comment-15357810
]
Hive QA commented on HIVE-11527:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12815274/HIVE-11527.11.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 20 failed/errored test(s), 10294 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_all
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_join
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testDelayedLocalityNodeCommErrorImmediateAllocation
org.apache.hive.jdbc.TestJdbcDriver2.testDataTypes
org.apache.hive.jdbc.TestJdbcDriver2.testFetchFirstNonMR
org.apache.hive.jdbc.TestJdbcDriver2.testInsertOverwriteExecAsync2
org.apache.hive.jdbc.TestJdbcDriver2.testInsertOverwriteExecAsync3
org.apache.hive.jdbc.TestJdbcDriver2.testParentReferences
org.apache.hive.jdbc.TestJdbcDriver2.testPrepareSetDate
org.apache.hive.jdbc.TestJdbcDriver2.testPrepareSetTimestamp
org.apache.hive.jdbc.TestJdbcDriver2.testResultSetMetaData
org.apache.hive.jdbc.TestJdbcDriver2.testSelectExecAsync2
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testAddJarConstructorUnCaching
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testParallelCompilation
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testParallelCompilation2
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testUnableUseBypassCase
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/328/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/328/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-328/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 20 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12815274 - PreCommit-HIVE-MASTER-Build
> bypass HiveServer2 thrift interface for query results
> -
>
> Key: HIVE-11527
> URL: https://issues.apache.org/jira/browse/HIVE-11527
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Reporter: Sergey Shelukhin
>Assignee: Takanobu Asanuma
> Attachments: HIVE-11527.10.patch, HIVE-11527.11.patch,
> HIVE-11527.WIP.patch
>
>
> Right now, HS2 reads query results and returns them to the caller via its
> thrift API.
> There should be an option for HS2 to return some pointer to results (an HDFS
> link?) and for the user to read the results directly off HDFS inside the
> cluster, or via something like WebHDFS outside the cluster
> Review board link: https://reviews.apache.org/r/40867
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14078) LLAP input split should get task attempt number from conf if available
[ https://issues.apache.org/jira/browse/HIVE-14078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-14078: -- Resolution: Fixed Fix Version/s: 2.2.0 Status: Resolved (was: Patch Available) committed to master > LLAP input split should get task attempt number from conf if available > -- > > Key: HIVE-14078 > URL: https://issues.apache.org/jira/browse/HIVE-14078 > Project: Hive > Issue Type: Sub-task > Components: llap >Reporter: Jason Dere >Assignee: Jason Dere > Fix For: 2.2.0 > > Attachments: HIVE-14078.1.patch, HIVE-14078.2.patch > > > Currently the attempt number is hard-coded to 0. If the split is being > fetched as part of a hadoop job we can get the task attempt ID from the conf > if it has been set, and use the attempt number from that. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13945) Decimal value is displayed as rounded when selecting where clause with that decimal value.
[
https://issues.apache.org/jira/browse/HIVE-13945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-13945:
Attachment: HIVE-13945.06.patch
One test update, the rest appear unrelated.
> Decimal value is displayed as rounded when selecting where clause with that
> decimal value.
> --
>
> Key: HIVE-13945
> URL: https://issues.apache.org/jira/browse/HIVE-13945
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: Takahiko Saito
>Assignee: Sergey Shelukhin
>Priority: Critical
> Attachments: HIVE-13945.01.patch, HIVE-13945.02.patch,
> HIVE-13945.03.patch, HIVE-13945.03.patch, HIVE-13945.04.patch,
> HIVE-13945.05.patch, HIVE-13945.06.patch, HIVE-13945.patch
>
>
> Create a table withe a column of decimal type(38,18) and insert
> '4327269606205.029297'. Then select with that value displays its rounded
> value, which is 4327269606205.029300
> {noformat}
> 0: jdbc:hive2://os-r7-mvjkcu-hiveserver2-11-4> drop table if exists test;
> No rows affected (0.229 seconds)
> 0: jdbc:hive2://os-r7-mvjkcu-hiveserver2-11-4>
> 0: jdbc:hive2://os-r7-mvjkcu-hiveserver2-11-4> create table test (dc
> decimal(38,18));
> No rows affected (0.125 seconds)
> 0: jdbc:hive2://os-r7-mvjkcu-hiveserver2-11-4>
> 0: jdbc:hive2://os-r7-mvjkcu-hiveserver2-11-4> insert into table test values
> (4327269606205.029297);
> No rows affected (2.372 seconds)
> 0: jdbc:hive2://os-r7-mvjkcu-hiveserver2-11-4>
> 0: jdbc:hive2://os-r7-mvjkcu-hiveserver2-11-4> select * from test;
> +---+--+
> | test.dc |
> +---+--+
> | 4327269606205.029297 |
> +---+--+
> 1 row selected (0.123 seconds)
> 0: jdbc:hive2://os-r7-mvjkcu-hiveserver2-11-4>
> 0: jdbc:hive2://os-r7-mvjkcu-hiveserver2-11-4> select * from test where dc =
> 4327269606205.029297;
> +---+--+
> | test.dc |
> +---+--+
> | 4327269606205.029300 |
> +---+--+
> 1 row selected (0.109 seconds)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14052) Cleanup of structures required when LLAP access from external clients completes
[ https://issues.apache.org/jira/browse/HIVE-14052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357752#comment-15357752 ] Jason Dere commented on HIVE-14052: --- [~sseth] [~sershe] can you take a look? > Cleanup of structures required when LLAP access from external clients > completes > --- > > Key: HIVE-14052 > URL: https://issues.apache.org/jira/browse/HIVE-14052 > Project: Hive > Issue Type: Bug > Components: llap >Reporter: Jason Dere >Assignee: Jason Dere > Attachments: HIVE-14052.1.patch > > > Per [~sseth]: There's no cleanup at the moment, and structures used in LLAP > to track a query will keep building up slowly over time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14119) LLAP external recordreader not returning non-ascii string properly
[ https://issues.apache.org/jira/browse/HIVE-14119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-14119: -- Attachment: HIVE-14119.2.patch precommit tests never ran - reattaching patch > LLAP external recordreader not returning non-ascii string properly > -- > > Key: HIVE-14119 > URL: https://issues.apache.org/jira/browse/HIVE-14119 > Project: Hive > Issue Type: Sub-task > Components: llap >Reporter: Jason Dere >Assignee: Jason Dere > Attachments: HIVE-14119.1.patch, HIVE-14119.2.patch > > > Strings with non-ascii chars showing up with "\�\�\� " -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14093) LLAP output format connection should wait for all writes to finish before closing channel
[ https://issues.apache.org/jira/browse/HIVE-14093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-14093: -- Resolution: Fixed Fix Version/s: 2.2.0 Status: Resolved (was: Patch Available) Committed to master > LLAP output format connection should wait for all writes to finish before > closing channel > - > > Key: HIVE-14093 > URL: https://issues.apache.org/jira/browse/HIVE-14093 > Project: Hive > Issue Type: Sub-task > Components: llap >Reporter: Jason Dere >Assignee: Jason Dere > Fix For: 2.2.0 > > Attachments: HIVE-14093.1.patch, HIVE-14093.2.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14091) some errors are not propagated to LLAP external clients
[ https://issues.apache.org/jira/browse/HIVE-14091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-14091: Resolution: Fixed Fix Version/s: 2.2.0 Status: Resolved (was: Patch Available) Committed to master > some errors are not propagated to LLAP external clients > --- > > Key: HIVE-14091 > URL: https://issues.apache.org/jira/browse/HIVE-14091 > Project: Hive > Issue Type: Bug >Reporter: Jason Dere >Assignee: Sergey Shelukhin > Fix For: 2.2.0 > > Attachments: HIVE-14091.01.patch, HIVE-14091.02.patch, > HIVE-14091.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14063) beeline to auto connect to the HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-14063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar updated HIVE-14063: --- Attachment: beeline.conf.template > beeline to auto connect to the HiveServer2 > -- > > Key: HIVE-14063 > URL: https://issues.apache.org/jira/browse/HIVE-14063 > Project: Hive > Issue Type: Improvement > Components: Beeline >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Minor > Attachments: beeline.conf.template > > > Currently one has to give an jdbc:hive2 url in order for Beeline to connect a > hiveserver2 instance. It would be great if Beeline can get the info somehow > (from a properties file at a well-known location?) and connect automatically > if user doesn't specify such a url. If the properties file is not present, > then beeline would expect user to provide the url and credentials using > !connect or ./beeline -u .. commands > While Beeline is flexible (being a mere JDBC client), most environments would > have just a single HS2. Having users to manually connect into this via either > "beeline ~/.propsfile" or -u or !connect statements is lowering the > experience part. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14063) beeline to auto connect to the HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-14063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357656#comment-15357656 ] Vihang Karajgaonkar commented on HIVE-14063: Thanks [~szehon] for your comment. I have attached first draft version of the how the conf file may look like. We can add more documentation of each parameter in the template file so that users can understand when each parameter is used. A template file called beeline.conf.template can be provided in the conf directory and user can use it as a template to create the beeline.conf at pre-defined set of locations (need to be determined) beeline.properties file is used to store the properties which include lastConnectedUrl when the user issues a !save command from beeline command line. Once it is saved the user can issue !reconnect to reconnect to the last connected url. Another way to use default urls is to provide environment variables starting with BEELINE_URL_ and then issue beeline connect -u . Key point being user needs to either issue !reconnect or ./beeline connect -u id command. The objective of providing a default conf file is to provide seemless experience to the user so that he/she can connect directly by issuing ./beeline (just like hiveCli) Both these methods are orthogonal so that they both can be independently used without affecting each other. User can choose one way or the other based on their preference. They can co-exist together too such that beeline.properties should take precedence over beeline.conf file. The reason being user explicitly issues !save command with the intention to reuse it by issuing !reconnect command. Eg. if the user gets auto-connected using the beeline.conf file, and issues !save it will create a ~/.beeline/beeline.properties and it will be used as long user fires the !reconnect command. In such case the design should detect that beeline.properties exists and it should not use the default conf file. Since the user explicitly issued !save command earlier, as long as ~/.beeline/beeline.properties exists, that file will be used and user can either issue !reconnect or the complete !connect -u .. url. If however, beeline.properties does not exist, and beeline.conf exists, beeline will attempt to initiate connection using the conf file. If none of the files are present current behavior will continue. [~spena] [~mohitsabharwal] [~sircodesalot] [~lskuff] please review and let us know your thoughts. Feel free to include anyone who might be interested. Thanks! > beeline to auto connect to the HiveServer2 > -- > > Key: HIVE-14063 > URL: https://issues.apache.org/jira/browse/HIVE-14063 > Project: Hive > Issue Type: Improvement > Components: Beeline >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Minor > > Currently one has to give an jdbc:hive2 url in order for Beeline to connect a > hiveserver2 instance. It would be great if Beeline can get the info somehow > (from a properties file at a well-known location?) and connect automatically > if user doesn't specify such a url. If the properties file is not present, > then beeline would expect user to provide the url and credentials using > !connect or ./beeline -u .. commands > While Beeline is flexible (being a mere JDBC client), most environments would > have just a single HS2. Having users to manually connect into this via either > "beeline ~/.propsfile" or -u or !connect statements is lowering the > experience part. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14137) Hive on Spark throws FileAlreadyExistsException for jobs with multiple empty tables
[
https://issues.apache.org/jira/browse/HIVE-14137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Peña updated HIVE-14137:
---
Component/s: Spark
> Hive on Spark throws FileAlreadyExistsException for jobs with multiple empty
> tables
> ---
>
> Key: HIVE-14137
> URL: https://issues.apache.org/jira/browse/HIVE-14137
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
> Attachments: HIVE-14137.1.patch, HIVE-14137.patch
>
>
> The following queries:
> {code}
> -- Setup
> drop table if exists empty1;
> create table empty1 (col1 bigint) stored as parquet tblproperties
> ('parquet.compress'='snappy');
> drop table if exists empty2;
> create table empty2 (col1 bigint, col2 bigint) stored as parquet
> tblproperties ('parquet.compress'='snappy');
> drop table if exists empty3;
> create table empty3 (col1 bigint) stored as parquet tblproperties
> ('parquet.compress'='snappy');
> -- All empty HDFS directories.
> -- Fails with [08S01]: Error while processing statement: FAILED: Execution
> Error, return code 3 from org.apache.hadoop.hive.ql.exec.spark.SparkTask.
> select empty1.col1
> from empty1
> inner join empty2
> on empty2.col1 = empty1.col1
> inner join empty3
> on empty3.col1 = empty2.col2;
> -- Two empty HDFS directories.
> -- Create an empty file in HDFS.
> insert into empty1 select * from empty1 where false;
> -- Same query fails with [08S01]: Error while processing statement: FAILED:
> Execution Error, return code 3 from
> org.apache.hadoop.hive.ql.exec.spark.SparkTask.
> select empty1.col1
> from empty1
> inner join empty2
> on empty2.col1 = empty1.col1
> inner join empty3
> on empty3.col1 = empty2.col2;
> -- One empty HDFS directory.
> -- Create an empty file in HDFS.
> insert into empty2 select * from empty2 where false;
> -- Same query succeeds.
> select empty1.col1
> from empty1
> inner join empty2
> on empty2.col1 = empty1.col1
> inner join empty3
> on empty3.col1 = empty2.col2;
> {code}
> Will result in the following exception:
> {code}
> org.apache.hadoop.fs.FileAlreadyExistsException:
> /tmp/hive/hive/1f3837aa-9407-4780-92b1-42a66d205139/hive_2016-06-24_15-45-23_206_79177714958655528-2/-mr-10004/0/emptyFile
> for client 172.26.14.151 already exists
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:2784)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInt(FSNamesystem.java:2676)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:2561)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.create(NameNodeRpcServer.java:593)
> at
> org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.create(AuthorizationProviderProxyClientProtocol.java:111)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.create(ClientNamenodeProtocolServerSideTranslatorPB.java:393)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2086)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2082)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2080)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at
> org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
> at
> org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)
> at
> org.apache.hadoop.hdfs.DFSOutputStream.newStreamForCreate(DFSOutputStream.java:1902)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1738)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1663)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$6.doCall(DistributedFileSystem.java:405)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$6.doCall(Distributed
[jira] [Commented] (HIVE-14137) Hive on Spark throws FileAlreadyExistsException for jobs with multiple empty tables
[
https://issues.apache.org/jira/browse/HIVE-14137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357628#comment-15357628
]
Hive QA commented on HIVE-14137:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12815273/HIVE-14137.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 48 failed/errored test(s), 10287 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_view_sqlstd
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join32
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_input26
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_rp_lineage2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_stats
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_compute_stats_empty_table
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_diff_part_input_formats
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_fileformat_mix
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_constcolval
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_6
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_hook_context_cs
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auth
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_creation
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input23
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input24
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input25
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input26
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input41
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert_into_with_schema
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insertoverwrite_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_empty
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_lineage2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mergejoin
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_metadataonly1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nestedvirtual
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullgroup5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullinput
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullinput2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_optimize_nullscan
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_boolexpr
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_pcs
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_join4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_union_view
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_reduce_deduplicate_exclude_gby
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_schema_evol_orc_nonvec_mapwork_part
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_semijoin5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_topn
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_unionall_join_nullconstant
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_unionall_unbalancedppd
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_join
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_queries
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_all
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_join
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testDelayedLocalityNodeCommErrorImmediateAllocation
org.apache.hive.service.cli.TestEmbeddedThriftBinaryCLIService.testExecuteStatementParallel
org.apache.hive.service.cli.TestEmbeddedThriftBinaryCLIService.testTaskStatus
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/327/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/327/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-327/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 48 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12815273 - PreCommit-HIVE-MASTER-Build
> Hive on Spark throws FileAlreadyExistsException for jobs with multiple empty
> tables
> ---
>
> Key: HIVE-14137
> URL: https://issues.ap
[jira] [Updated] (HIVE-14037) java.lang.ClassNotFoundException for the jar in hive.reloadable.aux.jars.path in mapreduce
[
https://issues.apache.org/jira/browse/HIVE-14037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu updated HIVE-14037:
Resolution: Fixed
Fix Version/s: 2.2.0
Status: Resolved (was: Patch Available)
Thanks [~Ferd] and [~Yibing] for reviewing. Pushed to master.
> java.lang.ClassNotFoundException for the jar in hive.reloadable.aux.jars.path
> in mapreduce
> ---
>
> Key: HIVE-14037
> URL: https://issues.apache.org/jira/browse/HIVE-14037
> Project: Hive
> Issue Type: Bug
>Reporter: Aihua Xu
>Assignee: Aihua Xu
> Fix For: 2.2.0
>
> Attachments: HIVE-14037.1.patch, HIVE-14037.2.patch,
> HIVE-14037.3.patch, HIVE-14037.4.patch, HIVE-14037.5.patch
>
>
> The jars in hive.reloadable.aux.jars.path seem to be available in HS2 process
> while they are not available in the Mapper or Reducer nodes which will throw
> the following exception.
> {noformat}
> org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.isDeterministic(ExprNodeGenericFuncEvaluator.java:152)
> at
> org.apache.hadoop.hive.ql.exec.ExprNodeEvaluatorFactory.iterate(ExprNodeEvaluatorFactory.java:97)
> at
> org.apache.hadoop.hive.ql.exec.ExprNodeEvaluatorFactory.toCachedEvals(ExprNodeEvaluatorFactory.java:71)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:59)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:385)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:469)
> at
> org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:425)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:193)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:385)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:431)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:385)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:126)
> ... 22 more
> Caused by: java.lang.ClassNotFoundException: test.UDF
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:270)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.getUdfClass(GenericUDFBridge.java:132)
> ... 36 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14139) NPE dropping permanent function
[
https://issues.apache.org/jira/browse/HIVE-14139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357596#comment-15357596
]
Sergey Shelukhin commented on HIVE-14139:
-
[~lirui] are you looking into it? I can take a look today or tomorrow otherwise.
> NPE dropping permanent function
> ---
>
> Key: HIVE-14139
> URL: https://issues.apache.org/jira/browse/HIVE-14139
> Project: Hive
> Issue Type: Bug
>Reporter: Rui Li
>Assignee: Rui Li
>
> To reproduce:
> 1. Start a CLI session and create a permanent function.
> 2. Exit current CLI session.
> 3. Start a new CLI session and drop the function.
> Stack trace:
> {noformat}
> FAILED: error during drop function: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.Registry.removePersistentFunctionUnderLock(Registry.java:513)
> at
> org.apache.hadoop.hive.ql.exec.Registry.unregisterFunction(Registry.java:501)
> at
> org.apache.hadoop.hive.ql.exec.FunctionRegistry.unregisterPermanentFunction(FunctionRegistry.java:1532)
> at
> org.apache.hadoop.hive.ql.exec.FunctionTask.dropPermanentFunction(FunctionTask.java:228)
> at
> org.apache.hadoop.hive.ql.exec.FunctionTask.execute(FunctionTask.java:95)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1860)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1564)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1316)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1085)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1073)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:232)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-5922) In orc.InStream.CompressedStream, the desired position passed to seek can equal offsets[i] + bytes[i].remaining() when ORC predicate pushdown is enabled
[
https://issues.apache.org/jira/browse/HIVE-5922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357581#comment-15357581
]
Frank Luo commented on HIVE-5922:
-
same issue here and I am using Apache Hive 1.2.1.
> In orc.InStream.CompressedStream, the desired position passed to seek can
> equal offsets[i] + bytes[i].remaining() when ORC predicate pushdown is enabled
>
>
> Key: HIVE-5922
> URL: https://issues.apache.org/jira/browse/HIVE-5922
> Project: Hive
> Issue Type: Bug
> Components: File Formats
>Reporter: Yin Huai
>
> Two stack traces ...
> {code}
> java.io.IOException: IO error in map input file
> hdfs://10.38.55.204:8020/user/hive/warehouse/ssdb_bin_compress_orc_large_0_13.db/cycle/04_0
> at
> org.apache.hadoop.mapred.MapTask$TrackedRecordReader.moveToNext(MapTask.java:236)
> at
> org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:210)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
> at org.apache.hadoop.mapred.Child.main(Child.java:249)
> Caused by: java.io.IOException: java.io.IOException: Seek outside of data in
> compressed stream Stream for column 9 kind DATA position: 21496054 length:
> 33790900 range: 2 offset: 1048588 limit: 1048588 range 0 = 13893791 to
> 1048588; range 1 = 17039555 to 1310735; range 2 = 20447466 to 1048588;
> range 3 = 23855377 to 1048588; range 4 = 27263288 to 1048588; range 5 =
> 30409052 to 1310735 uncompressed: 262144 to 262144 to 21496054
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderNextException(HiveIOExceptionHandlerChain.java:121)
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderNextException(HiveIOExceptionHandlerUtil.java:77)
> at
> org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:276)
> at
> org.apache.hadoop.hive.ql.io.HiveRecordReader.doNext(HiveRecordReader.java:79)
> at
> org.apache.hadoop.hive.ql.io.HiveRecordReader.doNext(HiveRecordReader.java:33)
> at
> org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.next(HiveContextAwareRecordReader.java:108)
> at
> org.apache.hadoop.mapred.MapTask$TrackedRecordReader.moveToNext(MapTask.java:230)
> ... 9 more
> Caused by: java.io.IOException: Seek outside of data in compressed stream
> Stream for column 9 kind DATA position: 21496054 length: 33790900 range: 2
> offset: 1048588 limit: 1048588 range 0 = 13893791 to 1048588; range 1 =
> 17039555 to 1310735; range 2 = 20447466 to 1048588; range 3 = 23855377 to
> 1048588; range 4 = 27263288 to 1048588; range 5 = 30409052 to 1310735
> uncompressed: 262144 to 262144 to 21496054
> at
> org.apache.hadoop.hive.ql.io.orc.InStream$CompressedStream.seek(InStream.java:328)
> at
> org.apache.hadoop.hive.ql.io.orc.InStream$CompressedStream.readHeader(InStream.java:161)
> at
> org.apache.hadoop.hive.ql.io.orc.InStream$CompressedStream.read(InStream.java:205)
> at
> org.apache.hadoop.hive.ql.io.orc.SerializationUtils.readInts(SerializationUtils.java:450)
> at
> org.apache.hadoop.hive.ql.io.orc.RunLengthIntegerReaderV2.readDirectValues(RunLengthIntegerReaderV2.java:240)
> at
> org.apache.hadoop.hive.ql.io.orc.RunLengthIntegerReaderV2.readValues(RunLengthIntegerReaderV2.java:53)
> at
> org.apache.hadoop.hive.ql.io.orc.RunLengthIntegerReaderV2.next(RunLengthIntegerReaderV2.java:288)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl$IntTreeReader.next(RecordReaderImpl.java:510)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl$StructTreeReader.next(RecordReaderImpl.java:1581)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.next(RecordReaderImpl.java:2707)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$OrcRecordReader.next(OrcInputFormat.java:110)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$OrcRecordReader.next(OrcInputFormat.java:86)
> at
> org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:274)
> ... 13 more
> {\code}
> {code}
> java.io.IOException: IO error in map input file
> hdfs://10.38.55.204:8020/user/hive/
[jira] [Updated] (HIVE-11402) HS2 - disallow parallel query execution within a single Session
[ https://issues.apache.org/jira/browse/HIVE-11402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-11402: Attachment: HIVE-11402.02.patch Fixed. > HS2 - disallow parallel query execution within a single Session > --- > > Key: HIVE-11402 > URL: https://issues.apache.org/jira/browse/HIVE-11402 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Thejas M Nair >Assignee: Sergey Shelukhin > Attachments: HIVE-11402.01.patch, HIVE-11402.02.patch, > HIVE-11402.patch > > > HiveServer2 currently allows concurrent queries to be run in a single > session. However, every HS2 session has an associated SessionState object, > and the use of SessionState in many places assumes that only one thread is > using it, ie it is not thread safe. > There are many places where SesssionState thread safety needs to be > addressed, and until then we should serialize all query execution for a > single HS2 session. -This problem can become more visible with HIVE-4239 now > allowing parallel query compilation.- > Note that running queries in parallel for single session is not > straightforward with jdbc, you need to spawn another thread as the > Statement.execute calls are blocking. I believe ODBC has non blocking query > execution API, and Hue is another well known application that shares sessions > for all queries that a user runs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14135) beeline output not formatted correctly for large column widths
[ https://issues.apache.org/jira/browse/HIVE-14135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357510#comment-15357510 ] Vihang Karajgaonkar commented on HIVE-14135: [~spena] can you please review the patch? > beeline output not formatted correctly for large column widths > -- > > Key: HIVE-14135 > URL: https://issues.apache.org/jira/browse/HIVE-14135 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 2.2.0 >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar > Attachments: HIVE-14135.1.patch > > > If the column width is too large then beeline uses the maximum column width > when normalizing all the column widths. In order to reproduce the issue, run > set -v; > Once the configuration variables is classpath which can be extremely large > width (41k characters in my environment). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13278) Many redundant 'File not found' messages appeared in container log during query execution with Hive on Spark
[
https://issues.apache.org/jira/browse/HIVE-13278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357452#comment-15357452
]
Sahil Takiar commented on HIVE-13278:
-
Ideally yes, but there are a few issues. The code that causes this exception is
in the {{Utilities.getBaseWork}} method. This method basically fetches the
{{Path}} to the plan file (e.g. {{reduce.xml}}), and tries to de-serialize it.
If the plan file does not exist, the code will throw a
{{FileNotFoundException}} which will then be caught and logged, and cause the
method to return null. The problem is that the code doesn't differentiate
between the scenarios where the plan file was accidentally deleted vs. no plan
file was ever created in the first place (maybe a NPE gets thrown somewhere
down the line, but there is guarantee). There is a comment in the code saying
that the the {{FileNotFoundException}} "happens. e.g.: no reduce work."
> Many redundant 'File not found' messages appeared in container log during
> query execution with Hive on Spark
>
>
> Key: HIVE-13278
> URL: https://issues.apache.org/jira/browse/HIVE-13278
> Project: Hive
> Issue Type: Bug
> Environment: Hive on Spark engine
> Found based on :
> Apache Hive 2.0.0
> Apache Spark 1.6.0
>Reporter: Xin Hao
>Assignee: Sahil Takiar
>Priority: Minor
>
> Many redundant 'File not found' messages appeared in container log during
> query execution with Hive on Spark.
> Certainly, it doesn't prevent the query from running successfully. So mark it
> as Minor currently.
> Error message example:
> {noformat}
> 16/03/14 01:45:06 INFO exec.Utilities: File not found: File does not exist:
> /tmp/hive/hadoop/2d378538-f5d3-493c-9276-c62dd6634fb4/hive_2016-03-14_01-44-16_835_623058724409492515-6/-mr-10010/0a6d0cae-1eb3-448c-883b-590b3b198a73/reduce.xml
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:66)
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:56)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsUpdateTimes(FSNamesystem.java:1932)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:1873)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1853)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1825)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:565)
> at
> org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.getBlockLocations(AuthorizationProviderProxyClientProtocol.java:87)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:363)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1060)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2086)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2082)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2080)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14027) NULL values produced by left outer join do not behave as NULL
[
https://issues.apache.org/jira/browse/HIVE-14027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357442#comment-15357442
]
Hive QA commented on HIVE-14027:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12814188/HIVE-14027.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 21 failed/errored test(s), 10287 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_all
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_join
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_auto_join29
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_auto_join30
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_auto_join_filters
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_auto_join_nulls
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_empty_join
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_union
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_groupby_mapjoin
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_join30
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_join_filters
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_join_nulls
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_nullsafe_join
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_outer_join0
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_outer_join1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_outer_join4
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_outer_join6
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/326/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/326/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-326/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 21 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12814188 - PreCommit-HIVE-MASTER-Build
> NULL values produced by left outer join do not behave as NULL
> -
>
> Key: HIVE-14027
> URL: https://issues.apache.org/jira/browse/HIVE-14027
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 1.2.1, 2.2.0
>Reporter: Vaibhav Gumashta
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-14027.01.patch, HIVE-14027.patch
>
>
> Consider the following setup:
> {code}
> create table tbl (n bigint, t string);
> insert into tbl values (1, 'one');
> insert into tbl values(2, 'two');
> select a.n, a.t, isnull(b.n), isnull(b.t) from (select * from tbl where n =
> 1) a left outer join (select * from tbl where 1 = 2) b on a.n = b.n;
> 1onefalsetrue
> {code}
> The query should return true for isnull(b.n).
> I've tested by inserting a row with null value for the bigint column into
> tbl, and isnull returns true in that case.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-11402) HS2 - disallow parallel query execution within a single Session
[ https://issues.apache.org/jira/browse/HIVE-11402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357330#comment-15357330 ] Mohit Sabharwal commented on HIVE-11402: [~sershe], we should keep the default for hive.server2.parallel.ops.in.session as true, otherwise this is a backward incompatible change for Hue and any other client code where queries currently share sessions. > HS2 - disallow parallel query execution within a single Session > --- > > Key: HIVE-11402 > URL: https://issues.apache.org/jira/browse/HIVE-11402 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Thejas M Nair >Assignee: Sergey Shelukhin > Attachments: HIVE-11402.01.patch, HIVE-11402.patch > > > HiveServer2 currently allows concurrent queries to be run in a single > session. However, every HS2 session has an associated SessionState object, > and the use of SessionState in many places assumes that only one thread is > using it, ie it is not thread safe. > There are many places where SesssionState thread safety needs to be > addressed, and until then we should serialize all query execution for a > single HS2 session. -This problem can become more visible with HIVE-4239 now > allowing parallel query compilation.- > Note that running queries in parallel for single session is not > straightforward with jdbc, you need to spawn another thread as the > Statement.execute calls are blocking. I believe ODBC has non blocking query > execution API, and Hue is another well known application that shares sessions > for all queries that a user runs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (HIVE-14027) NULL values produced by left outer join do not behave as NULL
[
https://issues.apache.org/jira/browse/HIVE-14027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-14027 started by Jesus Camacho Rodriguez.
--
> NULL values produced by left outer join do not behave as NULL
> -
>
> Key: HIVE-14027
> URL: https://issues.apache.org/jira/browse/HIVE-14027
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 1.2.1, 2.2.0
>Reporter: Vaibhav Gumashta
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-14027.01.patch, HIVE-14027.patch
>
>
> Consider the following setup:
> {code}
> create table tbl (n bigint, t string);
> insert into tbl values (1, 'one');
> insert into tbl values(2, 'two');
> select a.n, a.t, isnull(b.n), isnull(b.t) from (select * from tbl where n =
> 1) a left outer join (select * from tbl where 1 = 2) b on a.n = b.n;
> 1onefalsetrue
> {code}
> The query should return true for isnull(b.n).
> I've tested by inserting a row with null value for the bigint column into
> tbl, and isnull returns true in that case.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14027) NULL values produced by left outer join do not behave as NULL
[
https://issues.apache.org/jira/browse/HIVE-14027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-14027:
---
Attachment: HIVE-14027.01.patch
> NULL values produced by left outer join do not behave as NULL
> -
>
> Key: HIVE-14027
> URL: https://issues.apache.org/jira/browse/HIVE-14027
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 1.2.1, 2.2.0
>Reporter: Vaibhav Gumashta
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-14027.01.patch, HIVE-14027.patch
>
>
> Consider the following setup:
> {code}
> create table tbl (n bigint, t string);
> insert into tbl values (1, 'one');
> insert into tbl values(2, 'two');
> select a.n, a.t, isnull(b.n), isnull(b.t) from (select * from tbl where n =
> 1) a left outer join (select * from tbl where 1 = 2) b on a.n = b.n;
> 1onefalsetrue
> {code}
> The query should return true for isnull(b.n).
> I've tested by inserting a row with null value for the bigint column into
> tbl, and isnull returns true in that case.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14027) NULL values produced by left outer join do not behave as NULL
[
https://issues.apache.org/jira/browse/HIVE-14027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-14027:
---
Status: Patch Available (was: In Progress)
> NULL values produced by left outer join do not behave as NULL
> -
>
> Key: HIVE-14027
> URL: https://issues.apache.org/jira/browse/HIVE-14027
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 1.2.1, 2.2.0
>Reporter: Vaibhav Gumashta
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-14027.01.patch, HIVE-14027.patch
>
>
> Consider the following setup:
> {code}
> create table tbl (n bigint, t string);
> insert into tbl values (1, 'one');
> insert into tbl values(2, 'two');
> select a.n, a.t, isnull(b.n), isnull(b.t) from (select * from tbl where n =
> 1) a left outer join (select * from tbl where 1 = 2) b on a.n = b.n;
> 1onefalsetrue
> {code}
> The query should return true for isnull(b.n).
> I've tested by inserting a row with null value for the bigint column into
> tbl, and isnull returns true in that case.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14027) NULL values produced by left outer join do not behave as NULL
[
https://issues.apache.org/jira/browse/HIVE-14027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-14027:
---
Status: Open (was: Patch Available)
> NULL values produced by left outer join do not behave as NULL
> -
>
> Key: HIVE-14027
> URL: https://issues.apache.org/jira/browse/HIVE-14027
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 1.2.1, 2.2.0
>Reporter: Vaibhav Gumashta
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-14027.patch
>
>
> Consider the following setup:
> {code}
> create table tbl (n bigint, t string);
> insert into tbl values (1, 'one');
> insert into tbl values(2, 'two');
> select a.n, a.t, isnull(b.n), isnull(b.t) from (select * from tbl where n =
> 1) a left outer join (select * from tbl where 1 = 2) b on a.n = b.n;
> 1onefalsetrue
> {code}
> The query should return true for isnull(b.n).
> I've tested by inserting a row with null value for the bigint column into
> tbl, and isnull returns true in that case.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14138) CBO failed for select current_database()
[
https://issues.apache.org/jira/browse/HIVE-14138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357309#comment-15357309
]
Peter Vary commented on HIVE-14138:
---
[~jcamachorodriguez] Could you please take a look at this? Every time one of
the above queries is run, the error is logged, and the execution falls back to
the normal mode without CBO. I was able to come up a patch which turns of the
CBO for every query where there is no source table, but not sure that this is
the right solution, or even the problem :)
{noformat}
===
--- ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java
(revision de3d86cdd3db174d6bc5d8c65796a1e981171124)
+++ ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java
(revision )
@@ -387,8 +387,9 @@
boolean needToLogMessage = STATIC_LOG.isInfoEnabled();
boolean isSupportedRoot = root == HiveParser.TOK_QUERY || root ==
HiveParser.TOK_EXPLAIN
|| qb.isCTAS();
-boolean isSupportedType = qb.getIsQuery() || qb.isCTAS()
-|| cboCtx.type == PreCboCtx.Type.INSERT;
+// Queries without a source table currently are not supported by CBO
+boolean isSupportedType = (qb.getIsQuery() &&
!qb.containsQueryWithoutSourceTable())
+|| qb.isCTAS() || cboCtx.type == PreCboCtx.Type.INSERT;
boolean noBadTokens = HiveCalciteUtil.validateASTForUnsupportedTokens(ast);
boolean result = isSupportedRoot && isSupportedType && getCreateViewDesc()
== null
&& noBadTokens;
@@ -400,7 +401,7 @@
msg += "doesn't have QUERY or EXPLAIN as root and not a CTAS; ";
}
if (!isSupportedType) {
- msg += "is not a query, CTAS, or insert; ";
+ msg += "is not a query with at least one source table os subquery,
CTAS, or insert; ";
}
if (getCreateViewDesc() != null) {
msg += "has create view; ";
===
--- ql/src/java/org/apache/hadoop/hive/ql/parse/QBExpr.java (revision
de3d86cdd3db174d6bc5d8c65796a1e981171124)
+++ ql/src/java/org/apache/hadoop/hive/ql/parse/QBExpr.java (revision )
@@ -120,4 +120,17 @@
}
return qbexpr1.isSimpleSelectQuery() && qbexpr2.isSimpleSelectQuery();
}
+
+ /**
+ * returns true, if the query block contains any query, or subquery without
a source table
+ * Like select current_user(), select current_database()
+ * @return true, if the query block contains any query without a source table
+ */
+ public boolean containsQueryWithoutSourceTable() {
+if (qb != null) {
+ return qb.containsQueryWithoutSourceTable();
+} else {
+ return qbexpr1.containsQueryWithoutSourceTable() ||
qbexpr2.containsQueryWithoutSourceTable();
+}
+ }
}
===
--- ql/src/java/org/apache/hadoop/hive/ql/parse/QB.java (revision
de3d86cdd3db174d6bc5d8c65796a1e981171124)
+++ ql/src/java/org/apache/hadoop/hive/ql/parse/QB.java (revision )
@@ -437,4 +437,17 @@
return aliasInsideView;
}
+ /**
+ * returns true, if the query block contains any query, or subquery without
a source table
+ * Like select current_user(), select current_database()
+ * @return true, if the query block contains any query without a source table
+ */
+ public boolean containsQueryWithoutSourceTable() {
+for (QBExpr qbexpr : aliasToSubq.values()) {
+ if (qbexpr.containsQueryWithoutSourceTable()) {
+return true;
+ }
+}
+return aliasToTabs.size()==0 && aliasToSubq.size()==0;
+ }
}
{noformat}
> CBO failed for select current_database()
>
>
> Key: HIVE-14138
> URL: https://issues.apache.org/jira/browse/HIVE-14138
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Peter Vary
>Priority: Minor
>
> When issuing the following query, with hive.cbo.enable set to true:
> select current_database();
> The following exception is printed to the Hiveserver2 logs:
> 2016-06-30T09:58:24,146 ERROR [HiveServer2-Handler-Pool: Thread-33]
> parse.CalcitePlanner: CBO failed, skipping CBO.
> org.apache.hadoop.hive.ql.optimizer.calcite.CalciteSemanticException:
> Unsupported
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genLogicalPlan(CalcitePlanner.java:3136)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:940)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:894)
> at org.apache.calcite.tools.Frameworks$1.apply(Frameworks.java:113)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:969)
> at org.apache.calcite.to
[jira] [Commented] (HIVE-14118) Make the alter partition exception more meaningful
[ https://issues.apache.org/jira/browse/HIVE-14118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357247#comment-15357247 ] Sergio Peña commented on HIVE-14118: Great. Then I'm good with it. +1 > Make the alter partition exception more meaningful > -- > > Key: HIVE-14118 > URL: https://issues.apache.org/jira/browse/HIVE-14118 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 2.2.0 >Reporter: Aihua Xu >Assignee: Aihua Xu >Priority: Trivial > Attachments: HIVE-14118.1.patch > > > Right now when the alter partitions fails, "alter is not possible" is shown > in the log while the real exception/failure is hidden. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14118) Make the alter partition exception more meaningful
[ https://issues.apache.org/jira/browse/HIVE-14118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357240#comment-15357240 ] Aihua Xu commented on HIVE-14118: - Yeah. This is to log not to the console. > Make the alter partition exception more meaningful > -- > > Key: HIVE-14118 > URL: https://issues.apache.org/jira/browse/HIVE-14118 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 2.2.0 >Reporter: Aihua Xu >Assignee: Aihua Xu >Priority: Trivial > Attachments: HIVE-14118.1.patch > > > Right now when the alter partitions fails, "alter is not possible" is shown > in the log while the real exception/failure is hidden. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14118) Make the alter partition exception more meaningful
[ https://issues.apache.org/jira/browse/HIVE-14118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357236#comment-15357236 ] Sergio Peña commented on HIVE-14118: Can we send the stacktrace to the log then? I'd rather see clear messages on console when something happens instead of ugly stacktraces. > Make the alter partition exception more meaningful > -- > > Key: HIVE-14118 > URL: https://issues.apache.org/jira/browse/HIVE-14118 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 2.2.0 >Reporter: Aihua Xu >Assignee: Aihua Xu >Priority: Trivial > Attachments: HIVE-14118.1.patch > > > Right now when the alter partitions fails, "alter is not possible" is shown > in the log while the real exception/failure is hidden. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14138) CBO failed for select current_database()
[ https://issues.apache.org/jira/browse/HIVE-14138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357226#comment-15357226 ] Peter Vary commented on HIVE-14138: --- The same error gets logged with the following selects: - select current_user(); - select 1 + 1; - select * from dual union all select current_user(); I think the problem here is, that the CBO is not able to handle queries, where there is no source table. > CBO failed for select current_database() > > > Key: HIVE-14138 > URL: https://issues.apache.org/jira/browse/HIVE-14138 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Peter Vary >Priority: Minor > > When issuing the following query, with hive.cbo.enable set to true: > select current_database(); > The following exception is printed to the Hiveserver2 logs: > 2016-06-30T09:58:24,146 ERROR [HiveServer2-Handler-Pool: Thread-33] > parse.CalcitePlanner: CBO failed, skipping CBO. > org.apache.hadoop.hive.ql.optimizer.calcite.CalciteSemanticException: > Unsupported > at > org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genLogicalPlan(CalcitePlanner.java:3136) > at > org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:940) > at > org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:894) > at org.apache.calcite.tools.Frameworks$1.apply(Frameworks.java:113) > at > org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:969) > at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:149) > at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:106) > at > org.apache.hadoop.hive.ql.parse.CalcitePlanner.getOptimizedAST(CalcitePlanner.java:712) > at > org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:280) > at > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10795) > at > org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:239) > at > org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:250) > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:438) > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:329) > at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1159) > at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1146) > at > org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:191) > at > org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:276) > at > org.apache.hive.service.cli.operation.Operation.run(Operation.java:324) > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:464) > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:451) > at > org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:295) > at > org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:509) > at > org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1437) > at > org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1422) > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) > at > org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56) > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14111) better concurrency handling for TezSessionState - part I
[
https://issues.apache.org/jira/browse/HIVE-14111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357198#comment-15357198
]
Hive QA commented on HIVE-14111:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12815268/HIVE-14111.02.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 441 failed/errored test(s), 10286 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_schema_evol_orc_nonvec_mapwork_part
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_bucket_map_join_tez1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_bucket_map_join_tez2
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_constprog_dpp
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_2
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_3
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_4
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_5
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_2
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_3
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_4
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_5
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_dynamic_partition_pruning
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_dynamic_partition_pruning_2
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_hybridgrace_hashjoin_1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_hybridgrace_hashjoin_2
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_llap_nullscan
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_llap_udf
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_llapdecider
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_lvj_mapjoin
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_mapjoin_decimal
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_mrr
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_orc_llap_nonvector
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_orc_ppd_basic
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_bmj_schema_evolution
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_dml
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_dynpart_hashjoin_1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_dynpart_hashjoin_2
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_fsstat
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_insert_overwrite_local_directory_1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_join
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_join_hash
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_join_result_complex
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_join_tests
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_joins_explain
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_multi_union
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_schema_evolution
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_self_join
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_smb_1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_smb_main
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union2
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union_decimal
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union_dynamic_partition
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union_group_by
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union_multiinsert
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_vector_dynpart_hashjoin_1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_vector_dynpart_hashjoin_2
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_all
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_join
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_include_no_
[jira] [Commented] (HIVE-14118) Make the alter partition exception more meaningful
[ https://issues.apache.org/jira/browse/HIVE-14118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357195#comment-15357195 ] Aihua Xu commented on HIVE-14118: - Unfortunately there is no InvalidOperationException(message, Exception) but only InvalidOperationException(message). For debug purpose, actually the stacktrace is very helpful when the issue happens. > Make the alter partition exception more meaningful > -- > > Key: HIVE-14118 > URL: https://issues.apache.org/jira/browse/HIVE-14118 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 2.2.0 >Reporter: Aihua Xu >Assignee: Aihua Xu >Priority: Trivial > Attachments: HIVE-14118.1.patch > > > Right now when the alter partitions fails, "alter is not possible" is shown > in the log while the real exception/failure is hidden. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14118) Make the alter partition exception more meaningful
[ https://issues.apache.org/jira/browse/HIVE-14118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357192#comment-15357192 ] Sergio Peña commented on HIVE-14118: Also, could you paste an output example of the error to see how it will be displayed? > Make the alter partition exception more meaningful > -- > > Key: HIVE-14118 > URL: https://issues.apache.org/jira/browse/HIVE-14118 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 2.2.0 >Reporter: Aihua Xu >Assignee: Aihua Xu >Priority: Trivial > Attachments: HIVE-14118.1.patch > > > Right now when the alter partitions fails, "alter is not possible" is shown > in the log while the real exception/failure is hidden. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14118) Make the alter partition exception more meaningful
[
https://issues.apache.org/jira/browse/HIVE-14118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357191#comment-15357191
]
Sergio Peña commented on HIVE-14118:
Shouldn't be better to display e.getMessage() instead of of the whole
stacktrace? Mabye like {{throw new InvalidOperationException("Alter partition
operation fails: " + e.getMessage(), e);}}
> Make the alter partition exception more meaningful
> --
>
> Key: HIVE-14118
> URL: https://issues.apache.org/jira/browse/HIVE-14118
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
>Affects Versions: 2.2.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
>Priority: Trivial
> Attachments: HIVE-14118.1.patch
>
>
> Right now when the alter partitions fails, "alter is not possible" is shown
> in the log while the real exception/failure is hidden.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14109) query execuction throws NPE when hive.exec.submitviachild is set to true
[
https://issues.apache.org/jira/browse/HIVE-14109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu updated HIVE-14109:
Attachment: HIVE-14109.1.patch
> query execuction throws NPE when hive.exec.submitviachild is set to true
>
>
> Key: HIVE-14109
> URL: https://issues.apache.org/jira/browse/HIVE-14109
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 2.2.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
> Attachments: HIVE-14109.1.patch
>
>
> If we set hive.exec.submitviachild to true and execute select count(*) from
> src, the following exception is thrown.
> Seems queryState is not initialized when ExecDriver is called from main() in
> ExecDriver.
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.mr.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:262)
> at
> org.apache.hadoop.hive.ql.exec.mr.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:555)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:436)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecDriver.main(ExecDriver.java:756)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14109) query execuction throws NPE when hive.exec.submitviachild is set to true
[
https://issues.apache.org/jira/browse/HIVE-14109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu updated HIVE-14109:
Status: Open (was: Patch Available)
> query execuction throws NPE when hive.exec.submitviachild is set to true
>
>
> Key: HIVE-14109
> URL: https://issues.apache.org/jira/browse/HIVE-14109
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 2.2.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
> Attachments: HIVE-14109.1.patch
>
>
> If we set hive.exec.submitviachild to true and execute select count(*) from
> src, the following exception is thrown.
> Seems queryState is not initialized when ExecDriver is called from main() in
> ExecDriver.
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.mr.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:262)
> at
> org.apache.hadoop.hive.ql.exec.mr.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:555)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:436)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecDriver.main(ExecDriver.java:756)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14109) query execuction throws NPE when hive.exec.submitviachild is set to true
[
https://issues.apache.org/jira/browse/HIVE-14109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu updated HIVE-14109:
Status: Patch Available (was: Open)
> query execuction throws NPE when hive.exec.submitviachild is set to true
>
>
> Key: HIVE-14109
> URL: https://issues.apache.org/jira/browse/HIVE-14109
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 2.2.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
> Attachments: HIVE-14109.1.patch
>
>
> If we set hive.exec.submitviachild to true and execute select count(*) from
> src, the following exception is thrown.
> Seems queryState is not initialized when ExecDriver is called from main() in
> ExecDriver.
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.mr.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:262)
> at
> org.apache.hadoop.hive.ql.exec.mr.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:555)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:436)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecDriver.main(ExecDriver.java:756)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14109) query execuction throws NPE when hive.exec.submitviachild is set to true
[
https://issues.apache.org/jira/browse/HIVE-14109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu updated HIVE-14109:
Attachment: (was: HIVE-14109.1.patch)
> query execuction throws NPE when hive.exec.submitviachild is set to true
>
>
> Key: HIVE-14109
> URL: https://issues.apache.org/jira/browse/HIVE-14109
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 2.2.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
> Attachments: HIVE-14109.1.patch
>
>
> If we set hive.exec.submitviachild to true and execute select count(*) from
> src, the following exception is thrown.
> Seems queryState is not initialized when ExecDriver is called from main() in
> ExecDriver.
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.mr.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:262)
> at
> org.apache.hadoop.hive.ql.exec.mr.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:555)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:436)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecDriver.main(ExecDriver.java:756)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14037) java.lang.ClassNotFoundException for the jar in hive.reloadable.aux.jars.path in mapreduce
[
https://issues.apache.org/jira/browse/HIVE-14037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357065#comment-15357065
]
Aihua Xu commented on HIVE-14037:
-
Somehow the new test result is not reported here. See below for the new run.
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/310/testReport/
2 test failures are not related.
> java.lang.ClassNotFoundException for the jar in hive.reloadable.aux.jars.path
> in mapreduce
> ---
>
> Key: HIVE-14037
> URL: https://issues.apache.org/jira/browse/HIVE-14037
> Project: Hive
> Issue Type: Bug
>Reporter: Aihua Xu
>Assignee: Aihua Xu
> Attachments: HIVE-14037.1.patch, HIVE-14037.2.patch,
> HIVE-14037.3.patch, HIVE-14037.4.patch, HIVE-14037.5.patch
>
>
> The jars in hive.reloadable.aux.jars.path seem to be available in HS2 process
> while they are not available in the Mapper or Reducer nodes which will throw
> the following exception.
> {noformat}
> org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.isDeterministic(ExprNodeGenericFuncEvaluator.java:152)
> at
> org.apache.hadoop.hive.ql.exec.ExprNodeEvaluatorFactory.iterate(ExprNodeEvaluatorFactory.java:97)
> at
> org.apache.hadoop.hive.ql.exec.ExprNodeEvaluatorFactory.toCachedEvals(ExprNodeEvaluatorFactory.java:71)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:59)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:385)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:469)
> at
> org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:425)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:193)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:385)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:431)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:385)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:126)
> ... 22 more
> Caused by: java.lang.ClassNotFoundException: test.UDF
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:270)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.getUdfClass(GenericUDFBridge.java:132)
> ... 36 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14118) Make the alter partition exception more meaningful
[ https://issues.apache.org/jira/browse/HIVE-14118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-14118: Priority: Trivial (was: Major) > Make the alter partition exception more meaningful > -- > > Key: HIVE-14118 > URL: https://issues.apache.org/jira/browse/HIVE-14118 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 2.2.0 >Reporter: Aihua Xu >Assignee: Aihua Xu >Priority: Trivial > Attachments: HIVE-14118.1.patch > > > Right now when the alter partitions fails, "alter is not possible" is shown > in the log while the real exception/failure is hidden. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14118) Make the alter partition exception more meaningful
[ https://issues.apache.org/jira/browse/HIVE-14118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357056#comment-15357056 ] Aihua Xu commented on HIVE-14118: - [~spena] Can you help review the code? This is just to print better message for diagnostics. > Make the alter partition exception more meaningful > -- > > Key: HIVE-14118 > URL: https://issues.apache.org/jira/browse/HIVE-14118 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 2.2.0 >Reporter: Aihua Xu >Assignee: Aihua Xu > Attachments: HIVE-14118.1.patch > > > Right now when the alter partitions fails, "alter is not possible" is shown > in the log while the real exception/failure is hidden. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HIVE-14127) Hive on spark fails to find the plugged serde jar during runtime
[
https://issues.apache.org/jira/browse/HIVE-14127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu resolved HIVE-14127.
-
Resolution: Cannot Reproduce
Assignee: (was: Aihua Xu)
Seems it has been fixed.
> Hive on spark fails to find the plugged serde jar during runtime
>
>
> Key: HIVE-14127
> URL: https://issues.apache.org/jira/browse/HIVE-14127
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Affects Versions: 2.2.0
>Reporter: Aihua Xu
>
> Here is the stacktrace:
> {noformat}
> Driver stacktrace:
> at
> org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1431)
> at
> org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1419)
> at
> org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1418)
> at
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
> at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
> at
> org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1418)
> at
> org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:799)
> at
> org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:799)
> at scala.Option.foreach(Option.scala:236)
> at
> org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:799)
> at
> org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1640)
> at
> org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1599)
> at
> org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1588)
> at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
> Caused by: java.io.IOException: java.lang.reflect.InvocationTargetException
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
> at
> org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.initNextRecordReader(HadoopShimsSecure.java:265)
> at
> org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.(HadoopShimsSecure.java:212)
> at
> org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileInputFormatShim.getRecordReader(HadoopShimsSecure.java:332)
> at
> org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:721)
> at org.apache.spark.rdd.HadoopRDD$$anon$1.(HadoopRDD.scala:237)
> at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:208)
> at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:101)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
> at
> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:73)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
> at org.apache.spark.scheduler.Task.run(Task.scala:89)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at
> org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.initNextRecordReader(HadoopShimsSecure.java:251)
> ... 18 more
> Caused by: java.io.IOException: CombineHiveRecordReader: class not found
> com.ibm.spss.hive.serde2.xml.XmlInputFormat
> at
> org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.(CombineHiveRecordReader.java:55)
> ... 23 more
> {noformat}
--
This message was sent by Atlassian JIRA
[jira] [Updated] (HIVE-14123) Add beeline configuration option to show database in the prompt
[ https://issues.apache.org/jira/browse/HIVE-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Vary updated HIVE-14123: -- Attachment: HIVE-14123.3.patch Not to query the database in remote mode, if the showDbInPrompt is false. And some comments. > Add beeline configuration option to show database in the prompt > --- > > Key: HIVE-14123 > URL: https://issues.apache.org/jira/browse/HIVE-14123 > Project: Hive > Issue Type: Improvement > Components: Beeline, CLI >Affects Versions: 2.2.0 >Reporter: Peter Vary >Assignee: Peter Vary >Priority: Minor > Attachments: HIVE-14123.2.patch, HIVE-14123.3.patch, HIVE-14123.patch > > > There are several jira issues complaining that, the Beeline does not respect > hive.cli.print.current.db. > This is partially true, since in embedded mode, it uses the > hive.cli.print.current.db to change the prompt, since HIVE-10511. > In remote mode, I think this function should use a beeline command line > option instead, like for the showHeader option emphasizing, that this is a > client side option. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14136) LLAP ZK SecretManager should resolve _HOST in principal
[
https://issues.apache.org/jira/browse/HIVE-14136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357043#comment-15357043
]
Hive QA commented on HIVE-14136:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12815269/HIVE-14136.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10269 tests
executed
*Failed tests:*
{noformat}
TestMiniTezCliDriver-vectorized_parquet.q-tez_self_join.q-vector_date_1.q-and-12-more
- did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_all
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_join
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testDelayedLocalityNodeCommErrorImmediateAllocation
org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler.org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/324/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/324/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-324/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12815269 - PreCommit-HIVE-MASTER-Build
> LLAP ZK SecretManager should resolve _HOST in principal
> ---
>
> Key: HIVE-14136
> URL: https://issues.apache.org/jira/browse/HIVE-14136
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-14136.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14135) beeline output not formatted correctly for large column widths
[
https://issues.apache.org/jira/browse/HIVE-14135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15356915#comment-15356915
]
Hive QA commented on HIVE-14135:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12815263/HIVE-14135.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10271 tests
executed
*Failed tests:*
{noformat}
TestMiniTezCliDriver-vector_distinct_2.q-tez_joins_explain.q-cte_mat_1.q-and-12-more
- did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_all
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_join
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/323/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/323/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-323/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12815263 - PreCommit-HIVE-MASTER-Build
> beeline output not formatted correctly for large column widths
> --
>
> Key: HIVE-14135
> URL: https://issues.apache.org/jira/browse/HIVE-14135
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 2.2.0
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
> Attachments: HIVE-14135.1.patch
>
>
> If the column width is too large then beeline uses the maximum column width
> when normalizing all the column widths. In order to reproduce the issue, run
> set -v;
> Once the configuration variables is classpath which can be extremely large
> width (41k characters in my environment).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14123) Add beeline configuration option to show database in the prompt
[ https://issues.apache.org/jira/browse/HIVE-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Vary updated HIVE-14123: -- Status: Patch Available (was: Open) > Add beeline configuration option to show database in the prompt > --- > > Key: HIVE-14123 > URL: https://issues.apache.org/jira/browse/HIVE-14123 > Project: Hive > Issue Type: Improvement > Components: Beeline, CLI >Affects Versions: 2.2.0 >Reporter: Peter Vary >Assignee: Peter Vary >Priority: Minor > Attachments: HIVE-14123.2.patch, HIVE-14123.patch > > > There are several jira issues complaining that, the Beeline does not respect > hive.cli.print.current.db. > This is partially true, since in embedded mode, it uses the > hive.cli.print.current.db to change the prompt, since HIVE-10511. > In remote mode, I think this function should use a beeline command line > option instead, like for the showHeader option emphasizing, that this is a > client side option. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14032) INSERT OVERWRITE command failed with case sensitive partition key names
[ https://issues.apache.org/jira/browse/HIVE-14032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinna Rao Lalam updated HIVE-14032: Status: Patch Available (was: Open) > INSERT OVERWRITE command failed with case sensitive partition key names > --- > > Key: HIVE-14032 > URL: https://issues.apache.org/jira/browse/HIVE-14032 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 2.0.1 >Reporter: Chinna Rao Lalam >Assignee: Chinna Rao Lalam > Attachments: HIVE-14032.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14032) INSERT OVERWRITE command failed with case sensitive partition key names
[ https://issues.apache.org/jira/browse/HIVE-14032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinna Rao Lalam updated HIVE-14032: Attachment: HIVE-14032.patch Validating partition metadata was failed. > INSERT OVERWRITE command failed with case sensitive partition key names > --- > > Key: HIVE-14032 > URL: https://issues.apache.org/jira/browse/HIVE-14032 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 2.0.1 >Reporter: Chinna Rao Lalam >Assignee: Chinna Rao Lalam > Attachments: HIVE-14032.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14032) INSERT OVERWRITE command failed with case sensitive partition key names
[
https://issues.apache.org/jira/browse/HIVE-14032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15356840#comment-15356840
]
Chinna Rao Lalam commented on HIVE-14032:
-
INSERT OVERWRITE TABLE with IF NOT EXISTS for a partition is failed
{code}
drop table sample1;
create table sample1(a STRING, b int) partitioned by (PARTITIONID string) ROW
FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE;
LOAD DATA LOCAL INPATH '/data/file1.txt' INTO TABLE sample1 partition
(PARTITIONID = "one");
drop table sample2;
create table sample2(a STRING, b int) partitioned by (PARTITIONID string) ROW
FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE;
LOAD DATA LOCAL INPATH '/data/file2.txt' INTO TABLE sample2 partition
(PARTITIONID = "one");
INSERT OVERWRITE TABLE sample2 PARTITION (PARTITIONID = 'one') if not exists
select a,b from sample1;
{code}
> INSERT OVERWRITE command failed with case sensitive partition key names
> ---
>
> Key: HIVE-14032
> URL: https://issues.apache.org/jira/browse/HIVE-14032
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 2.0.1
>Reporter: Chinna Rao Lalam
>Assignee: Chinna Rao Lalam
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14032) INSERT OVERWRITE command failed with case sensitive partition key names
[ https://issues.apache.org/jira/browse/HIVE-14032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinna Rao Lalam updated HIVE-14032: Summary: INSERT OVERWRITE command failed with case sensitive partition key names (was: INSERT OVERWRITE command failed with sensitive names.) > INSERT OVERWRITE command failed with case sensitive partition key names > --- > > Key: HIVE-14032 > URL: https://issues.apache.org/jira/browse/HIVE-14032 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 2.0.1 >Reporter: Chinna Rao Lalam >Assignee: Chinna Rao Lalam > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14092) Kryo exception when deserializing VectorFileSinkOperator
[
https://issues.apache.org/jira/browse/HIVE-14092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15356830#comment-15356830
]
Rui Li commented on HIVE-14092:
---
I guess this fixes HIVE-13277 and HIVE-12677? The root cause should be with
[kryo|https://github.com/EsotericSoftware/kryo/issues/341].
> Kryo exception when deserializing VectorFileSinkOperator
>
>
> Key: HIVE-14092
> URL: https://issues.apache.org/jira/browse/HIVE-14092
> Project: Hive
> Issue Type: Bug
> Components: Serializers/Deserializers
>Affects Versions: 2.2.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
> Fix For: 2.2.0, 2.1.1
>
> Attachments: HIVE-14092.1.patch
>
>
> Following exception is thrown for queries using VectorFileSinkOperator
> {code}
> Caused by: java.lang.IllegalArgumentException: Unable to create serializer
> "org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer" for
> class: org.apache.hadoop.hive.ql.exec.vector.VectorFileSinkOperator
> at
> org.apache.hive.com.esotericsoftware.kryo.factories.ReflectionSerializerFactory.makeSerializer(ReflectionSerializerFactory.java:67)
> at
> org.apache.hive.com.esotericsoftware.kryo.factories.ReflectionSerializerFactory.makeSerializer(ReflectionSerializerFactory.java:45)
> at
> org.apache.hive.com.esotericsoftware.kryo.Kryo.newDefaultSerializer(Kryo.java:380)
> at
> org.apache.hive.com.esotericsoftware.kryo.Kryo.getDefaultSerializer(Kryo.java:364)
> at
> org.apache.hive.com.esotericsoftware.kryo.util.DefaultClassResolver.registerImplicit(DefaultClassResolver.java:74)
> at
> org.apache.hive.com.esotericsoftware.kryo.Kryo.getRegistration(Kryo.java:490)
> at
> org.apache.hive.com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:166)
> at
> org.apache.hive.com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:133)
> at
> org.apache.hive.com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:670)
> at
> org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readClass(SerializationUtilities.java:180)
> at
> org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:781)
> at
> org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readClassAndObject(SerializationUtilities.java:175)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:134)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:40)
> at
> org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:708)
> at
> org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readObject(SerializationUtilities.java:213)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
> ... 46 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.GeneratedConstructorAccessor6.newInstance(Unknown Source)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hive.com.esotericsoftware.kryo.factories.ReflectionSerializerFactory.makeSerializer(ReflectionSerializerFactory.java:54)
> ... 62 more
> Caused by: java.lang.StackOverflowError
> at java.util.HashMap.hash(HashMap.java:338)
> at java.util.HashMap.get(HashMap.java:556)
> at
> org.apache.hive.com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:61)
> at
> org.apache.hive.com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:62)
> at
> org.apache.hive.com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:62)
> at
> org.apache.hive.com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:62)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14126) With ranger enabled, partitioned columns is returned first when you execute 'select *'
[
https://issues.apache.org/jira/browse/HIVE-14126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15356794#comment-15356794
]
Hive QA commented on HIVE-14126:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12815265/HIVE-14126.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 10286 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_all
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_join
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/322/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/322/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-322/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12815265 - PreCommit-HIVE-MASTER-Build
> With ranger enabled, partitioned columns is returned first when you execute
> 'select *'
> --
>
> Key: HIVE-14126
> URL: https://issues.apache.org/jira/browse/HIVE-14126
> Project: Hive
> Issue Type: Bug
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-14126.01.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14138) CBO failed for select current_database()
[ https://issues.apache.org/jira/browse/HIVE-14138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15356710#comment-15356710 ] Peter Vary commented on HIVE-14138: --- The patch of HIVE-14123 uses this select, and the error log might confuse users > CBO failed for select current_database() > > > Key: HIVE-14138 > URL: https://issues.apache.org/jira/browse/HIVE-14138 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Peter Vary >Priority: Minor > > When issuing the following query, with hive.cbo.enable set to true: > select current_database(); > The following exception is printed to the Hiveserver2 logs: > 2016-06-30T09:58:24,146 ERROR [HiveServer2-Handler-Pool: Thread-33] > parse.CalcitePlanner: CBO failed, skipping CBO. > org.apache.hadoop.hive.ql.optimizer.calcite.CalciteSemanticException: > Unsupported > at > org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genLogicalPlan(CalcitePlanner.java:3136) > at > org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:940) > at > org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:894) > at org.apache.calcite.tools.Frameworks$1.apply(Frameworks.java:113) > at > org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:969) > at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:149) > at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:106) > at > org.apache.hadoop.hive.ql.parse.CalcitePlanner.getOptimizedAST(CalcitePlanner.java:712) > at > org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:280) > at > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10795) > at > org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:239) > at > org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:250) > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:438) > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:329) > at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1159) > at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1146) > at > org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:191) > at > org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:276) > at > org.apache.hive.service.cli.operation.Operation.run(Operation.java:324) > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:464) > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:451) > at > org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:295) > at > org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:509) > at > org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1437) > at > org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1422) > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) > at > org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56) > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14138) CBO failed for select current_database()
[ https://issues.apache.org/jira/browse/HIVE-14138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Vary updated HIVE-14138: -- Component/s: CBO > CBO failed for select current_database() > > > Key: HIVE-14138 > URL: https://issues.apache.org/jira/browse/HIVE-14138 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Peter Vary >Priority: Minor > > When issuing the following query, with hive.cbo.enable set to true: > select current_database(); > The following exception is printed to the Hiveserver2 logs: > 2016-06-30T09:58:24,146 ERROR [HiveServer2-Handler-Pool: Thread-33] > parse.CalcitePlanner: CBO failed, skipping CBO. > org.apache.hadoop.hive.ql.optimizer.calcite.CalciteSemanticException: > Unsupported > at > org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genLogicalPlan(CalcitePlanner.java:3136) > at > org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:940) > at > org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:894) > at org.apache.calcite.tools.Frameworks$1.apply(Frameworks.java:113) > at > org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:969) > at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:149) > at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:106) > at > org.apache.hadoop.hive.ql.parse.CalcitePlanner.getOptimizedAST(CalcitePlanner.java:712) > at > org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:280) > at > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10795) > at > org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:239) > at > org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:250) > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:438) > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:329) > at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1159) > at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1146) > at > org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:191) > at > org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:276) > at > org.apache.hive.service.cli.operation.Operation.run(Operation.java:324) > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:464) > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:451) > at > org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:295) > at > org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:509) > at > org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1437) > at > org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1422) > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) > at > org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56) > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14123) Add beeline configuration option to show database in the prompt
[ https://issues.apache.org/jira/browse/HIVE-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Vary updated HIVE-14123: -- Attachment: HIVE-14123.2.patch Adding patch again, since Jenkins job is not started, and a small modification is there as well > Add beeline configuration option to show database in the prompt > --- > > Key: HIVE-14123 > URL: https://issues.apache.org/jira/browse/HIVE-14123 > Project: Hive > Issue Type: Improvement > Components: Beeline, CLI >Affects Versions: 2.2.0 >Reporter: Peter Vary >Assignee: Peter Vary >Priority: Minor > Attachments: HIVE-14123.2.patch, HIVE-14123.patch > > > There are several jira issues complaining that, the Beeline does not respect > hive.cli.print.current.db. > This is partially true, since in embedded mode, it uses the > hive.cli.print.current.db to change the prompt, since HIVE-10511. > In remote mode, I think this function should use a beeline command line > option instead, like for the showHeader option emphasizing, that this is a > client side option. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HIVE-14100) current_user() returns invalid information
[ https://issues.apache.org/jira/browse/HIVE-14100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Vary reassigned HIVE-14100: - Assignee: Peter Vary > current_user() returns invalid information > -- > > Key: HIVE-14100 > URL: https://issues.apache.org/jira/browse/HIVE-14100 > Project: Hive > Issue Type: Bug > Components: Authentication, Beeline >Reporter: Peter Vary >Assignee: Peter Vary >Priority: Minor > > Using HadoopDeaultAuthenticator the current_user() returns the username of > the unix user running hiveservice2. > Using SessionStateAuthenticator the current_user returns the username which > is provided when the connection started. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13945) Decimal value is displayed as rounded when selecting where clause with that decimal value.
[
https://issues.apache.org/jira/browse/HIVE-13945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15356657#comment-15356657
]
Hive QA commented on HIVE-13945:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12815260/HIVE-13945.05.patch
{color:green}SUCCESS:{color} +1 due to 9 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10288 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ansi_sql_arithmetic
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_partitioned
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_all
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_join
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/321/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/321/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-321/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12815260 - PreCommit-HIVE-MASTER-Build
> Decimal value is displayed as rounded when selecting where clause with that
> decimal value.
> --
>
> Key: HIVE-13945
> URL: https://issues.apache.org/jira/browse/HIVE-13945
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: Takahiko Saito
>Assignee: Sergey Shelukhin
>Priority: Critical
> Attachments: HIVE-13945.01.patch, HIVE-13945.02.patch,
> HIVE-13945.03.patch, HIVE-13945.03.patch, HIVE-13945.04.patch,
> HIVE-13945.05.patch, HIVE-13945.patch
>
>
> Create a table withe a column of decimal type(38,18) and insert
> '4327269606205.029297'. Then select with that value displays its rounded
> value, which is 4327269606205.029300
> {noformat}
> 0: jdbc:hive2://os-r7-mvjkcu-hiveserver2-11-4> drop table if exists test;
> No rows affected (0.229 seconds)
> 0: jdbc:hive2://os-r7-mvjkcu-hiveserver2-11-4>
> 0: jdbc:hive2://os-r7-mvjkcu-hiveserver2-11-4> create table test (dc
> decimal(38,18));
> No rows affected (0.125 seconds)
> 0: jdbc:hive2://os-r7-mvjkcu-hiveserver2-11-4>
> 0: jdbc:hive2://os-r7-mvjkcu-hiveserver2-11-4> insert into table test values
> (4327269606205.029297);
> No rows affected (2.372 seconds)
> 0: jdbc:hive2://os-r7-mvjkcu-hiveserver2-11-4>
> 0: jdbc:hive2://os-r7-mvjkcu-hiveserver2-11-4> select * from test;
> +---+--+
> | test.dc |
> +---+--+
> | 4327269606205.029297 |
> +---+--+
> 1 row selected (0.123 seconds)
> 0: jdbc:hive2://os-r7-mvjkcu-hiveserver2-11-4>
> 0: jdbc:hive2://os-r7-mvjkcu-hiveserver2-11-4> select * from test where dc =
> 4327269606205.029297;
> +---+--+
> | test.dc |
> +---+--+
> | 4327269606205.029300 |
> +---+--+
> 1 row selected (0.109 seconds)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
