Re: [VOTE] Apache Hive 2.3.9 Release Candidate 0
Thank you, Chao. +1 according to the test result in Apache Spark. https://github.com/apache/spark/pull/32750 Bests, Dongjoon. On 2021/06/02 01:02:16, Chao Sun wrote: > Apache Hive 2.3.9 Release Candidate 0 is available here: > https://people.apache.org/~sunchao/apache-hive-2.3.9-rc-0/ > Maven artifacts are available here: > https://repository.apache.org/content/repositories/orgapachehive-1106/ > The tag release-2.3.9-rc0 has been applied to the source for this > release in github, you can see it at > https://github.com/apache/hive/tree/release-2.3.9-rc0 > Voting will conclude in 72 hours (or whenever I scrounge together enough > votes). > > Hive PMC Members: Please test and vote. > > Thanks. >
[jira] [Created] (HIVE-25198) CTAS external table with camelcase & HMS translation ON is returning 0 records
Naresh P R created HIVE-25198:
-
Summary: CTAS external table with camelcase & HMS translation ON
is returning 0 records
Key: HIVE-25198
URL: https://issues.apache.org/jira/browse/HIVE-25198
Project: Hive
Issue Type: Bug
Reporter: Naresh P R
Assignee: Naresh P R
create external table TarGet as select * from source;
Above query creates tableLocation with CamelCase if HMS Translation is ON,
whereas MoveTask will use lowerCase table path.
eg.,
{code:java}
==> Desc formatted target <==
Location: hdfs:///warehouse/tablespace/external/hive/test.db/TarGet
==> MoveTask <==
INFO : Moving data to directory
hdfs:///warehouse/tablespace/external/hive/test.db/target from
hdfs:///warehouse/tablespace/external/hive/test.db/.hive-staging_hive_2021-06-04_03-02-36_272_669287187808252905-12/-ext-10002
==> HMS Translation <==
2021-06-04 03:02:45,772 INFO
org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer:
[pool-7-thread-8]: Transformer returning table:Table(tableName:TarGet,
dbName:test, owner:hive, createTime:1622775765, lastAccessTime:0, retention:0,
sd:StorageDescriptor(cols:[FieldSchema(name:id, type:varchar(10),
comment:null)], location:
hdfs:///warehouse/tablespace/external/hive/ajay.db/TarGet,{code}
Select query after CTAS will return 0 rows because of this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HIVE-25197) logger level doesn't effect when set on hive-cli starting command
Spongebob created HIVE-25197: Summary: logger level doesn't effect when set on hive-cli starting command Key: HIVE-25197 URL: https://issues.apache.org/jira/browse/HIVE-25197 Project: Hive Issue Type: Bug Components: CLI Affects Versions: 3.1.2 Environment: hive: 3.1.2 Reporter: Spongebob Attachments: image-2021-06-04-09-43-21-502.png I am trying to start hive via " hive --hiveconf hive.root.logger=ERROR,DRFA" but it doesn't effect, how can I hide these info log when using hive cli ? !image-2021-06-04-09-43-21-502.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-25196) Native Vectorization of GenericUDFSplit function
Matt McCline created HIVE-25196: --- Summary: Native Vectorization of GenericUDFSplit function Key: HIVE-25196 URL: https://issues.apache.org/jira/browse/HIVE-25196 Project: Hive Issue Type: Improvement Reporter: Matt McCline Assignee: Matt McCline Provide faster 'split' function for vector-mode. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-25195) Store Iceberg write commit and ctas information in QueryState
Marton Bod created HIVE-25195: - Summary: Store Iceberg write commit and ctas information in QueryState Key: HIVE-25195 URL: https://issues.apache.org/jira/browse/HIVE-25195 Project: Hive Issue Type: Improvement Reporter: Marton Bod Assignee: Marton Bod We should replace the current method of passing Iceberg write commit-related information (jobID, task num) and CTAS info via the session conf using prefixed keys. We have a new way of doing that more cleanly, using the QueryState object. This should make the code easier to maintain and guard against accidental session conf pollution. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-25194) Add support for STORED AS ORC/PARQUET/AVRO for Iceberg
László Pintér created HIVE-25194: Summary: Add support for STORED AS ORC/PARQUET/AVRO for Iceberg Key: HIVE-25194 URL: https://issues.apache.org/jira/browse/HIVE-25194 Project: Hive Issue Type: New Feature Reporter: László Pintér Assignee: László Pintér Currently we have to specify the fileformat in TBLPROPERTIES during Iceberg create table statements. The ideal syntax would be: CREATE TABLE tbl STORED BY ICEBERG STORED AS ORC ... One complication is that currently stored by and stored as are not permitted within the same query, so that needs to be amended. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-25193) Vectorized Query Execution: ClassCastException when use nvl() function which default_value is decimal type
qiang.bi created HIVE-25193:
---
Summary: Vectorized Query Execution: ClassCastException when use
nvl() function which default_value is decimal type
Key: HIVE-25193
URL: https://issues.apache.org/jira/browse/HIVE-25193
Project: Hive
Issue Type: Bug
Components: Vectorization
Affects Versions: 4.0.0
Reporter: qiang.bi
Assignee: qiang.bi
Problem statement:
{code:java}
set hive.vectorized.execution.enabled = true;
select nvl(get_json_object(attr_json,'$.correctedPrice'),0.88) corrected_price,
from dw_mdm_sync_asset;
{code}
The error log:
{code:java}
[2021-05-24 08:06:05.627]], TaskAttempt 3 failed, info=[Error: Error while
running task ( failure ) :
attempt_1619882873092_4567_1_03_00_3:java.lang.RuntimeException:
java.lang.RuntimeException: Hive Runtime Error while closing
operators[2021-05-24 08:06:05.627]], TaskAttempt 3 failed, info=[Error: Error
while running task ( failure ) :
attempt_1619882873092_4567_1_03_00_3:java.lang.RuntimeException:
java.lang.RuntimeException: Hive Runtime Error while closing
operators[2021-05-24 08:06:05.628] at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)[2021-05-24
08:06:05.628] at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)[2021-05-24
08:06:05.628] at
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)[2021-05-24
08:06:05.628] at
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)[2021-05-24
08:06:05.628] at
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)[2021-05-24
08:06:05.628] at java.security.AccessController.doPrivileged(Native
Method)[2021-05-24 08:06:05.628] at
javax.security.auth.Subject.doAs(Subject.java:422)[2021-05-24 08:06:05.628] at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)[2021-05-24
08:06:05.628] at
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)[2021-05-24
08:06:05.628] at
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)[2021-05-24
08:06:05.628] at
org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)[2021-05-24
08:06:05.628] at
com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)[2021-05-24
08:06:05.628] at
com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69)[2021-05-24
08:06:05.628] at
com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)[2021-05-24
08:06:05.628] at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)[2021-05-24
08:06:05.628] at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)[2021-05-24
08:06:05.628] at java.lang.Thread.run(Thread.java:748)[2021-05-24
08:06:05.628]Caused by: java.lang.RuntimeException: Hive Runtime Error while
closing operators[2021-05-24 08:06:05.628] at
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.close(MapRecordProcessor.java:495)[2021-05-24
08:06:05.628] at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:284)[2021-05-24
08:06:05.628] ... 16 more[2021-05-24 08:06:05.628]Caused by:
org.apache.hadoop.hive.ql.metadata.HiveException:
org.apache.hadoop.hive.ql.metadata.HiveException:
org.apache.hadoop.hive.ql.metadata.HiveException:
org.apache.hadoop.hive.ql.metadata.HiveException: Error evaluating
null[2021-05-24 08:06:05.628] at
org.apache.hadoop.hive.ql.exec.MapJoinOperator.closeOp(MapJoinOperator.java:621)[2021-05-24
08:06:05.628] at
org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinGenerateResultOperator.closeOp(VectorMapJoinGenerateResultOperator.java:681)[2021-05-24
08:06:05.628] at
org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:732)[2021-05-24
08:06:05.628] at
org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:756)[2021-05-24
08:06:05.628] at
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.close(MapRecordProcessor.java:484)[2021-05-24
08:06:05.628] ... 17 more[2021-05-24 08:06:05.628]Caused by:
org.apache.hadoop.hive.ql.metadata.HiveException:
org.apache.hadoop.hive.ql.metadata.HiveException:
org.apache.hadoop.hive.ql.metadata.HiveException: Error evaluating
null[2021-05-24 08:06:05.628] at
org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinGenerateResultOperator.reProcessBigTable(VectorMapJoinGenerateResultOperator.java:609)[2021-05-24
08:06:05.628] at
org.apache.hadoop.hive.ql.exec.MapJoinOperator.continueProcess(MapJoinOperator.java:681)[2021-05-24
08:06:05.628] at
[jira] [Created] (HIVE-25192) No need to create table directory for the non-native table
Zhihua Deng created HIVE-25192: -- Summary: No need to create table directory for the non-native table Key: HIVE-25192 URL: https://issues.apache.org/jira/browse/HIVE-25192 Project: Hive Issue Type: Bug Reporter: Zhihua Deng When creating non-native tables like kudu, hbase and so on, we always create a warehouse location for these tables, though these tables may not use the location to store data or for job plan, so there is no need to create such location. We also should skip getting the input summary of non-native tables in some cases, this will avoid oom problem of building the hash table when the non-native table is on build side. -- This message was sent by Atlassian Jira (v8.3.4#803005)
