Re: [ANNOUNCE] Apache Hive 2.3.5 Released
It appears I made some mistake in the upload process, as I didn't see it even in Apache's repo. I've redone the upload and it's showing up in Apache's repo now. Not sure how long before it replicates to maven central. Alan. On Wed, May 15, 2019 at 7:24 PM Wang, Yuming wrote: > Thank you, Alan. > > The Hive 2.3.5 jars is not in the maven repository.< > https://repo.maven.apache.org/maven2/org/apache/hive/hive-exec/> Do we > need to wait for a while? > > > > > > On 16/05/2019, 02:43, "Alan Gates" wrote: > > > > The Apache Hive team is proud to announce the release of Apache Hive > > version 2.3.5. > > > > The Apache Hive (TM) data warehouse software facilitates querying and > > managing large datasets residing in distributed storage. Built on top > > of Apache Hadoop (TM), it provides, among others: > > > > * Tools to enable easy data extract/transform/load (ETL) > > > > * Interactive query over terabytes sized datasets > > > > * A mechanism to impose structure on a variety of data formats > > > > * Access to files stored either directly in Apache HDFS (TM) or in > other > > data storage systems such as Apache HBase (TM) > > > > * Query execution via Apache Hadoop MapReduce, Apache Tez and Apache > Spark > > frameworks. > > > > For Hive release details and downloads, please visit: > > > https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhive.apache.org%2Fdownloads.htmldata=02%7C01%7Cyumwang%40ebay.com%7Cdfa8e1a7af9c4c2e68e008d6d965316b%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C636935425976809307sdata=bfAaYbr%2FWyiuVQlAz0pfWG%2Bh%2FB9%2Fn0v1N4GaGseN9mU%3Dreserved=0 > > > > Hive 2.3.5 Release Notes are available here: > > > https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fsecure%2FReleaseNote.jspa%3Fversion%3D12345394%26styleName%3DText%26projectId%3D12310843data=02%7C01%7Cyumwang%40ebay.com%7Cdfa8e1a7af9c4c2e68e008d6d965316b%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C636935425976809307sdata=14Xgmb70H1A6CovUzwA2Koyz%2FNHGCejfPIFDQyQRR6g%3Dreserved=0 > > > > We would like to thank the many contributors who made this release > > possible. > > > > Regards, > > > > The Apache Hive Team > > >
[jira] [Created] (HIVE-21746) ArrayIndexOutOfBoundsException during dynamically partitioned hash join, with CBO disabled
Jason Dere created HIVE-21746:
-
Summary: ArrayIndexOutOfBoundsException during dynamically
partitioned hash join, with CBO disabled
Key: HIVE-21746
URL: https://issues.apache.org/jira/browse/HIVE-21746
Project: Hive
Issue Type: Bug
Components: Query Planning
Reporter: Jason Dere
Assignee: Jason Dere
ArrayIndexOutOfBounds exception during query execution with dynamically
partitioned hash join.
Found on Hive 2.x. Seems to occur with CBO disabled/failed.
Disabling constant propagation seems to allow the query to succeed.
{noformat}
java.lang.ArrayIndexOutOfBoundsException: 203
at
org.apache.hadoop.hive.serde2.io.TimestampWritable.getTotalLength(TimestampWritable.java:217)
~[hive-exec-2.1.0.2.6.4.119-3.jar:2.1.0.2.6.4.119-3]
at
org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryUtils.checkObjectByteInfo(LazyBinaryUtils.java:205)
~[hive-exec-2.1.0.2.6.4.119-3.jar:2.1.0.2.6.4.119-3]
at
org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct.parse(LazyBinaryStruct.java:142)
~[hive-exec-2.1.0.2.6.4.119-3.jar:2.1.0.2.6.4.119-3]
at
org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct.getFieldsAsList(LazyBinaryStruct.java:281)
~[hive-exec-2.1.0.2.6.4.119-3.jar:2.1.0.2.6.4.119-3]
at
org.apache.hadoop.hive.ql.exec.persistence.MapJoinBytesTableContainer$ReusableRowContainer.unpack(MapJoinBytesTableContainer.java:744)
~[hive-exec-2.1.0.2.6.4.119-3.jar:2.1.0.2.6.4.119-3]
at
org.apache.hadoop.hive.ql.exec.persistence.MapJoinBytesTableContainer$ReusableRowContainer.next(MapJoinBytesTableContainer.java:730)
~[hive-exec-2.1.0.2.6.4.119-3.jar:2.1.0.2.6.4.119-3]
at
org.apache.hadoop.hive.ql.exec.persistence.MapJoinBytesTableContainer$ReusableRowContainer.next(MapJoinBytesTableContainer.java:605)
~[hive-exec-2.1.0.2.6.4.119-3.jar:2.1.0.2.6.4.119-3]
at
org.apache.hadoop.hive.ql.exec.persistence.UnwrapRowContainer.next(UnwrapRowContainer.java:70)
~[hive-exec-2.1.0.2.6.4.119-3.jar:2.1.0.2.6.4.119-3]
at
org.apache.hadoop.hive.ql.exec.persistence.UnwrapRowContainer.next(UnwrapRowContainer.java:34)
~[hive-exec-2.1.0.2.6.4.119-3.jar:2.1.0.2.6.4.119-3]
at
org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genUniqueJoinObject(CommonJoinOperator.java:819)
~[hive-exec-2.1.0.2.6.4.119-3.jar:2.1.0.2.6.4.119-3]
at
org.apache.hadoop.hive.ql.exec.CommonJoinOperator.checkAndGenObject(CommonJoinOperator.java:924)
~[hive-exec-2.1.0.2.6.4.119-3.jar:2.1.0.2.6.4.119-3]
at
org.apache.hadoop.hive.ql.exec.MapJoinOperator.process(MapJoinOperator.java:456)
~[hive-exec-2.1.0.2.6.4.119-3.jar:2.1.0.2.6.4.119-3]
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:359)
~[hive-exec-2.1.0.2.6.4.119-3.jar:2.1.0.2.6.4.119-3]
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:290)
~[hive-exec-2.1.0.2.6.4.119-3.jar:2.1.0.2.6.4.119-3]
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:319)
~[hive-exec-2.1.0.2.6.4.119-3.jar:2.1.0.2.6.4.119-3]
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:189)
~[hive-exec-2.1.0.2.6.4.119-3.jar:2.1.0.2.6.4.119-3]
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:172)
~[hive-exec-2.1.0.2.6.4.119-3.jar:2.1.0.2.6.4.119-3]
at
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:377)
~[tez-runtime-internals-0.8.4.2.6.4.119-3.jar:0.8.4.2.6.4.119-3]
at
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
~[tez-runtime-internals-0.8.4.2.6.4.119-3.jar:0.8.4.2.6.4.119-3]
at
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
~[tez-runtime-internals-0.8.4.2.6.4.119-3.jar:0.8.4.2.6.4.119-3]
at java.security.AccessController.doPrivileged(Native Method)
~[?:1.8.0_112]
at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_112]
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
~[hadoop-common-2.7.3.2.6.4.119-3.jar:?]
at
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
~[tez-runtime-internals-0.8.4.2.6.4.119-3.jar:0.8.4.2.6.4.119-3]
at
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
~[tez-runtime-internals-0.8.4.2.6.4.119-3.jar:0.8.4.2.6.4.119-3]
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
~[tez-common-0.8.4.2.6.4.119-3.jar:0.8.4.2.6.4.119-3]
at
org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:118)
[jira] [Created] (HIVE-21745) Change in join order causes query parse to fail
Andre Araujo created HIVE-21745:
---
Summary: Change in join order causes query parse to fail
Key: HIVE-21745
URL: https://issues.apache.org/jira/browse/HIVE-21745
Project: Hive
Issue Type: Bug
Components: Hive
Affects Versions: 1.1.0
Reporter: Andre Araujo
I ran into the following case, where a query fails to parse if the join order
is changed:
{code}
reate database if not exists test;
drop table if exists test.table1;
create table test.table1 (
id string,
col_a string
)
stored as textfile;
drop table if exists test.table2;
create table test.table2 (
id string
)
stored as textfile;
drop table if exists test.table3;
create table test.table3 (
col_a string,
col_b string
)
stored as textfile;
drop table if exists test.table4;
create table test.table4 (
id string
)
stored as textfile;
-- This fails with: Invalid table alias or column reference 't3': (possible
column names are: id, col_a)
drop view if exists test.v;
create view test.v as
select
1
from
test.table1 as t1
left join test.table2 as t2 on t2.id = t1.id
left join test.table3 as t3 on t1.col_a = t3.col_a
left join test.table4 as t4 on t1.id = t4.id and t3.col_b = 'X'
;
-- This works
drop view if exists test.v;
create view test.v as
select
1
from
test.table1 as t1
left join test.table3 as t3 on t1.col_a = t3.col_a
left join test.table4 as t4 on t1.id = t4.id and t3.col_b = 'X'
left join test.table2 as t2 on t2.id = t1.id
;
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HIVE-21744) Make hive side changes to enforce table access type on queries.
Naveen Gangam created HIVE-21744: Summary: Make hive side changes to enforce table access type on queries. Key: HIVE-21744 URL: https://issues.apache.org/jira/browse/HIVE-21744 Project: Hive Issue Type: Sub-task Affects Versions: 4.0.0 Reporter: Naveen Gangam -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-21743) day( ) gives wrong day from the date in Apache Hive 3.1 server
Adarshdeep Cheema created HIVE-21743: Summary: day( ) gives wrong day from the date in Apache Hive 3.1 server Key: HIVE-21743 URL: https://issues.apache.org/jira/browse/HIVE-21743 Project: Hive Issue Type: Bug Components: Hive Affects Versions: 3.0.0 Environment: Server: Apache Hive 3.1 Driver hive-jdbc-3.1.0.3.1.0.0-78 Reporter: Adarshdeep Cheema Fix For: 3.1.2 Using Apache Hive 3.1 server Run the following SQL and you will get 3 instead of i. SELECT (day( DATE '0001-01-01')) FROM `tabke` PLEASE NOTE THIS DOES NOT HAPPEN WITH Apache HIVE 2.1 SERVER -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-21742) Vectorization: CASE result type casting
Gopal V created HIVE-21742:
--
Summary: Vectorization: CASE result type casting
Key: HIVE-21742
URL: https://issues.apache.org/jira/browse/HIVE-21742
Project: Hive
Issue Type: Bug
Components: Logical Optimizer, Vectorization
Affects Versions: 3.1.1
Reporter: Gopal V
{code}
create temporary table foo(q548284 int);
insert into foo values(1),(2),(3),(4),(5),(6);
select q548284, CASE WHEN ((q548284 = 1)) THEN (0.2) WHEN ((q548284 = 2)) THEN
(0.4) WHEN ((q548284 = 3)) THEN (0.6) WHEN ((q548284 = 4)) THEN (0.8) WHEN
((q548284 = 5)) THEN (1) ELSE (null) END from foo order by q548284 limit 1;
{code}
Fails with
{code}
Caused by: java.lang.ClassCastException:
org.apache.hadoop.hive.ql.exec.vector.LongColumnVector cannot be cast to
org.apache.hadoop.hive.ql.exec.vector.DecimalColumnVector
at
org.apache.hadoop.hive.ql.exec.vector.DecimalColumnVector.setElement(DecimalColumnVector.java:130)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprColumnNull.evaluate(IfExprColumnNull.java:101)
{code}
This gets fixed if the case return of (1) is turned into a (1.0).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HIVE-21741) Backport metastore SQL commits to branch-3
David Lavati created HIVE-21741: --- Summary: Backport metastore SQL commits to branch-3 Key: HIVE-21741 URL: https://issues.apache.org/jira/browse/HIVE-21741 Project: Hive Issue Type: Bug Reporter: David Lavati Assignee: David Lavati This is an umbrella for backporting the following metastore-related tickets to branch-3: * HIVE-20221 * HIVE-20833 * HIVE-21404 * HIVE-21462 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-21740) Collect LLAP execution latency metrics
Peter Vary created HIVE-21740: - Summary: Collect LLAP execution latency metrics Key: HIVE-21740 URL: https://issues.apache.org/jira/browse/HIVE-21740 Project: Hive Issue Type: New Feature Reporter: Peter Vary Assignee: Peter Vary Collect metrics for LLAP task execution times -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-21739) Make metastore DB backward compatible with pre-catalog versions of hive.
Aditya Shah created HIVE-21739:
--
Summary: Make metastore DB backward compatible with pre-catalog
versions of hive.
Key: HIVE-21739
URL: https://issues.apache.org/jira/browse/HIVE-21739
Project: Hive
Issue Type: Sub-task
Affects Versions: 2.1.1, 1.2.0
Reporter: Aditya Shah
Assignee: Aditya Shah
Since the addition of foreign key constraint between Database ('DBS') table and
catalogs ('CTLGS') table in HIVE-18755 we are able to run a simple create
database command with an older version of Metastore Server. This is due to
older versions having JDO schema as per older schema of 'DBS' which did not
have an additional 'CTLG_NAME' column.
The error is as follows:
{code:java}
org.apache.hadoop.hive.ql.metadata.HiveException:
MetaException(message:Exception thrown flushing changes to datastore)
java.sql.BatchUpdateException: Cannot add or update a child row: a foreign key
constraint fails ("metastore_1238"."DBS", CONSTRAINT "CTLG_FK1" FOREIGN KEY
("CTLG_NAME") REFERENCES "CTLGS" ("NAME"))
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HIVE-21738) TestReplAcidTablesBootstrapWithJsonMessage#testBootstrapAcidTablesDuringIncrementalWithConcurrentWrites is flaky
Laszlo Bodor created HIVE-21738: --- Summary: TestReplAcidTablesBootstrapWithJsonMessage#testBootstrapAcidTablesDuringIncrementalWithConcurrentWrites is flaky Key: HIVE-21738 URL: https://issues.apache.org/jira/browse/HIVE-21738 Project: Hive Issue Type: Bug Reporter: Laszlo Bodor -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-21737) Upgrade Avro to version 1.9.0
Ismaël Mejía created HIVE-21737: --- Summary: Upgrade Avro to version 1.9.0 Key: HIVE-21737 URL: https://issues.apache.org/jira/browse/HIVE-21737 Project: Hive Issue Type: Improvement Components: Hive Reporter: Ismaël Mejía Avro 0.9.0 was released recently. It brings a lot of fixes including a leaner version of Avro without Jackson in the public API. Worth the update. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
