[jira] [Updated] (HIVE-9786) CBO (Calcite Return Path): HiveJoinAddNotNullRule causes StackOverflowError [CBO branch]
[ https://issues.apache.org/jira/browse/HIVE-9786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-9786: -- Attachment: HIVE-9786.cbo.patch [~ashutoshc], this patch solves the StackOverflowError. Thanks CBO (Calcite Return Path): HiveJoinAddNotNullRule causes StackOverflowError [CBO branch] Key: HIVE-9786 URL: https://issues.apache.org/jira/browse/HIVE-9786 Project: Hive Issue Type: Sub-task Components: CBO Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: 1.2.0 Attachments: HIVE-9786.cbo.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9571) CBO (Calcite Return Path): Generate FileSink Op [CBO branch]
[ https://issues.apache.org/jira/browse/HIVE-9571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14336420#comment-14336420 ] Jesus Camacho Rodriguez commented on HIVE-9571: --- The generation of FS went in with HIVE-9581. CBO (Calcite Return Path): Generate FileSink Op [CBO branch] Key: HIVE-9571 URL: https://issues.apache.org/jira/browse/HIVE-9571 Project: Hive Issue Type: Sub-task Components: CBO Affects Versions: cbo-branch Reporter: Laljo John Pullokkaran Assignee: Laljo John Pullokkaran Fix For: 1.2.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9781) Utilize spark.kryo.registrator [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-9781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14336683#comment-14336683
]
Jimmy Xiang commented on HIVE-9781:
---
Cool. Let me handle the other place which doesn't use this setting.
Utilize spark.kryo.registrator [Spark Branch]
-
Key: HIVE-9781
URL: https://issues.apache.org/jira/browse/HIVE-9781
Project: Hive
Issue Type: Sub-task
Components: Spark
Reporter: Brock Noland
Assignee: Jimmy Xiang
I noticed in several thread dumps that it appears kyro is serializing the
class names associated with our keys and values.
Kyro supports pre-registering classes so that you don't have to serialize the
class name and spark supports this via the {{spark.kryo.registrator}}
property. We should do this so we don't have to serialize class names.
{noformat}
Thread 12154: (state = BLOCKED)
- java.lang.Object.hashCode() @bci=0 (Compiled frame; information may be
imprecise)
- com.esotericsoftware.kryo.util.ObjectMap.get(java.lang.Object) @bci=1,
line=265 (Compiled frame)
-
com.esotericsoftware.kryo.util.DefaultClassResolver.getRegistration(java.lang.Class)
@bci=18, line=61 (Compiled frame)
- com.esotericsoftware.kryo.Kryo.getRegistration(java.lang.Class) @bci=20,
line=429 (Compiled frame)
-
com.esotericsoftware.kryo.util.DefaultClassResolver.readName(com.esotericsoftware.kryo.io.Input)
@bci=242, line=148 (Compiled frame)
-
com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(com.esotericsoftware.kryo.io.Input)
@bci=65, line=115 (Compiled frame)
-
com.esotericsoftware.kryo.Kryo.readClass(com.esotericsoftware.kryo.io.Input)
@bci=20, line=610 (Compiled frame)
-
com.esotericsoftware.kryo.Kryo.readClassAndObject(com.esotericsoftware.kryo.io.Input)
@bci=21, line=721 (Compiled frame)
- com.twitter.chill.Tuple2Serializer.read(com.esotericsoftware.kryo.Kryo,
com.esotericsoftware.kryo.io.Input, java.lang.Class) @bci=6, line=41
(Compiled frame)
- com.twitter.chill.Tuple2Serializer.read(com.esotericsoftware.kryo.Kryo,
com.esotericsoftware.kryo.io.Input, java.lang.Class) @bci=4, line=33
(Compiled frame)
-
com.esotericsoftware.kryo.Kryo.readClassAndObject(com.esotericsoftware.kryo.io.Input)
@bci=126, line=729 (Compiled frame)
-
org.apache.spark.serializer.KryoDeserializationStream.readObject(scala.reflect.ClassTag)
@bci=8, line=142 (Compiled frame)
- org.apache.spark.serializer.DeserializationStream$$anon$1.getNext()
@bci=10, line=133 (Compiled frame)
- org.apache.spark.util.NextIterator.hasNext() @bci=16, line=71 (Compiled
frame)
- org.apache.spark.util.CompletionIterator.hasNext() @bci=4, line=32
(Compiled frame)
- scala.collection.Iterator$$anon$13.hasNext() @bci=4, line=371 (Compiled
frame)
- org.apache.spark.util.CompletionIterator.hasNext() @bci=4, line=32
(Compiled frame)
- org.apache.spark.InterruptibleIterator.hasNext() @bci=22, line=39
(Compiled frame)
- scala.collection.Iterator$$anon$11.hasNext() @bci=4, line=327 (Compiled
frame)
-
org.apache.spark.util.collection.ExternalSorter.insertAll(scala.collection.Iterator)
@bci=191, line=217 (Compiled frame)
- org.apache.spark.shuffle.hash.HashShuffleReader.read() @bci=278, line=61
(Interpreted frame)
- org.apache.spark.rdd.ShuffledRDD.compute(org.apache.spark.Partition,
org.apache.spark.TaskContext) @bci=46, line=92 (Interpreted frame)
-
org.apache.spark.rdd.RDD.computeOrReadCheckpoint(org.apache.spark.Partition,
org.apache.spark.TaskContext) @bci=26, line=263 (Interpreted frame)
- org.apache.spark.rdd.RDD.iterator(org.apache.spark.Partition,
org.apache.spark.TaskContext) @bci=33, line=230 (Interpreted frame)
- org.apache.spark.rdd.MapPartitionsRDD.compute(org.apache.spark.Partition,
org.apache.spark.TaskContext) @bci=24, line=35 (Interpreted frame)
-
org.apache.spark.rdd.RDD.computeOrReadCheckpoint(org.apache.spark.Partition,
org.apache.spark.TaskContext) @bci=26, line=263 (Interpreted frame)
- org.apache.spark.rdd.RDD.iterator(org.apache.spark.Partition,
org.apache.spark.TaskContext) @bci=33, line=230 (Interpreted frame)
- org.apache.spark.rdd.MapPartitionsRDD.compute(org.apache.spark.Partition,
org.apache.spark.TaskContext) @bci=24, line=35 (Interpreted frame)
-
org.apache.spark.rdd.RDD.computeOrReadCheckpoint(org.apache.spark.Partition,
org.apache.spark.TaskContext) @bci=26, line=263 (Interpreted frame)
- org.apache.spark.rdd.RDD.iterator(org.apache.spark.Partition,
org.apache.spark.TaskContext) @bci=33, line=230 (Interpreted frame)
- org.apache.spark.rdd.UnionRDD.compute(org.apache.spark.Partition,
org.apache.spark.TaskContext) @bci=22, line=87 (Interpreted frame)
-
org.apache.spark.rdd.RDD.computeOrReadCheckpoint(org.apache.spark.Partition,
[jira] [Assigned] (HIVE-9781) Utilize spark.kryo.registrator [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-9781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jimmy Xiang reassigned HIVE-9781:
-
Assignee: Jimmy Xiang
Utilize spark.kryo.registrator [Spark Branch]
-
Key: HIVE-9781
URL: https://issues.apache.org/jira/browse/HIVE-9781
Project: Hive
Issue Type: Sub-task
Components: Spark
Reporter: Brock Noland
Assignee: Jimmy Xiang
I noticed in several thread dumps that it appears kyro is serializing the
class names associated with our keys and values.
Kyro supports pre-registering classes so that you don't have to serialize the
class name and spark supports this via the {{spark.kryo.registrator}}
property. We should do this so we don't have to serialize class names.
{noformat}
Thread 12154: (state = BLOCKED)
- java.lang.Object.hashCode() @bci=0 (Compiled frame; information may be
imprecise)
- com.esotericsoftware.kryo.util.ObjectMap.get(java.lang.Object) @bci=1,
line=265 (Compiled frame)
-
com.esotericsoftware.kryo.util.DefaultClassResolver.getRegistration(java.lang.Class)
@bci=18, line=61 (Compiled frame)
- com.esotericsoftware.kryo.Kryo.getRegistration(java.lang.Class) @bci=20,
line=429 (Compiled frame)
-
com.esotericsoftware.kryo.util.DefaultClassResolver.readName(com.esotericsoftware.kryo.io.Input)
@bci=242, line=148 (Compiled frame)
-
com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(com.esotericsoftware.kryo.io.Input)
@bci=65, line=115 (Compiled frame)
-
com.esotericsoftware.kryo.Kryo.readClass(com.esotericsoftware.kryo.io.Input)
@bci=20, line=610 (Compiled frame)
-
com.esotericsoftware.kryo.Kryo.readClassAndObject(com.esotericsoftware.kryo.io.Input)
@bci=21, line=721 (Compiled frame)
- com.twitter.chill.Tuple2Serializer.read(com.esotericsoftware.kryo.Kryo,
com.esotericsoftware.kryo.io.Input, java.lang.Class) @bci=6, line=41
(Compiled frame)
- com.twitter.chill.Tuple2Serializer.read(com.esotericsoftware.kryo.Kryo,
com.esotericsoftware.kryo.io.Input, java.lang.Class) @bci=4, line=33
(Compiled frame)
-
com.esotericsoftware.kryo.Kryo.readClassAndObject(com.esotericsoftware.kryo.io.Input)
@bci=126, line=729 (Compiled frame)
-
org.apache.spark.serializer.KryoDeserializationStream.readObject(scala.reflect.ClassTag)
@bci=8, line=142 (Compiled frame)
- org.apache.spark.serializer.DeserializationStream$$anon$1.getNext()
@bci=10, line=133 (Compiled frame)
- org.apache.spark.util.NextIterator.hasNext() @bci=16, line=71 (Compiled
frame)
- org.apache.spark.util.CompletionIterator.hasNext() @bci=4, line=32
(Compiled frame)
- scala.collection.Iterator$$anon$13.hasNext() @bci=4, line=371 (Compiled
frame)
- org.apache.spark.util.CompletionIterator.hasNext() @bci=4, line=32
(Compiled frame)
- org.apache.spark.InterruptibleIterator.hasNext() @bci=22, line=39
(Compiled frame)
- scala.collection.Iterator$$anon$11.hasNext() @bci=4, line=327 (Compiled
frame)
-
org.apache.spark.util.collection.ExternalSorter.insertAll(scala.collection.Iterator)
@bci=191, line=217 (Compiled frame)
- org.apache.spark.shuffle.hash.HashShuffleReader.read() @bci=278, line=61
(Interpreted frame)
- org.apache.spark.rdd.ShuffledRDD.compute(org.apache.spark.Partition,
org.apache.spark.TaskContext) @bci=46, line=92 (Interpreted frame)
-
org.apache.spark.rdd.RDD.computeOrReadCheckpoint(org.apache.spark.Partition,
org.apache.spark.TaskContext) @bci=26, line=263 (Interpreted frame)
- org.apache.spark.rdd.RDD.iterator(org.apache.spark.Partition,
org.apache.spark.TaskContext) @bci=33, line=230 (Interpreted frame)
- org.apache.spark.rdd.MapPartitionsRDD.compute(org.apache.spark.Partition,
org.apache.spark.TaskContext) @bci=24, line=35 (Interpreted frame)
-
org.apache.spark.rdd.RDD.computeOrReadCheckpoint(org.apache.spark.Partition,
org.apache.spark.TaskContext) @bci=26, line=263 (Interpreted frame)
- org.apache.spark.rdd.RDD.iterator(org.apache.spark.Partition,
org.apache.spark.TaskContext) @bci=33, line=230 (Interpreted frame)
- org.apache.spark.rdd.MapPartitionsRDD.compute(org.apache.spark.Partition,
org.apache.spark.TaskContext) @bci=24, line=35 (Interpreted frame)
-
org.apache.spark.rdd.RDD.computeOrReadCheckpoint(org.apache.spark.Partition,
org.apache.spark.TaskContext) @bci=26, line=263 (Interpreted frame)
- org.apache.spark.rdd.RDD.iterator(org.apache.spark.Partition,
org.apache.spark.TaskContext) @bci=33, line=230 (Interpreted frame)
- org.apache.spark.rdd.UnionRDD.compute(org.apache.spark.Partition,
org.apache.spark.TaskContext) @bci=22, line=87 (Interpreted frame)
-
org.apache.spark.rdd.RDD.computeOrReadCheckpoint(org.apache.spark.Partition,
org.apache.spark.TaskContext) @bci=26, line=263 (Interpreted frame)
-
[jira] [Commented] (HIVE-9086) Add language support to PURGE data while dropping partitions.
[ https://issues.apache.org/jira/browse/HIVE-9086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14336585#comment-14336585 ] Timothy Driscoll commented on HIVE-9086: The way it is documented seems to be more in line with how the DROP TABLE syntax is formatted, which is why I brought it up. We've applied the patch (along with HIVE-9083) and it seems to be working as expected. I just wanted to validate (to whatever extent possible) that the patch syntax is going to be the syntax going forward before we update our scripts. Add language support to PURGE data while dropping partitions. - Key: HIVE-9086 URL: https://issues.apache.org/jira/browse/HIVE-9086 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.15.0 Reporter: Mithun Radhakrishnan Assignee: Mithun Radhakrishnan Attachments: HIVE-9086.1.patch HIVE-9083 adds metastore-support to skip-trash while dropping partitions. This patch includes language support to do the same. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HIVE-9786) CBO (Calcite Return Path): HiveJoinAddNotNullRule causes StackOverflowError [CBO branch]
[ https://issues.apache.org/jira/browse/HIVE-9786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan resolved HIVE-9786. Resolution: Fixed Fix Version/s: (was: 1.2.0) cbo-branch Committed to branch. Thanks, Jesus! CBO (Calcite Return Path): HiveJoinAddNotNullRule causes StackOverflowError [CBO branch] Key: HIVE-9786 URL: https://issues.apache.org/jira/browse/HIVE-9786 Project: Hive Issue Type: Sub-task Components: CBO Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: cbo-branch Attachments: HIVE-9786.cbo.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9413) CBO(Calcite Return Path): Translate TS, Select to Hive OP [CBO branch]
[ https://issues.apache.org/jira/browse/HIVE-9413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14336771#comment-14336771 ] Ashutosh Chauhan commented on HIVE-9413: Committed to branch. Thanks, John Jesus! CBO(Calcite Return Path): Translate TS, Select to Hive OP [CBO branch] -- Key: HIVE-9413 URL: https://issues.apache.org/jira/browse/HIVE-9413 Project: Hive Issue Type: Sub-task Components: CBO Affects Versions: cbo-branch Reporter: Laljo John Pullokkaran Assignee: Laljo John Pullokkaran Fix For: cbo-branch Attachments: HIVE-9413.cbo.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HIVE-9562) CBO(Calcite Return Path): Translate FilterRel to Hive OP [CBO branch]
[ https://issues.apache.org/jira/browse/HIVE-9562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan resolved HIVE-9562. Resolution: Fixed Fix Version/s: (was: 1.2.0) cbo-branch Committed to branch. Thanks, John Jesus! CBO(Calcite Return Path): Translate FilterRel to Hive OP [CBO branch] - Key: HIVE-9562 URL: https://issues.apache.org/jira/browse/HIVE-9562 Project: Hive Issue Type: Sub-task Components: CBO Affects Versions: cbo-branch Reporter: Laljo John Pullokkaran Assignee: Laljo John Pullokkaran Fix For: cbo-branch Attachments: HIVE-9562.cbo.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9786) CBO (Calcite Return Path): HiveJoinAddNotNullRule causes StackOverflowError [CBO branch]
[ https://issues.apache.org/jira/browse/HIVE-9786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-9786: --- Affects Version/s: cbo-branch CBO (Calcite Return Path): HiveJoinAddNotNullRule causes StackOverflowError [CBO branch] Key: HIVE-9786 URL: https://issues.apache.org/jira/browse/HIVE-9786 Project: Hive Issue Type: Sub-task Components: CBO Affects Versions: cbo-branch Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: cbo-branch Attachments: HIVE-9786.cbo.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9413) CBO(Calcite Return Path): Translate TS, Select to Hive OP [CBO branch]
[ https://issues.apache.org/jira/browse/HIVE-9413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-9413: --- Fix Version/s: (was: 1.2.0) cbo-branch CBO(Calcite Return Path): Translate TS, Select to Hive OP [CBO branch] -- Key: HIVE-9413 URL: https://issues.apache.org/jira/browse/HIVE-9413 Project: Hive Issue Type: Sub-task Components: CBO Affects Versions: cbo-branch Reporter: Laljo John Pullokkaran Assignee: Laljo John Pullokkaran Fix For: cbo-branch Attachments: HIVE-9413.cbo.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9788) Make double quote optional in tsv/csv/dsv output
[
https://issues.apache.org/jira/browse/HIVE-9788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14337066#comment-14337066
]
Brock Noland commented on HIVE-9788:
[~Ferd] - is this something you'd like to pickup?
Make double quote optional in tsv/csv/dsv output
Key: HIVE-9788
URL: https://issues.apache.org/jira/browse/HIVE-9788
Project: Hive
Issue Type: Improvement
Reporter: Brock Noland
Similar to HIVE-7390 some customers would like the double quotes to be
optional. So if the data is {{A}} then the output from beeline should be
{{A}} which is the same as the Hive CLI.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8119) Implement Date in ParquetSerde
[ https://issues.apache.org/jira/browse/HIVE-8119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14337075#comment-14337075 ] Brock Noland commented on HIVE-8119: Shoot the patch doesn't apply any more. [~dongc] can you rebase? Implement Date in ParquetSerde -- Key: HIVE-8119 URL: https://issues.apache.org/jira/browse/HIVE-8119 Project: Hive Issue Type: Sub-task Reporter: Brock Noland Assignee: Dong Chen Attachments: HIVE-8119.1.patch, HIVE-8119.patch Date type in Parquet is discussed here: http://mail-archives.apache.org/mod_mbox/incubator-parquet-dev/201406.mbox/%3CCAKa9qDkp7xn+H8fNZC7ms3ckd=xr8gdpe7gqgj5o+pybdem...@mail.gmail.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9086) Add language support to PURGE data while dropping partitions.
[ https://issues.apache.org/jira/browse/HIVE-9086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14336980#comment-14336980 ] Eugene Koifman commented on HIVE-9086: -- [~mithun], do you want me to check in the patch as is or are you planning to make changes based on above comments? Add language support to PURGE data while dropping partitions. - Key: HIVE-9086 URL: https://issues.apache.org/jira/browse/HIVE-9086 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.15.0 Reporter: Mithun Radhakrishnan Assignee: Mithun Radhakrishnan Attachments: HIVE-9086.1.patch HIVE-9083 adds metastore-support to skip-trash while dropping partitions. This patch includes language support to do the same. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9445) Revert HIVE-5700 - enforce single date format for partition column storage
[
https://issues.apache.org/jira/browse/HIVE-9445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14336967#comment-14336967
]
Dhruv Kumar commented on HIVE-9445:
---
Was this checked in to 0.13 and 0.14 branches?
Revert HIVE-5700 - enforce single date format for partition column storage
--
Key: HIVE-9445
URL: https://issues.apache.org/jira/browse/HIVE-9445
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.13.0, 0.14.0, 0.13.1, 0.15.0, 0.14.1
Reporter: Brock Noland
Assignee: Brock Noland
Priority: Blocker
Fix For: 1.1.0
Attachments: HIVE-9445.1.patch, HIVE-9445.1.patch
HIVE-5700 has the following issues:
* HIVE-8730 - fails mysql upgrades
* Does not upgrade all metadata, e.g. {{PARTITIONS.PART_NAME}} See comments
in HIVE-5700.
* Completely corrupts postgres, see below.
With a postgres metastore on 0.12, I executed the following:
{noformat}
CREATE TABLE HIVE5700_DATE_PARTED (line string) PARTITIONED BY (ddate date);
CREATE TABLE HIVE5700_STRING_PARTED (line string) PARTITIONED BY (ddate
string);
ALTER TABLE HIVE5700_DATE_PARTED ADD PARTITION (ddate='NOT_DATE');
ALTER TABLE HIVE5700_DATE_PARTED ADD PARTITION (ddate='20150121');
ALTER TABLE HIVE5700_DATE_PARTED ADD PARTITION (ddate='20150122');
ALTER TABLE HIVE5700_DATE_PARTED ADD PARTITION (ddate='2015-01-23');
ALTER TABLE HIVE5700_STRING_PARTED ADD PARTITION (ddate='NOT_DATE');
ALTER TABLE HIVE5700_STRING_PARTED ADD PARTITION (ddate='20150121');
ALTER TABLE HIVE5700_STRING_PARTED ADD PARTITION (ddate='20150122');
ALTER TABLE HIVE5700_STRING_PARTED ADD PARTITION (ddate='2015-01-23');
LOAD DATA LOCAL INPATH '/tmp/single-line-of-data' INTO TABLE
HIVE5700_DATE_PARTED PARTITION (ddate='NOT_DATE');
LOAD DATA LOCAL INPATH '/tmp/single-line-of-data' INTO TABLE
HIVE5700_DATE_PARTED PARTITION (ddate='20150121');
LOAD DATA LOCAL INPATH '/tmp/single-line-of-data' INTO TABLE
HIVE5700_DATE_PARTED PARTITION (ddate='20150122');
LOAD DATA LOCAL INPATH '/tmp/single-line-of-data' INTO TABLE
HIVE5700_DATE_PARTED PARTITION (ddate='2015-01-23');
LOAD DATA LOCAL INPATH '/tmp/single-line-of-data' INTO TABLE
HIVE5700_STRING_PARTED PARTITION (ddate='NOT_DATE');
LOAD DATA LOCAL INPATH '/tmp/single-line-of-data' INTO TABLE
HIVE5700_STRING_PARTED PARTITION (ddate='20150121');
LOAD DATA LOCAL INPATH '/tmp/single-line-of-data' INTO TABLE
HIVE5700_STRING_PARTED PARTITION (ddate='20150122');
LOAD DATA LOCAL INPATH '/tmp/single-line-of-data' INTO TABLE
HIVE5700_STRING_PARTED PARTITION (ddate='2015-01-23');
hive show partitions HIVE5700_DATE_PARTED;
OK
ddate=20150121
ddate=20150122
ddate=2015-01-23
ddate=NOT_DATE
Time taken: 0.052 seconds, Fetched: 4 row(s)
hive show partitions HIVE5700_STRING_PARTED;
OK
ddate=20150121
ddate=20150122
ddate=2015-01-23
ddate=NOT_DATE
Time taken: 0.051 seconds, Fetched: 4 row(s)
{noformat}
I then took a dump of the database named {{postgres-pre-upgrade.sql}} and the
data in the dump looks good:
{noformat}
[root@hive5700-1-1 ~]# egrep -A9 '^COPY PARTITIONS|^COPY
PARTITION_KEY_VALS' postgres-pre-upgrade.sql
COPY PARTITIONS (PART_ID, CREATE_TIME, LAST_ACCESS_TIME, PART_NAME,
SD_ID, TBL_ID) FROM stdin;
3 1421943647 0 ddate=NOT_DATE 6 2
4 1421943647 0 ddate=20150121 7 2
5 1421943648 0 ddate=20150122 8 2
6 1421943664 0 ddate=NOT_DATE 9 3
7 1421943664 0 ddate=20150121 10 3
8 1421943665 0 ddate=20150122 11 3
9 1421943694 0 ddate=2015-01-2312 2
101421943695 0 ddate=2015-01-2313 3
\.
--
COPY PARTITION_KEY_VALS (PART_ID, PART_KEY_VAL, INTEGER_IDX) FROM
stdin;
3 NOT_DATE0
4 201501210
5 201501220
6 NOT_DATE0
7 201501210
8 201501220
9 2015-01-23 0
102015-01-23 0
\.
{noformat}
I then upgraded to 0.13 and subsequently upgraded the MS with the following
command: {{schematool -dbType postgres -upgradeSchema -verbose}}
The file {{postgres-post-upgrade.sql}} is the post-upgrade db dump. As you
can see the data is completely corrupt.
{noformat}
[root@hive5700-1-1 ~]# egrep -A9 '^COPY PARTITIONS|^COPY
PARTITION_KEY_VALS' postgres-post-upgrade.sql
COPY PARTITIONS (PART_ID, CREATE_TIME, LAST_ACCESS_TIME, PART_NAME,
SD_ID, TBL_ID) FROM stdin;
3 1421943647 0 ddate=NOT_DATE 6 2
4 1421943647 0 ddate=20150121 7 2
5 1421943648 0 ddate=20150122 8 2
6 1421943664 0 ddate=NOT_DATE 9 3
7 1421943664 0
[jira] [Commented] (HIVE-7390) Make quote character optional and configurable in BeeLine CSV/TSV output
[
https://issues.apache.org/jira/browse/HIVE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14336940#comment-14336940
]
Brock Noland commented on HIVE-7390:
FYI this jira makes the single quote character optional but not the double
quote. e.g.:
source data:
{noformat}
beeline -u jdbc:hive2://localhost:1 -e select * from quote_test 21 |
grep -Ev '^SLF4J'
+++--+
| quote_test.c1 | quote_test.c2 |
+++--+
| A | B |
| C| D|
+++--+
{noformat}
csv:
{noformat}
beeline -u jdbc:hive2://localhost:1 --outputformat=csv -e select * from
quote_test 21 | grep -Ev '^SLF4J'
'quote_test.c1','quote_test.c2'
'A','B'
'C','D'
{noformat}
csv2:
{noformat}
beeline -u jdbc:hive2://localhost:1 --outputformat=csv2 -e select * from
quote_test 21 | grep -Ev '^SLF4J'
quote_test.c1,quote_test.c2
A,B
C,D
{noformat}
Make quote character optional and configurable in BeeLine CSV/TSV output
Key: HIVE-7390
URL: https://issues.apache.org/jira/browse/HIVE-7390
Project: Hive
Issue Type: New Feature
Components: Clients
Affects Versions: 0.13.1
Reporter: Jim Halfpenny
Assignee: Ferdinand Xu
Fix For: 0.14.0
Attachments: HIVE-7390.1.patch, HIVE-7390.2.patch, HIVE-7390.3.patch,
HIVE-7390.4.patch, HIVE-7390.5.patch, HIVE-7390.6.patch, HIVE-7390.7.patch,
HIVE-7390.8.patch, HIVE-7390.9.patch, HIVE-7390.patch
Currently when either the CSV or TSV output formats are used in beeline each
column is wrapped in single quotes. Quote wrapping of columns should be
optional and the user should be able to choose the character used to wrap the
columns.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9642) Hive metastore client retries don't happen consistently for all api calls
[ https://issues.apache.org/jira/browse/HIVE-9642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14337036#comment-14337036 ] Thejas M Nair commented on HIVE-9642: - Looks good. Just a minor nit , the javadoc has @inheritDoc and this is adding a partial javadoc of this function @throws to it . Can you remove that @throws that got added ? (the full javadoc is available via the superclass). Hive metastore client retries don't happen consistently for all api calls - Key: HIVE-9642 URL: https://issues.apache.org/jira/browse/HIVE-9642 Project: Hive Issue Type: Bug Affects Versions: 1.0.0 Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Attachments: HIVE-9642.1.patch, HIVE-9642.2.patch When org.apache.thrift.transport.TTransportException is thrown for issues like socket timeout, the retry via RetryingMetaStoreClient happens only in certain cases. Retry happens for the getDatabase call in but not for getAllDatabases(). The reason is RetryingMetaStoreClient checks for TTransportException being the cause for InvocationTargetException. But in case of some calls such as getAllDatabases in HiveMetastoreClient, all exceptions get wrapped in a MetaException. We should remove this unnecessary wrapping of exceptions for certain functions in HMC. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9642) Hive metastore client retries don't happen consistently for all api calls
[ https://issues.apache.org/jira/browse/HIVE-9642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9642: Attachment: HIVE-9642.2.patch Patch V2 Hive metastore client retries don't happen consistently for all api calls - Key: HIVE-9642 URL: https://issues.apache.org/jira/browse/HIVE-9642 Project: Hive Issue Type: Bug Affects Versions: 1.0.0 Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Attachments: HIVE-9642.1.patch, HIVE-9642.2.patch When org.apache.thrift.transport.TTransportException is thrown for issues like socket timeout, the retry via RetryingMetaStoreClient happens only in certain cases. Retry happens for the getDatabase call in but not for getAllDatabases(). The reason is RetryingMetaStoreClient checks for TTransportException being the cause for InvocationTargetException. But in case of some calls such as getAllDatabases in HiveMetastoreClient, all exceptions get wrapped in a MetaException. We should remove this unnecessary wrapping of exceptions for certain functions in HMC. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9784) (Alternate) incorrect result set for left outer join when executed with tez versus mapreduce
[ https://issues.apache.org/jira/browse/HIVE-9784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-9784: --- Attachment: alternative.patch.txt (Alternate) incorrect result set for left outer join when executed with tez versus mapreduce Key: HIVE-9784 URL: https://issues.apache.org/jira/browse/HIVE-9784 Project: Hive Issue Type: Bug Components: Vectorization Affects Versions: 0.14.0 Reporter: Matt McCline Assignee: Matt McCline Priority: Critical Attachments: alternative.patch.txt This is a 2nd JIRA for HIVE-9277 that allows for submitting a patch for an alternative solution. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9277) Hybrid Hybrid Grace Hash Join
[ https://issues.apache.org/jira/browse/HIVE-9277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14337137#comment-14337137 ] Sergey Shelukhin commented on HIVE-9277: Also high level note: I still see stuff like // TODO this info can be more accurate when memory mgmt is available; how does this patch function without memory management? Hybrid Hybrid Grace Hash Join - Key: HIVE-9277 URL: https://issues.apache.org/jira/browse/HIVE-9277 Project: Hive Issue Type: New Feature Components: Physical Optimizer Reporter: Wei Zheng Assignee: Wei Zheng Labels: join Attachments: HIVE-9277.01.patch, HIVE-9277.02.patch, HIVE-9277.03.patch, High-leveldesignforHybridHybridGraceHashJoinv1.0.pdf We are proposing an enhanced hash join algorithm called _“hybrid hybrid grace hash join”_. We can benefit from this feature as illustrated below: * The query will not fail even if the estimated memory requirement is slightly wrong * Expensive garbage collection overhead can be avoided when hash table grows * Join execution using a Map join operator even though the small table doesn't fit in memory as spilling some data from the build and probe sides will still be cheaper than having to shuffle the large fact table The design was based on Hadoop’s parallel processing capability and significant amount of memory available. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9642) Hive metastore client retries don't happen consistently for all api calls
[
https://issues.apache.org/jira/browse/HIVE-9642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14337229#comment-14337229
]
Xiaobing Zhou commented on HIVE-9642:
-
Thanks [~thejas] for comments. With patch V2, there are some compile issues
since the function signature changed, e.g.,
{noformat}
public ListString getAllTables(String dbname) throws MetaException {
{noformat}, and now
{noformat}
public ListString getAllTables(String dbname) throws TException {
{noformat}
This will cause many places to be changed, moreover, incompatible exception
declaration happened. I think we can resort back to patch V1. Any thoughts?
Hive metastore client retries don't happen consistently for all api calls
-
Key: HIVE-9642
URL: https://issues.apache.org/jira/browse/HIVE-9642
Project: Hive
Issue Type: Bug
Affects Versions: 1.0.0
Reporter: Xiaobing Zhou
Assignee: Xiaobing Zhou
Attachments: HIVE-9642.1.patch, HIVE-9642.2.patch
When org.apache.thrift.transport.TTransportException is thrown for issues
like socket timeout, the retry via RetryingMetaStoreClient happens only in
certain cases.
Retry happens for the getDatabase call in but not for getAllDatabases().
The reason is RetryingMetaStoreClient checks for TTransportException being
the cause for InvocationTargetException. But in case of some calls such as
getAllDatabases in HiveMetastoreClient, all exceptions get wrapped in a
MetaException. We should remove this unnecessary wrapping of exceptions for
certain functions in HMC.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9743) incorrect result set for left outer join when executed with tez versus mapreduce
[
https://issues.apache.org/jira/browse/HIVE-9743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14337257#comment-14337257
]
Matt McCline commented on HIVE-9743:
Postgres
{noformat}
mmccline=# create table if not exists TJOIN1 (RNUM int , C1 int, C2 int);
CREATE TABLE
mmccline=# create table if not exists TJOIN2 (RNUM int , C1 int, C2 char(2));
CREATE TABLE
mmccline=# insert into TJOIN1 values (0,10,15),(1,20,25),(2,null,50);
INSERT 0 3
mmccline=# insert into TJOIN2 values
(0,10,'BB'),(1,15,'DD'),(2,null,'EE'),(3,10,'FF');
INSERT 0 4
mmccline=# select tjoin1.rnum, tjoin1.c1, tjoin1.c2, tjoin2.c2 as c2j2 from
tjoin1 left outer join tjoin2 on ( tjoin1.c1 = tjoin2.c1 and tjoin1.c2 15 );
rnum | c1 | c2 | c2j2
--+++--
0 | 10 | 15 |
1 | 20 | 25 |
2 || 50 |
(3 rows)
mmccline=#
{noformat}
incorrect result set for left outer join when executed with tez versus
mapreduce
Key: HIVE-9743
URL: https://issues.apache.org/jira/browse/HIVE-9743
Project: Hive
Issue Type: Bug
Components: SQL
Affects Versions: 0.14.0
Reporter: N Campbell
Assignee: Matt McCline
Attachments: HIVE-9743.01.patch, HIVE-9743.02.patch
This query is supposed to return 3 rows and will when run without Tez but
returns 2 rows when run with Tez.
select tjoin1.rnum, tjoin1.c1, tjoin1.c2, tjoin2.c2 as c2j2 from tjoin1 left
outer join tjoin2 on ( tjoin1.c1 = tjoin2.c1 and tjoin1.c2 15 )
tjoin1.rnum tjoin1.c1 tjoin1.c2 c2j2
1 20 25 null
2 null 50 null
instead of
tjoin1.rnum tjoin1.c1 tjoin1.c2 c2j2
0 10 15 null
1 20 25 null
2 null 50 null
create table if not exists TJOIN1 (RNUM int , C1 int, C2 int)
STORED AS orc ;
0|10|15
1|20|25
2|\N|50
create table if not exists TJOIN2 (RNUM int , C1 int, C2 char(2))
ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n'
STORED AS TEXTFILE ;
0|10|BB
1|15|DD
2|\N|EE
3|10|FF
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HIVE-9788) Make double quote optional in tsv/csv/dsv output
[
https://issues.apache.org/jira/browse/HIVE-9788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ferdinand Xu reassigned HIVE-9788:
--
Assignee: Ferdinand Xu
Make double quote optional in tsv/csv/dsv output
Key: HIVE-9788
URL: https://issues.apache.org/jira/browse/HIVE-9788
Project: Hive
Issue Type: Improvement
Reporter: Brock Noland
Assignee: Ferdinand Xu
Similar to HIVE-7390 some customers would like the double quotes to be
optional. So if the data is {{A}} then the output from beeline should be
{{A}} which is the same as the Hive CLI.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9791) insert into table throws NPE
[
https://issues.apache.org/jira/browse/HIVE-9791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Pivovarov updated HIVE-9791:
--
Affects Version/s: 1.2.0
insert into table throws NPE
Key: HIVE-9791
URL: https://issues.apache.org/jira/browse/HIVE-9791
Project: Hive
Issue Type: Bug
Affects Versions: 1.2.0
Reporter: Alexander Pivovarov
to reproduce NPE run the following
{code}
create table a as select 'A' letter;
OK
insert into table a select 'B' letter;
FAILED: NullPointerException null
-- works fine if add from table to select statement
insert into table a select 'B' letter from dual;
OK
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-6586) Update parameters in HiveConf.java after commit HIVE-6037
[ https://issues.apache.org/jira/browse/HIVE-6586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lefty Leverenz updated HIVE-6586: - Labels: (was: TODOC14) Update parameters in HiveConf.java after commit HIVE-6037 - Key: HIVE-6586 URL: https://issues.apache.org/jira/browse/HIVE-6586 Project: Hive Issue Type: Bug Affects Versions: 0.13.0, 0.14.0 Reporter: Lefty Leverenz Assignee: Lefty Leverenz Priority: Blocker Fix For: 0.14.0 Attachments: HIVE-6586.2.patch, HIVE-6586.3.patch, HIVE-6586.4.patch, HIVE-6586.5.patch, HIVE-6586.patch, hive-default.xml.template, hive-default.xml.template.patch2, hive-default.xml.template.patch3, hive-default.xml.template.patch4 HIVE-6037 puts the definitions of configuration parameters into the HiveConf.java file, but several recent jiras for release 0.13.0 introduce new parameters that aren't in HiveConf.java yet and some parameter definitions need to be altered for 0.13.0. This jira will patch HiveConf.java after HIVE-6037 gets committed. Also, four typos patched in HIVE-6582 need to be fixed in the new HiveConf.java. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9480) Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
[
https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14337786#comment-14337786
]
Hive QA commented on HIVE-9480:
---
{color:green}Overall{color}: +1 all checks pass
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12700903/HIVE-9480.8.patch
{color:green}SUCCESS:{color} +1 7576 tests passed
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2873/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2873/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2873/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12700903 - PreCommit-HIVE-TRUNK-Build
Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
Key: HIVE-9480
URL: https://issues.apache.org/jira/browse/HIVE-9480
Project: Hive
Issue Type: Bug
Components: UDF
Affects Versions: 0.14.0
Reporter: Xiaobing Zhou
Assignee: Xiaobing Zhou
Attachments: HIVE-9480.1.patch, HIVE-9480.3.patch, HIVE-9480.4.patch,
HIVE-9480.5.patch, HIVE-9480.6.patch, HIVE-9480.7.patch, HIVE-9480.8.patch
Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to
do date/timestamp related computation. This JIRA is to track such an
implementation. Choose to impl TRUNC, a more standard way to get the first
day of a a month, e.g., SELECT TRUNC('2009-12-12', 'MM'); will return
2009-12-01, SELECT TRUNC('2009-12-12', 'YEAR'); will return 2009-01-01.
BTW, this TRUNC is not as feature complete as aligned with Oracle one. only
'MM' and 'YEAR' are supported as format, however, it's a base to add on other
formats.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9782) LLAP: InStream creation for every row group is expensive
[ https://issues.apache.org/jira/browse/HIVE-9782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-9782: Attachment: HIVE-9782-llap.patch LLAP: InStream creation for every row group is expensive Key: HIVE-9782 URL: https://issues.apache.org/jira/browse/HIVE-9782 Project: Hive Issue Type: Sub-task Affects Versions: llap Reporter: Prasanth Jayachandran Assignee: Prasanth Jayachandran Fix For: llap Attachments: HIVE-9782-llap.patch Orc column vector producer should not create InStream for every row group it process. It should instead create InStream once and keep updating the buffers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9793) Remove hard coded paths from cli driver tests
[
https://issues.apache.org/jira/browse/HIVE-9793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14337810#comment-14337810
]
Xuefu Zhang commented on HIVE-9793:
---
Looks good to me. What about the result directory, which is also using basedir?
Remove hard coded paths from cli driver tests
-
Key: HIVE-9793
URL: https://issues.apache.org/jira/browse/HIVE-9793
Project: Hive
Issue Type: Improvement
Components: Tests
Affects Versions: 1.2.0
Reporter: Brock Noland
Assignee: Brock Noland
Attachments: HIVE-9793.patch
At some point a change which generates a hard coded path into the test files
snuck in. Insert we should use the {{HIVE_ROOT}} directory as this is better
for ptest environments.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9793) Remove hard coded paths from cli driver tests
[
https://issues.apache.org/jira/browse/HIVE-9793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brock Noland updated HIVE-9793:
---
Attachment: HIVE-9793.patch
Remove hard coded paths from cli driver tests
-
Key: HIVE-9793
URL: https://issues.apache.org/jira/browse/HIVE-9793
Project: Hive
Issue Type: Improvement
Components: Tests
Affects Versions: 1.2.0
Reporter: Brock Noland
Attachments: HIVE-9793.patch
At some point a change which generates a hard coded path into the test files
snuck in. Insert we should use the {{HIVE_ROOT}} directory as this is better
for ptest environments.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9785) CBO (Calcite Return Path): Translate Exchange to Hive Op [CBO branch]
[ https://issues.apache.org/jira/browse/HIVE-9785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-9785: -- Attachment: HIVE-9785.cbo.patch [~ashutoshc], I attach the patch with the translation for the Exchange operator. Thanks CBO (Calcite Return Path): Translate Exchange to Hive Op [CBO branch] - Key: HIVE-9785 URL: https://issues.apache.org/jira/browse/HIVE-9785 Project: Hive Issue Type: Sub-task Components: CBO Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: 1.2.0 Attachments: HIVE-9785.cbo.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9792) Support interval type in expressions/predicates
[
https://issues.apache.org/jira/browse/HIVE-9792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Dere updated HIVE-9792:
-
Attachment: HIVE-9792.1.patch
Attaching initial patch.
This adds 2 separate interval types:
- year-month interval, which represents a date difference in years and months.
Internally represented as an offset of months.
- day-time interval, which represents a date difference in
days/hours/minutes/seconds/nanoseconds. 1 day = 24 hours = 1440 minutes = 86400
seconds. Internally represented as an offset of seconds (long) and nanoseconds
(int).
I've tried using the Oracle's [Datetime and Interval
Arithmetic|http://docs.oracle.com/cd/B19306_01/server.102/b14225/ch4datetime.htm#i1006312]
rules with regard to timestamp arithmetic, where the timestamp arithmetic is
done in UTC time. This results in similar behavior to below depending on the
default time zone in Hive:
{noformat}
SQL select timestamp '1999-09-15 00:00:00 America/Los_Angeles' - timestamp
'1999-12-15 00:00:00 America/Los_Angeles' from dual;
TIMESTAMP'1999-09-1500:00:00AMERICA/LOS_ANGELES'-TIMESTAMP'1999-12-1500:00:
---
-00091 01:00:00.0
SQL select timestamp '1999-09-15 00:00:00 UTC' - timestamp '1999-12-15
00:00:00 UTC' from dual;
TIMESTAMP'1999-09-1500:00:00UTC'-TIMESTAMP'1999-12-1500:00:00UTC'
---
-00091 00:00:00.0
{noformat}
Unlike Oracle, however, date '2001-01-31' + interval '1' month returns date
'2001-02-28', whereas this causes an error in Oracle.
Support interval type in expressions/predicates
Key: HIVE-9792
URL: https://issues.apache.org/jira/browse/HIVE-9792
Project: Hive
Issue Type: Sub-task
Components: Types
Reporter: Jason Dere
Assignee: Jason Dere
Attachments: HIVE-9792.1.patch
Provide partial support for the interval year-month/interval day-time types
in Hive. Intervals will be usable in expressions/predicates/joins:
{noformat}
select birthdate + interval '30-0' year to month as thirtieth_birthday
from table
where (current_timestamp - ts1 interval '3 0:0:0' day to second)
{noformat}
This stops short of adding making the interval types usable as a storable
column type.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HIVE-9782) LLAP: InStream creation for every row group is expensive
[ https://issues.apache.org/jira/browse/HIVE-9782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran resolved HIVE-9782. - Resolution: Fixed Committed to llap branch. LLAP: InStream creation for every row group is expensive Key: HIVE-9782 URL: https://issues.apache.org/jira/browse/HIVE-9782 Project: Hive Issue Type: Sub-task Affects Versions: llap Reporter: Prasanth Jayachandran Assignee: Prasanth Jayachandran Fix For: llap Attachments: HIVE-9782-llap.patch Orc column vector producer should not create InStream for every row group it process. It should instead create InStream once and keep updating the buffers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9480) Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
[
https://issues.apache.org/jira/browse/HIVE-9480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaobing Zhou updated HIVE-9480:
Attachment: HIVE-9480.8.patch
patch V8. Can you again review it [~apivovarov]? Thanks.
Build UDF TRUNC to implement FIRST_DAY as compared with LAST_DAY
Key: HIVE-9480
URL: https://issues.apache.org/jira/browse/HIVE-9480
Project: Hive
Issue Type: Bug
Components: UDF
Affects Versions: 0.14.0
Reporter: Xiaobing Zhou
Assignee: Xiaobing Zhou
Attachments: HIVE-9480.1.patch, HIVE-9480.3.patch, HIVE-9480.4.patch,
HIVE-9480.5.patch, HIVE-9480.6.patch, HIVE-9480.7.patch, HIVE-9480.8.patch
Hive already supports LAST_DAY UDF, in some cases, FIRST_DAY is necessary to
do date/timestamp related computation. This JIRA is to track such an
implementation. Choose to impl TRUNC, a more standard way to get the first
day of a a month, e.g., SELECT TRUNC('2009-12-12', 'MM'); will return
2009-12-01, SELECT TRUNC('2009-12-12', 'YEAR'); will return 2009-01-01.
BTW, this TRUNC is not as feature complete as aligned with Oracle one. only
'MM' and 'YEAR' are supported as format, however, it's a base to add on other
formats.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9642) Hive metastore client retries don't happen consistently for all api calls
[ https://issues.apache.org/jira/browse/HIVE-9642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14337492#comment-14337492 ] Thejas M Nair commented on HIVE-9642: - The interface IMetaStoreClient that HiveMetaStoreClient implements does throw TException. It would require changes only in few places. Probably just one file - SessionHiveMetaStoreClient.java, and maybe some places in hcatalog. But there is already one patch (HIVE-9582) that changes that. Hive metastore client retries don't happen consistently for all api calls - Key: HIVE-9642 URL: https://issues.apache.org/jira/browse/HIVE-9642 Project: Hive Issue Type: Bug Affects Versions: 1.0.0 Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Attachments: HIVE-9642.1.patch, HIVE-9642.2.patch When org.apache.thrift.transport.TTransportException is thrown for issues like socket timeout, the retry via RetryingMetaStoreClient happens only in certain cases. Retry happens for the getDatabase call in but not for getAllDatabases(). The reason is RetryingMetaStoreClient checks for TTransportException being the cause for InvocationTargetException. But in case of some calls such as getAllDatabases in HiveMetastoreClient, all exceptions get wrapped in a MetaException. We should remove this unnecessary wrapping of exceptions for certain functions in HMC. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9741) Refactor MetaStoreDirectSql constructor by removing DB queries out of critical section
[ https://issues.apache.org/jira/browse/HIVE-9741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HIVE-9741: Attachment: HIVE-9741.6.patch V6 that removed setDbANSIMode. Refactor MetaStoreDirectSql constructor by removing DB queries out of critical section -- Key: HIVE-9741 URL: https://issues.apache.org/jira/browse/HIVE-9741 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 1.0.0 Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou Attachments: HIVE-9741.1.patch, HIVE-9741.2.patch, HIVE-9741.3.patch, HIVE-9741.4.patch, HIVE-9741.5.patch, HIVE-9741.6.patch MetaStoreDirectSql constructor is querying DB to determine dbType, which leads to too many DB queries to make megastore slow as ObjectStore.setConf might be called frequently. Moreover, ObjectStore.setConf begins/ends with lock acquire/release, if the underlying DB hangs somehow, lock is never released and all hereafter incoming requests are blocked. Two points: 1. Using getProductName based JDBC driver to get dbType info. 2. Since metastore auto-creaton is disabled by default, it'd better bypass ensureDbInit() and runTestQuery() in order to avoid DB queries within critical section of setConf. Here’s stack trace: MetaStoreDirectSql.determineDbType(...) MetaStoreDirectSql.MetaStoreDirectSql(...) ObjectStore.initialize(...) ObjectStore.setConf(…) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9086) Add language support to PURGE data while dropping partitions.
[ https://issues.apache.org/jira/browse/HIVE-9086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14337437#comment-14337437 ] Xuefu Zhang commented on HIVE-9086: --- Could we get a summary on the disagreement here? If the syntax for table is adding PURGE after table name, we should be adding PURGE after partition spec just to be consistent. Add language support to PURGE data while dropping partitions. - Key: HIVE-9086 URL: https://issues.apache.org/jira/browse/HIVE-9086 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.15.0 Reporter: Mithun Radhakrishnan Assignee: Mithun Radhakrishnan Attachments: HIVE-9086.1.patch HIVE-9083 adds metastore-support to skip-trash while dropping partitions. This patch includes language support to do the same. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9086) Add language support to PURGE data while dropping partitions.
[ https://issues.apache.org/jira/browse/HIVE-9086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14337374#comment-14337374 ] Lefty Leverenz commented on HIVE-9086: -- bq. FWIW, I prefer the syntax from the patch. I think it reads more cleanly. Agreed, DROP PURGE PARTITION is harder to understand than DROP PARTITION PURGE. The IF EXISTS option doesn't have the same natural language problem. But it's confusing to put PURGE after PARTITION in the ALTER statement (separate from IF EXISTS) while the DROP TABLE statement puts it next to IF EXISTS. * Would the DROP TABLE syntax be better if it were DROP TABLE IF EXISTS table_name PURGE? But that breaks backward compatibility (PURGE was added in 0.14.0). * Should the syntax of both statements be changed to tolerate either placement of PURGE? Add language support to PURGE data while dropping partitions. - Key: HIVE-9086 URL: https://issues.apache.org/jira/browse/HIVE-9086 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.15.0 Reporter: Mithun Radhakrishnan Assignee: Mithun Radhakrishnan Attachments: HIVE-9086.1.patch HIVE-9083 adds metastore-support to skip-trash while dropping partitions. This patch includes language support to do the same. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9086) Add language support to PURGE data while dropping partitions.
[
https://issues.apache.org/jira/browse/HIVE-9086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14337520#comment-14337520
]
Mithun Radhakrishnan commented on HIVE-9086:
Judging from [the
patch|https://issues.apache.org/jira/secure/attachment/12670435/HIVE-7100.11.patch#file-12],
HIVE-7100 added the drop-table-purge functionality to read thus:
{code:sql}
DROP TABLE IF EXISTS my_doomed_table PURGE;
{code}
The current alter table drop partitions reads as follows:
{code:sql}
ALTER TABLE my_doomed_table DROP IF EXISTS PARTITION (part_key = sayonara)
IGNORE PROTECTION;
{code}
HIVE-9086 extends HIVE-7100's purge-functionality to partitions, and suggests
that the {{PURGE}} keyword go at the end, thus:
{code:sql}
ALTER TABLE my_doomed_table DROP IF EXISTS PARTITION (part_key = sayonara)
IGNORE PROTECTION PURGE;
{code}
Should {{PURGE}} sit before/after {{IF EXISTS}} or after {{IGNORE PROTECTION}}?
We can't break backward compatibility, so we shouldn't be changing what we
released in 0.14.
Add language support to PURGE data while dropping partitions.
-
Key: HIVE-9086
URL: https://issues.apache.org/jira/browse/HIVE-9086
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.15.0
Reporter: Mithun Radhakrishnan
Assignee: Mithun Radhakrishnan
Attachments: HIVE-9086.1.patch
HIVE-9083 adds metastore-support to skip-trash while dropping partitions.
This patch includes language support to do the same.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9473) sql std auth should disallow built-in udfs that allow any java methods to be called
[ https://issues.apache.org/jira/browse/HIVE-9473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lefty Leverenz updated HIVE-9473: - Labels: TODOC1.0 (was: ) sql std auth should disallow built-in udfs that allow any java methods to be called --- Key: HIVE-9473 URL: https://issues.apache.org/jira/browse/HIVE-9473 Project: Hive Issue Type: Bug Components: Authorization, SQLStandardAuthorization Reporter: Thejas M Nair Assignee: Thejas M Nair Labels: TODOC1.0 Fix For: 1.0.0 Attachments: HIVE-9473.1.patch As mentioned in HIVE-8893, some udfs can be used to execute arbitrary java methods. This should be disallowed when sql standard authorization is used. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9445) Revert HIVE-5700 - enforce single date format for partition column storage
[
https://issues.apache.org/jira/browse/HIVE-9445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14337548#comment-14337548
]
Sergey Shelukhin commented on HIVE-9445:
Can you please also include HIVE-9509, which reverts part of this revert not
related to faulty scripts
Revert HIVE-5700 - enforce single date format for partition column storage
--
Key: HIVE-9445
URL: https://issues.apache.org/jira/browse/HIVE-9445
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.13.0, 0.14.0, 0.13.1, 0.15.0, 0.14.1
Reporter: Brock Noland
Assignee: Brock Noland
Priority: Blocker
Fix For: 1.1.0
Attachments: HIVE-9445.1.patch, HIVE-9445.1.patch
HIVE-5700 has the following issues:
* HIVE-8730 - fails mysql upgrades
* Does not upgrade all metadata, e.g. {{PARTITIONS.PART_NAME}} See comments
in HIVE-5700.
* Completely corrupts postgres, see below.
With a postgres metastore on 0.12, I executed the following:
{noformat}
CREATE TABLE HIVE5700_DATE_PARTED (line string) PARTITIONED BY (ddate date);
CREATE TABLE HIVE5700_STRING_PARTED (line string) PARTITIONED BY (ddate
string);
ALTER TABLE HIVE5700_DATE_PARTED ADD PARTITION (ddate='NOT_DATE');
ALTER TABLE HIVE5700_DATE_PARTED ADD PARTITION (ddate='20150121');
ALTER TABLE HIVE5700_DATE_PARTED ADD PARTITION (ddate='20150122');
ALTER TABLE HIVE5700_DATE_PARTED ADD PARTITION (ddate='2015-01-23');
ALTER TABLE HIVE5700_STRING_PARTED ADD PARTITION (ddate='NOT_DATE');
ALTER TABLE HIVE5700_STRING_PARTED ADD PARTITION (ddate='20150121');
ALTER TABLE HIVE5700_STRING_PARTED ADD PARTITION (ddate='20150122');
ALTER TABLE HIVE5700_STRING_PARTED ADD PARTITION (ddate='2015-01-23');
LOAD DATA LOCAL INPATH '/tmp/single-line-of-data' INTO TABLE
HIVE5700_DATE_PARTED PARTITION (ddate='NOT_DATE');
LOAD DATA LOCAL INPATH '/tmp/single-line-of-data' INTO TABLE
HIVE5700_DATE_PARTED PARTITION (ddate='20150121');
LOAD DATA LOCAL INPATH '/tmp/single-line-of-data' INTO TABLE
HIVE5700_DATE_PARTED PARTITION (ddate='20150122');
LOAD DATA LOCAL INPATH '/tmp/single-line-of-data' INTO TABLE
HIVE5700_DATE_PARTED PARTITION (ddate='2015-01-23');
LOAD DATA LOCAL INPATH '/tmp/single-line-of-data' INTO TABLE
HIVE5700_STRING_PARTED PARTITION (ddate='NOT_DATE');
LOAD DATA LOCAL INPATH '/tmp/single-line-of-data' INTO TABLE
HIVE5700_STRING_PARTED PARTITION (ddate='20150121');
LOAD DATA LOCAL INPATH '/tmp/single-line-of-data' INTO TABLE
HIVE5700_STRING_PARTED PARTITION (ddate='20150122');
LOAD DATA LOCAL INPATH '/tmp/single-line-of-data' INTO TABLE
HIVE5700_STRING_PARTED PARTITION (ddate='2015-01-23');
hive show partitions HIVE5700_DATE_PARTED;
OK
ddate=20150121
ddate=20150122
ddate=2015-01-23
ddate=NOT_DATE
Time taken: 0.052 seconds, Fetched: 4 row(s)
hive show partitions HIVE5700_STRING_PARTED;
OK
ddate=20150121
ddate=20150122
ddate=2015-01-23
ddate=NOT_DATE
Time taken: 0.051 seconds, Fetched: 4 row(s)
{noformat}
I then took a dump of the database named {{postgres-pre-upgrade.sql}} and the
data in the dump looks good:
{noformat}
[root@hive5700-1-1 ~]# egrep -A9 '^COPY PARTITIONS|^COPY
PARTITION_KEY_VALS' postgres-pre-upgrade.sql
COPY PARTITIONS (PART_ID, CREATE_TIME, LAST_ACCESS_TIME, PART_NAME,
SD_ID, TBL_ID) FROM stdin;
3 1421943647 0 ddate=NOT_DATE 6 2
4 1421943647 0 ddate=20150121 7 2
5 1421943648 0 ddate=20150122 8 2
6 1421943664 0 ddate=NOT_DATE 9 3
7 1421943664 0 ddate=20150121 10 3
8 1421943665 0 ddate=20150122 11 3
9 1421943694 0 ddate=2015-01-2312 2
101421943695 0 ddate=2015-01-2313 3
\.
--
COPY PARTITION_KEY_VALS (PART_ID, PART_KEY_VAL, INTEGER_IDX) FROM
stdin;
3 NOT_DATE0
4 201501210
5 201501220
6 NOT_DATE0
7 201501210
8 201501220
9 2015-01-23 0
102015-01-23 0
\.
{noformat}
I then upgraded to 0.13 and subsequently upgraded the MS with the following
command: {{schematool -dbType postgres -upgradeSchema -verbose}}
The file {{postgres-post-upgrade.sql}} is the post-upgrade db dump. As you
can see the data is completely corrupt.
{noformat}
[root@hive5700-1-1 ~]# egrep -A9 '^COPY PARTITIONS|^COPY
PARTITION_KEY_VALS' postgres-post-upgrade.sql
COPY PARTITIONS (PART_ID, CREATE_TIME, LAST_ACCESS_TIME, PART_NAME,
SD_ID, TBL_ID) FROM stdin;
3 1421943647 0 ddate=NOT_DATE 6 2
4 1421943647 0 ddate=20150121 7 2
5 1421943648 0 ddate=20150122 8 2
6 1421943664 0
[jira] [Updated] (HIVE-6679) HiveServer2 should support configurable the server side socket timeout and keepalive for various transports types where applicable
[ https://issues.apache.org/jira/browse/HIVE-6679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lefty Leverenz updated HIVE-6679: - Labels: TODOC15 (was: TODOC14 TODOC15) HiveServer2 should support configurable the server side socket timeout and keepalive for various transports types where applicable -- Key: HIVE-6679 URL: https://issues.apache.org/jira/browse/HIVE-6679 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.13.0, 0.14.0 Reporter: Prasad Mujumdar Assignee: Navis Labels: TODOC15 Fix For: 1.1.0 Attachments: HIVE-6679.1.patch.txt, HIVE-6679.2.patch.txt, HIVE-6679.3.patch, HIVE-6679.4.patch, HIVE-6679.5.patch, HIVE-6679.6.patch HiveServer2 should support configurable the server side socket read timeout and TCP keep-alive option. Metastore server already support this (and the so is the old hive server). We now have multiple client connectivity options like Kerberos, Delegation Token (Digest-MD5), Plain SASL, Plain SASL with SSL and raw sockets. The configuration should be applicable to all types (if possible). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-7100) Users of hive should be able to specify skipTrash when dropping tables.
[ https://issues.apache.org/jira/browse/HIVE-7100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14337403#comment-14337403 ] Lefty Leverenz commented on HIVE-7100: -- The syntax for PURGE in a drop partition statement is under discussion in HIVE-9086. The question is whether PURGE belongs after or before the partition specification. I'm bringing it up here because this jira has more watchers who might want to voice their opinions. Users of hive should be able to specify skipTrash when dropping tables. --- Key: HIVE-7100 URL: https://issues.apache.org/jira/browse/HIVE-7100 Project: Hive Issue Type: Improvement Components: Query Processor Affects Versions: 0.13.0 Reporter: Ravi Prakash Assignee: david serafini Fix For: 0.14.0 Attachments: HIVE-7100.1.patch, HIVE-7100.10.patch, HIVE-7100.11.patch, HIVE-7100.2.patch, HIVE-7100.3.patch, HIVE-7100.4.patch, HIVE-7100.5.patch, HIVE-7100.8.patch, HIVE-7100.9.patch, HIVE-7100.patch Users of our clusters are often running up against their quota limits because of Hive tables. When they drop tables, they have to then manually delete the files from HDFS using skipTrash. This is cumbersome and unnecessary. We should enable users to skipTrash directly when dropping tables. We should also be able to provide this functionality without polluting SQL syntax. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9780) Add another level of explain for RDBMS audience
[
https://issues.apache.org/jira/browse/HIVE-9780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14337433#comment-14337433
]
Hive QA commented on HIVE-9780:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12700867/HIVE-9780.01.patch
{color:red}ERROR:{color} -1 due to 1450 failed/errored test(s), 7567 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_add_part_multiple
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alias_casted_column
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_allcolref_in_udf
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_rename_table
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ambiguous_col
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_filter
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_groupby
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_groupby2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_join
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_join_pkfk
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_limit
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_part
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_select
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_table
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_union
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ansi_sql_arithmetic
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_array_map_access_nonconstant
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join0
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join10
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join11
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join14
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join15
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join16
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join17
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join18
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join18_multi_distinct
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join19
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join20
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join21
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join22
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join23
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join24
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join26
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join27
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join28
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join29
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join30
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join31
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join32
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join33
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join6
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join7
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join9
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join_stats
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join_stats2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join_without_localtask
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_smb_mapjoin_14
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_10
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_14
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_15
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_6
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_9
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_binarysortable_1
[jira] [Commented] (HIVE-9793) Remove hard coded paths from cli driver tests
[
https://issues.apache.org/jira/browse/HIVE-9793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14338038#comment-14338038
]
Hive QA commented on HIVE-9793:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12700966/HIVE-9793.patch
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 6743 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.initializationError
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.initializationError
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.initializationError
org.apache.hadoop.hive.cli.TestSparkCliDriver.initializationError
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2876/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2876/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2876/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12700966 - PreCommit-HIVE-TRUNK-Build
Remove hard coded paths from cli driver tests
-
Key: HIVE-9793
URL: https://issues.apache.org/jira/browse/HIVE-9793
Project: Hive
Issue Type: Improvement
Components: Tests
Affects Versions: 1.2.0
Reporter: Brock Noland
Assignee: Brock Noland
Attachments: HIVE-9793.patch, HIVE-9793.patch
At some point a change which generates a hard coded path into the test files
snuck in. Insert we should use the {{HIVE_ROOT}} directory as this is better
for ptest environments.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-6617) Reduce ambiguity in grammar
[ https://issues.apache.org/jira/browse/HIVE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong updated HIVE-6617: -- Attachment: (was: HIVE-6617.19.patch) Reduce ambiguity in grammar --- Key: HIVE-6617 URL: https://issues.apache.org/jira/browse/HIVE-6617 Project: Hive Issue Type: Task Reporter: Ashutosh Chauhan Assignee: Pengcheng Xiong Attachments: HIVE-6617.01.patch, HIVE-6617.02.patch, HIVE-6617.03.patch, HIVE-6617.04.patch, HIVE-6617.05.patch, HIVE-6617.06.patch, HIVE-6617.07.patch, HIVE-6617.08.patch, HIVE-6617.09.patch, HIVE-6617.10.patch, HIVE-6617.11.patch, HIVE-6617.12.patch, HIVE-6617.13.patch, HIVE-6617.14.patch, HIVE-6617.15.patch, HIVE-6617.16.patch, HIVE-6617.17.patch, HIVE-6617.18.patch CLEAR LIBRARY CACHE As of today, antlr reports 214 warnings. Need to bring down this number, ideally to 0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-6617) Reduce ambiguity in grammar
[ https://issues.apache.org/jira/browse/HIVE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong updated HIVE-6617: -- Attachment: HIVE-6617.19.patch Reduce ambiguity in grammar --- Key: HIVE-6617 URL: https://issues.apache.org/jira/browse/HIVE-6617 Project: Hive Issue Type: Task Reporter: Ashutosh Chauhan Assignee: Pengcheng Xiong Attachments: HIVE-6617.01.patch, HIVE-6617.02.patch, HIVE-6617.03.patch, HIVE-6617.04.patch, HIVE-6617.05.patch, HIVE-6617.06.patch, HIVE-6617.07.patch, HIVE-6617.08.patch, HIVE-6617.09.patch, HIVE-6617.10.patch, HIVE-6617.11.patch, HIVE-6617.12.patch, HIVE-6617.13.patch, HIVE-6617.14.patch, HIVE-6617.15.patch, HIVE-6617.16.patch, HIVE-6617.17.patch, HIVE-6617.18.patch, HIVE-6617.19.patch CLEAR LIBRARY CACHE As of today, antlr reports 214 warnings. Need to bring down this number, ideally to 0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-6617) Reduce ambiguity in grammar
[ https://issues.apache.org/jira/browse/HIVE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong updated HIVE-6617: -- Attachment: HIVE-6617.19.patch Reduce ambiguity in grammar --- Key: HIVE-6617 URL: https://issues.apache.org/jira/browse/HIVE-6617 Project: Hive Issue Type: Task Reporter: Ashutosh Chauhan Assignee: Pengcheng Xiong Attachments: HIVE-6617.01.patch, HIVE-6617.02.patch, HIVE-6617.03.patch, HIVE-6617.04.patch, HIVE-6617.05.patch, HIVE-6617.06.patch, HIVE-6617.07.patch, HIVE-6617.08.patch, HIVE-6617.09.patch, HIVE-6617.10.patch, HIVE-6617.11.patch, HIVE-6617.12.patch, HIVE-6617.13.patch, HIVE-6617.14.patch, HIVE-6617.15.patch, HIVE-6617.16.patch, HIVE-6617.17.patch, HIVE-6617.18.patch CLEAR LIBRARY CACHE As of today, antlr reports 214 warnings. Need to bring down this number, ideally to 0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9743) incorrect result set for left outer join when executed with tez versus mapreduce
[
https://issues.apache.org/jira/browse/HIVE-9743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14336336#comment-14336336
]
Hive QA commented on HIVE-9743:
---
{color:green}Overall{color}: +1 all checks pass
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12700696/HIVE-9743.02.patch
{color:green}SUCCESS:{color} +1 7569 tests passed
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2868/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2868/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2868/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12700696 - PreCommit-HIVE-TRUNK-Build
incorrect result set for left outer join when executed with tez versus
mapreduce
Key: HIVE-9743
URL: https://issues.apache.org/jira/browse/HIVE-9743
Project: Hive
Issue Type: Bug
Components: SQL
Affects Versions: 0.14.0
Reporter: N Campbell
Assignee: Matt McCline
Attachments: HIVE-9743.01.patch, HIVE-9743.02.patch
This query is supposed to return 3 rows and will when run without Tez but
returns 2 rows when run with Tez.
select tjoin1.rnum, tjoin1.c1, tjoin1.c2, tjoin2.c2 as c2j2 from tjoin1 left
outer join tjoin2 on ( tjoin1.c1 = tjoin2.c1 and tjoin1.c2 15 )
tjoin1.rnum tjoin1.c1 tjoin1.c2 c2j2
1 20 25 null
2 null 50 null
instead of
tjoin1.rnum tjoin1.c1 tjoin1.c2 c2j2
0 10 15 null
1 20 25 null
2 null 50 null
create table if not exists TJOIN1 (RNUM int , C1 int, C2 int)
STORED AS orc ;
0|10|15
1|20|25
2|\N|50
create table if not exists TJOIN2 (RNUM int , C1 int, C2 char(2))
ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n'
STORED AS TEXTFILE ;
0|10|BB
1|15|DD
2|\N|EE
3|10|FF
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9413) CBO(Calcite Return Path): Translate TS, Select to Hive OP [CBO branch]
[ https://issues.apache.org/jira/browse/HIVE-9413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-9413: -- Attachment: HIVE-9413.cbo.patch CBO(Calcite Return Path): Translate TS, Select to Hive OP [CBO branch] -- Key: HIVE-9413 URL: https://issues.apache.org/jira/browse/HIVE-9413 Project: Hive Issue Type: Sub-task Components: CBO Affects Versions: cbo-branch Reporter: Laljo John Pullokkaran Assignee: Laljo John Pullokkaran Fix For: 1.2.0 Attachments: HIVE-9413.cbo.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
