[jira] [Created] (HIVE-18822) INSERT VALUES - HoS + Avro

2018-02-27 Thread BELUGA BEHR (JIRA) 
BELUGA BEHR created HIVE-18822:
--

 Summary: INSERT VALUES - HoS + Avro
 Key: HIVE-18822
 URL: https://issues.apache.org/jira/browse/HIVE-18822
 Project: Hive
  Issue Type: Improvement
  Components: HiveServer2
Affects Versions: 3.0.0
Reporter: BELUGA BEHR


Please optimize the INSERT VALUES function.  When HoS is being used, and a 
streaming format such as TEXT or AVRO are being used, INSERT VALUES statements 
should be quick.  The HiveServer2 should pass the vales to the Executor and the 
Executor should simply append the data to an existing HDFS file instead of 
creating a new one.  This will reduce the number of small files that exist in 
the file system... or perhaps the HiveServer2 performs the append without 
having to first sent the data to the processing engine at all.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 65634: HIVE-18264: CachedStore: Store cached partitions/col stats within the table cache

2018-02-27 Thread Daniel Dai 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65634/#review198377
---


Fix it, then Ship it!




Ship It!


standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java
Line 583 (original), 568 (patched)


Blindly update cache may overwrite newer content. But will leave this to 
notification based update.


- Daniel Dai


On Feb. 26, 2018, 9:47 p.m., Vaibhav Gumashta wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65634/
> ---
> 
> (Updated Feb. 26, 2018, 9:47 p.m.)
> 
> 
> Review request for hive, Daniel Dai and Thejas Nair.
> 
> 
> Bugs: HIVE-18264
> https://issues.apache.org/jira/browse/HIVE-18264
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/HIVE-18264
> 
> 
> Diffs
> -
> 
>   
> itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
>  a3725c5395 
>   service/src/java/org/apache/hive/service/server/HiveServer2.java 6c1a0b98cc 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
>  c6e34a8a22 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java
>  7b44df4128 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/RawStore.java
>  f500d63725 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java
>  f0f650ddcf 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java
>  0d132f2074 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java
>  32ea17495f 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreUtils.java
>  50f873a013 
>   
> standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
>  75ea8c4a77 
>   
> standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
>  207d842f94 
>   
> standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java
>  ab6feb6f0b 
>   standalone-metastore/src/test/resources/log4j2.properties 365687e1c9 
> 
> 
> Diff: https://reviews.apache.org/r/65634/diff/3/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Vaibhav Gumashta
> 
>

Re: Review Request 65634: HIVE-18264: CachedStore: Store cached partitions/col stats within the table cache

2018-02-27 Thread Daniel Dai 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65634/#review198378
---


Ship it!




Ship It!

- Daniel Dai


On Feb. 26, 2018, 9:47 p.m., Vaibhav Gumashta wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65634/
> ---
> 
> (Updated Feb. 26, 2018, 9:47 p.m.)
> 
> 
> Review request for hive, Daniel Dai and Thejas Nair.
> 
> 
> Bugs: HIVE-18264
> https://issues.apache.org/jira/browse/HIVE-18264
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/HIVE-18264
> 
> 
> Diffs
> -
> 
>   
> itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
>  a3725c5395 
>   service/src/java/org/apache/hive/service/server/HiveServer2.java 6c1a0b98cc 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
>  c6e34a8a22 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java
>  7b44df4128 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/RawStore.java
>  f500d63725 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java
>  f0f650ddcf 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java
>  0d132f2074 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java
>  32ea17495f 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreUtils.java
>  50f873a013 
>   
> standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
>  75ea8c4a77 
>   
> standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
>  207d842f94 
>   
> standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java
>  ab6feb6f0b 
>   standalone-metastore/src/test/resources/log4j2.properties 365687e1c9 
> 
> 
> Diff: https://reviews.apache.org/r/65634/diff/3/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Vaibhav Gumashta
> 
>

[jira] [Created] (HIVE-18821) dfs -ls not working when executed from beeline

2018-02-27 Thread Ranjan Choudhury (JIRA) 
Ranjan Choudhury created HIVE-18821:
---

 Summary: dfs -ls not working when executed from beeline
 Key: HIVE-18821
 URL: https://issues.apache.org/jira/browse/HIVE-18821
 Project: Hive
  Issue Type: Bug
  Components: Beeline
Affects Versions: 1.2.1
 Environment: Hive 1.2.1000.2.5.3.0-37

Hadoop 2.7.3.2.5.3.0-37
Reporter: Ranjan Choudhury


The command dfs -ls is not producing any output when executed from beeline even 
though there are files in hdfs.
Steps to reproduce:
1. Run beeline
2. Connect to hive using command !connect jdbc:hive2://localhost:1
3. Provide username/password
4. Execute dfs -ls /user

There is not output of the command dfs -ls /user, even though there are files 
in HDFS. When i execute the command hive CLI, it works.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HIVE-18820) Operation doesn't always clean up log4j for operation log

2018-02-27 Thread Sergey Shelukhin (JIRA) 
Sergey Shelukhin created HIVE-18820:
---

 Summary: Operation doesn't always clean up log4j for operation log
 Key: HIVE-18820
 URL: https://issues.apache.org/jira/browse/HIVE-18820
 Project: Hive
  Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
 Attachments: HIVE-18820.patch

Operation log may only be enabled for some operations, however the routing 
appender still creates appenders for the queries where it's not enabled as far 
as I see, that don't get cleaned up.




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HIVE-18819) Vectorization: Optimize IF statement expression evaluation of THEN/ELSE

2018-02-27 Thread Matt McCline (JIRA) 
Matt McCline created HIVE-18819:
---

 Summary: Vectorization: Optimize IF statement expression 
evaluation of THEN/ELSE
 Key: HIVE-18819
 URL: https://issues.apache.org/jira/browse/HIVE-18819
 Project: Hive
  Issue Type: Bug
  Components: Hive
Affects Versions: 3.0.0
Reporter: Matt McCline
Assignee: Matt McCline


Currently, all the rows of a batch are evaluated for the THEN and ELSE 
expressions even though only a value from one of them is needed for any 
particular row.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HIVE-18818) Alter table add constraint unique fails with direct sql set to false

2018-02-27 Thread Vineet Garg (JIRA) 
Vineet Garg created HIVE-18818:
--

 Summary: Alter table add constraint unique fails with direct sql 
set to false
 Key: HIVE-18818
 URL: https://issues.apache.org/jira/browse/HIVE-18818
 Project: Hive
  Issue Type: Bug
  Components: Metastore
Affects Versions: 3.0.0
Reporter: Vineet Garg
Assignee: Vineet Garg
 Fix For: 3.0.0


*Reproducer*

{code:sql}
set hive.metastore.try.direct.sql=false;

CREATE TABLE numericDataType(a TINYINT CONSTRAINT tinyint_constraint DEFAULT 
127Y ENABLE, b SMALLINT DEFAULT 32767S, c INT DEFAULT 2147483647,
 d BIGINT DEFAULT 9223372036854775807L, e DOUBLE DEFAULT 3.4E38, f DECIMAL(9,2) 
DEFAULT 1234567.89);

ALTER TABLE numericDataType ADD CONSTRAINT uk1 UNIQUE(a,b) DISABLE NOVALIDATE;

DESC FORMATTED numericDataType;
{code}

*Error*
{code:sql}
FAILED: Execution Error, return code 1 from 
org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.NullPointerException
{code}

*Error Stack*
{noformat}
java.lang.NullPointerException: null
at 
org.apache.hadoop.hive.metastore.ObjectStore.getUniqueConstraintsViaJdo(ObjectStore.java:9620)
 ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at 
org.apache.hadoop.hive.metastore.ObjectStore.access$1900(ObjectStore.java:225) 
~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at 
org.apache.hadoop.hive.metastore.ObjectStore$17.getJdoResult(ObjectStore.java:9591)
 ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at 
org.apache.hadoop.hive.metastore.ObjectStore$17.getJdoResult(ObjectStore.java:9580)
 ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at 
org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.run(ObjectStore.java:3249)
 ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at 
org.apache.hadoop.hive.metastore.ObjectStore.getUniqueConstraintsInternal(ObjectStore.java:9593)
 ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at 
org.apache.hadoop.hive.metastore.ObjectStore.getUniqueConstraints(ObjectStore.java:9569)
 ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
~[?:1.8.0_101]
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
~[?:1.8.0_101]
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 ~[?:1.8.0_101]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_101]
at 
org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97) 
~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at com.sun.proxy.$Proxy35.getUniqueConstraints(Unknown Source) ~[?:?]
at 
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_unique_constraints(HiveMetaStore.java:7356)
 ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
~[?:1.8.0_101]
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
~[?:1.8.0_101]
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 ~[?:1.8.0_101]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_101]
at 
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
 ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at 
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
 ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at com.sun.proxy.$Proxy37.get_unique_constraints(Unknown Source) ~[?:?]
at 
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getUniqueConstraints(HiveMetaStoreClient.java:1764)
 ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
~[?:1.8.0_101]
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
~[?:1.8.0_101]
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 ~[?:1.8.0_101]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_101]
at 
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:212)
 ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at com.sun.proxy.$Proxy38.getUniqueConstraints(Unknown Source) ~[?:?]
at 
org.apache.hadoop.hive.ql.metadata.Hive.getUniqueConstraints(Hive.java:4582) 
~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.metadata.Hive.getUniqueConstraints(Hive.java:4564) 
~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.exec.DDLTask.describeTable(DDLTask.java:3626) 
~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:455) 
~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at

[jira] [Created] (HIVE-18817) ArrayIndexOutOfBounds exception during read of ACID table.

2018-02-27 Thread Jason Dere (JIRA) 
Jason Dere created HIVE-18817:
-

 Summary: ArrayIndexOutOfBounds exception during read of ACID table.
 Key: HIVE-18817
 URL: https://issues.apache.org/jira/browse/HIVE-18817
 Project: Hive
  Issue Type: Bug
  Components: ORC, Transactions
Reporter: Jason Dere
 Attachments: repro.patch

Seeing some users hitting the following stack trace:
{noformat}
2018-02-26 05:49:45,876 [ERROR] [TezChild] |tez.TezProcessor|: 
java.lang.RuntimeException: java.io.IOException: 
java.lang.ArrayIndexOutOfBoundsException: 7
at 
org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:196)
at 
org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.next(TezGroupedSplitsInputFormat.java:142)
at 
org.apache.tez.mapreduce.lib.MRReaderMapred.next(MRReaderMapred.java:113)
at 
org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:66)
at 
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:325)
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:150)
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
at 
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:347)
at 
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:194)
at 
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:185)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
at 
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:185)
at 
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:181)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: java.lang.ArrayIndexOutOfBoundsException: 7
at 
org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
at 
org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
at 
org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:258)
at 
org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:193)
... 19 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: 7
at 
org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger.discoverKeyBounds(OrcRawRecordMerger.java:388)
at 
org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger.(OrcRawRecordMerger.java:457)
at 
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getReader(OrcInputFormat.java:1456)
at 
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:1342)
at 
org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:255)
... 20 more
{noformat}

Have a JUnit test that appears to produce a similar stack trace - looks like 
this occurs if there is an OrcSplit of an ACID table where the split offset is 
beyond the starting offset of the last stripe in the ORC file.

cc [~ekoifman]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HIVE-18816) CREATE TABLE (ACID) doesn't work with TIMESTAMPLOCALTZ column type

2018-02-27 Thread Vineet Garg (JIRA) 
Vineet Garg created HIVE-18816:
--

 Summary: CREATE TABLE (ACID) doesn't work with TIMESTAMPLOCALTZ 
column type
 Key: HIVE-18816
 URL: https://issues.apache.org/jira/browse/HIVE-18816
 Project: Hive
  Issue Type: Bug
Affects Versions: 3.0.0
Reporter: Vineet Garg


*Reproducer*
{code:sql}
set hive.support.concurrency=true;
set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;

CREATE TABLE table_acid(d int, tz timestamp with local time zone)
clustered by (d) into 2 buckets stored as orc TBLPROPERTIES 
('transactional'='true');
{code}

*Error*
{code:sql}
FAILED: Execution Error, return code 1 from 
org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.IllegalArgumentException: 
Unknown primitive type TIMESTAMPLOCALTZ
{code}

*Error stack*
{noformat}
org.apache.hadoop.hive.ql.metadata.HiveException: 
java.lang.IllegalArgumentException: Unknown primitive type TIMESTAMPLOCALTZ
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:906) 
~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:4788) 
[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:389) 
[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:205) 
[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:97) 
[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2314) 
[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1985) 
[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1687) 
[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1438) 
[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1427) 
[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at 
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:239) 
[hive-cli-3.0.0-SNAPSHOT.jar:?]
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:188) 
[hive-cli-3.0.0-SNAPSHOT.jar:?]
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:402) 
[hive-cli-3.0.0-SNAPSHOT.jar:?]
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:335) 
[hive-cli-3.0.0-SNAPSHOT.jar:?]
at 
org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:1345) 
[hive-it-util-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:1319) 
[hive-it-util-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at 
org.apache.hadoop.hive.cli.control.CoreCliDriver.runTest(CoreCliDriver.java:173)
 [hive-it-util-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at 
org.apache.hadoop.hive.cli.control.CliAdapter.runTest(CliAdapter.java:104) 
[hive-it-util-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at 
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver(TestMiniLlapLocalCliDriver.java:59)
 [test-classes/:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
~[?:1.8.0_101]
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
~[?:1.8.0_101]
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 ~[?:1.8.0_101]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_101]
at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
 [junit-4.11.jar:?]
at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
 [junit-4.11.jar:?]
at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
 [junit-4.11.jar:?]
at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
 [junit-4.11.jar:?]
at 
org.apache.hadoop.hive.cli.control.CliAdapter$2$1.evaluate(CliAdapter.java:92) 
[hive-it-util-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at org.junit.rules.RunRules.evaluate(RunRules.java:20) 
[junit-4.11.jar:?]
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) 
[junit-4.11.jar:?]
at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
 [junit-4.11.jar:?]
at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
 [junit-4.11.jar:?]
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) 
[junit-4.11.jar:?]
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) 
[junit-4.11.jar:?]
at

[jira] [Created] (HIVE-18815) Remove unused feature in HPL/SQL

2018-02-27 Thread Daniel Dai (JIRA) 
Daniel Dai created HIVE-18815:
-

 Summary: Remove unused feature in HPL/SQL
 Key: HIVE-18815
 URL: https://issues.apache.org/jira/browse/HIVE-18815
 Project: Hive
  Issue Type: Bug
  Components: hpl/sql
Reporter: Daniel Dai
Assignee: Daniel Dai


Remove FTP feature in HPL/SQL.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HIVE-18814) Support Add Partition For Acid tables

2018-02-27 Thread Eugene Koifman (JIRA) 
Eugene Koifman created HIVE-18814:
-

 Summary: Support Add Partition For Acid tables
 Key: HIVE-18814
 URL: https://issues.apache.org/jira/browse/HIVE-18814
 Project: Hive
  Issue Type: New Feature
  Components: Transactions
Reporter: Eugene Koifman
Assignee: Eugene Koifman


[https://cwiki.apache.org/confluence/display/Hive/LanguageManual%2BDDL#LanguageManualDDL-AddPartitions]

Add Partition command creates a \{{Partition}} metadata object and set the 
location to the directory containing data files.

In current master (Hive 3.0), Add partition on an acid table doesn't fail and 
at read time the data is decorated with row__id but the original transaction is 
0.  I suspect in earlier Hive versions this will throw or return no data.

 

One option is follow Load Data approach and create a new delta_x_x/ and 
move/copy the data there.

 

Another is to allocate a new writeid and save it in Partition metadata.  This 
could then be used to decorate data with ROW__IDs.  This avoids move/copy but 
retains data "outside" of the table tree which make it more likely that this 
data will be modified in some way which can really break things if done after 
and SQL update/delete on this data have happened. 

 

 

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 65422: HIVE-17626

2018-02-27 Thread Zoltan Haindrich 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65422/
---

(Updated Feb. 27, 2018, 6:36 p.m.)


Review request for hive and Ashutosh Chauhan.


Changes
---

5


Repository: hive-git


Description
---

preview


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 0880a960e6 
  data/conf/hive-site.xml b56cbd2469 
  data/conf/llap/hive-site.xml c4c299c5de 
  data/conf/perf-reg/spark/hive-site.xml 5ca660d4e7 
  data/conf/perf-reg/tez/hive-site.xml 62ecb74f3f 
  data/conf/rlist/hive-site.xml 630e481edc 
  data/conf/spark/local/hive-site.xml 8ff6256638 
  data/conf/spark/standalone/hive-site.xml 84851c769c 
  data/conf/spark/yarn-client/hive-site.xml 6c633623ab 
  data/conf/tez/hive-site.xml 236adc7087 
  
druid-handler/src/java/org/apache/hadoop/hive/druid/serde/DruidScanQueryRecordReader.java
 cbeac2c00a 
  itests/src/test/resources/testconfiguration.properties 2776fe95f1 
  itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java 6cd7a136ae 
  itests/util/src/test/java/org/apache/hadoop/hive/ql/TestQTestUtil.java 
c01d87bf51 
  ql/src/java/org/apache/hadoop/hive/ql/Context.java dba2dbb15b 
  ql/src/java/org/apache/hadoop/hive/ql/Driver.java 94999fed93 
  ql/src/java/org/apache/hadoop/hive/ql/DriverFactory.java 60e8de8fd4 
  ql/src/java/org/apache/hadoop/hive/ql/HookRunner.java 2a32a51588 
  ql/src/java/org/apache/hadoop/hive/ql/IDriver.java 9f13fa8e88 
  ql/src/java/org/apache/hadoop/hive/ql/cache/results/CacheUsage.java 
08b791ad42 
  ql/src/java/org/apache/hadoop/hive/ql/cache/results/QueryResultsCache.java 
131127e50d 
  ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 32fc257b03 
  ql/src/java/org/apache/hadoop/hive/ql/exec/MaterializedViewDesc.java 
1e28ca843f 
  ql/src/java/org/apache/hadoop/hive/ql/exec/MaterializedViewTask.java 
2b345d6ec7 
  ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java 199b181290 
  ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java 395a5f450f 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/HiveInputCounters.java 
085d6a7d94 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/LlapObjectSubCache.java 
0d31e6e422 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/reducesink/VectorReduceSinkCommonOperator.java
 8dd7cfe58c 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/reducesink/VectorReduceSinkEmptyKeyOperator.java
 134fc0ff0b 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/reducesink/VectorReduceSinkObjectHashOperator.java
 1eb72ce4d9 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/reducesink/VectorReduceSinkUniformHashOperator.java
 384bd74686 
  ql/src/java/org/apache/hadoop/hive/ql/hooks/PrivateHookContext.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveException.java b75850760f 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/SharedWorkOptimizer.java 
b0cf3bd94e 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRelOpMaterializationValidator.java
 8c1bcb3f62 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java 
783a672c47 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java
 9a3f81c98f 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g 78cbf25c43 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 733ec79ce1 
  ql/src/java/org/apache/hadoop/hive/ql/plan/Statistics.java 0057f0c2c6 
  ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/EmptyStatsSource.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/GroupTransformer.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/PlanMapper.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/PlanMapperProcess.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/RuntimeStatsSource.java 
PRE-CREATION 
  
ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/SimpleRuntimeStatsSource.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/StatsSource.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/refs/OperatorRef.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/reexec/IReExecutionPlugin.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/reexec/ReExecDriver.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/reexec/ReExecutionOverlayPlugin.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/reexec/ReOptimizePlugin.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/stats/OperatorStats.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/stats/OperatorStatsReaderHook.java 
PRE-CREATION 
  
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAssertTrueOOM.java 
PRE-CREATION 
  
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFEnforceNotNullConstraint.java
 6c469bf944

[jira] [Created] (HIVE-18813) Fix mapjoin_hook

2018-02-27 Thread Zoltan Haindrich (JIRA) 
Zoltan Haindrich created HIVE-18813:
---

 Summary: Fix mapjoin_hook
 Key: HIVE-18813
 URL: https://issues.apache.org/jira/browse/HIVE-18813
 Project: Hive
  Issue Type: Bug
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich
 Attachments: HIVE-18813.01.patch





--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] hive pull request #314: HIVE-18750: Exchange partition should be disabled on...

2018-02-27 Thread sankarh 
GitHub user sankarh opened a pull request:

https://github.com/apache/hive/pull/314

HIVE-18750: Exchange partition should be disabled on ACID/Insert-only 
tables with per table write ID.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sankarh/hive HIVE-18750

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/hive/pull/314.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #314


commit 08dc82a8ec743ea8c5a85cd596396f0e4ac26767
Author: Sankar Hariappan 
Date:   2018-02-27T16:33:02Z

HIVE-18750: Exchange partition should be disabled on ACID/Insert-only 
tables with per table write ID.




---

Re: Review Request 65778: HIVE-18726 Implement DEFAULT constraint

2018-02-27 Thread Jesús Camacho Rodríguez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65778/#review198327
---




ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java
Lines 400 (patched)


Just a thought. Wouldn't it be better to store the default constraint as 
the SQL text? For instance, that might be useful in case UDF changes in the 
future or other system relies on the metastore now that it will be separated 
from Hive?


- Jesús Camacho Rodríguez


On Feb. 23, 2018, 9:52 p.m., Vineet Garg wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65778/
> ---
> 
> (Updated Feb. 23, 2018, 9:52 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan and Jesús Camacho Rodríguez.
> 
> 
> Bugs: HIVE-18726
> https://issues.apache.org/jira/browse/HIVE-18726
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> This patch adds DEFAULT constraint
> 
> 
> Diffs
> -
> 
>   
> itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
>  a3725c5395 
>   itests/src/test/resources/testconfiguration.properties 4a52eb5559 
>   itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java 
> 6cd7a136ae 
>   metastore/scripts/upgrade/derby/hive-schema-3.0.0.derby.sql a8f227b775 
>   metastore/scripts/upgrade/hive/hive-schema-3.0.0.hive.sql 84d523e1d7 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java f99178dbc7 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 32fc257b03 
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/DefaultConstraint.java 
> PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 8b0af3e5c8 
>   
> ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/JsonMetaDataFormatter.java
>  77e5678f80 
>   
> ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java
>  a5b6a4b0c3 
>   
> ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatter.java
>  88d5554e1d 
>   
> ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/TextMetaDataFormatter.java
>  607e111c97 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java 
> 171825eb74 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 
> e926b63764 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 733ec79ce1 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 
> cd6f1ee692 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 
> 14217e3978 
>   
> ql/src/java/org/apache/hadoop/hive/ql/parse/repl/load/message/AddNotNullConstraintHandler.java
>  9c12e7e2af 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/AlterTableDesc.java 00c0381107 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java 6228d4c803 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeConstantDesc.java 
> 73f449fc28 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/ImportTableDesc.java fcbac7d840 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToChar.java 
> b98ec68158 
>   ql/src/test/queries/clientnegative/alter_external_with_constraint.q  
>   ql/src/test/queries/clientnegative/alter_external_with_default_constraint.q 
> PRE-CREATION 
>   
> ql/src/test/queries/clientnegative/alter_tableprops_external_with_constraint.q
>   
>   
> ql/src/test/queries/clientnegative/alter_tableprops_external_with_default_constraint.q
>  PRE-CREATION 
>   ql/src/test/queries/clientnegative/constraint_duplicate_name.q PRE-CREATION 
>   ql/src/test/queries/clientnegative/constraint_invalide_name.q PRE-CREATION 
>   ql/src/test/queries/clientnegative/constraint_partition_columns.q 
> PRE-CREATION 
>   ql/src/test/queries/clientnegative/create_external_with_constraint.q  
>   
> ql/src/test/queries/clientnegative/create_external_with_default_constraint.q 
> PRE-CREATION 
>   
> ql/src/test/queries/clientnegative/default_constraint_invalid_default_value.q 
> PRE-CREATION 
>   
> ql/src/test/queries/clientnegative/default_constraint_invalid_default_value2.q
>  PRE-CREATION 
>   
> ql/src/test/queries/clientnegative/default_constraint_invalid_default_value_length.q
>  PRE-CREATION 
>   
> ql/src/test/queries/clientnegative/default_constraint_invalid_default_value_type.q
>  PRE-CREATION 
>   ql/src/test/queries/clientnegative/default_constraint_invalid_type.q 
> PRE-CREATION 
>   ql/src/test/queries/clientpositive/create_with_constraints.q 7b2594b79f 
>   ql/src/test/queries/clientpositive/default_constraint.q PRE-CREATION 
>   ql/src/test/results/clientnegative/alter_external_with_constraint.q.out  
>   
> ql/src/test/results/clientnegative/alter_external_with_notnull_constraint.q.out
>

Re: Review Request 65422: HIVE-17626

2018-02-27 Thread Zoltan Haindrich 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65422/
---

(Updated Feb. 27, 2018, 11:17 a.m.)


Review request for hive and Ashutosh Chauhan.


Changes
---

03


Repository: hive-git


Description
---

preview


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 0880a960e6 
  data/conf/hive-site.xml b56cbd2469 
  data/conf/llap/hive-site.xml c4c299c5de 
  data/conf/perf-reg/spark/hive-site.xml 5ca660d4e7 
  data/conf/perf-reg/tez/hive-site.xml 62ecb74f3f 
  data/conf/rlist/hive-site.xml 630e481edc 
  data/conf/spark/local/hive-site.xml 8ff6256638 
  data/conf/spark/standalone/hive-site.xml 84851c769c 
  data/conf/spark/yarn-client/hive-site.xml 6c633623ab 
  data/conf/tez/hive-site.xml 236adc7087 
  itests/src/test/resources/testconfiguration.properties 2776fe95f1 
  itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java 6cd7a136ae 
  ql/src/java/org/apache/hadoop/hive/ql/AbstractReExecDriver.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/Context.java dba2dbb15b 
  ql/src/java/org/apache/hadoop/hive/ql/Driver.java 94999fed93 
  ql/src/java/org/apache/hadoop/hive/ql/DriverFactory.java 60e8de8fd4 
  ql/src/java/org/apache/hadoop/hive/ql/HookRunner.java 2a32a51588 
  ql/src/java/org/apache/hadoop/hive/ql/IDriver.java 9f13fa8e88 
  ql/src/java/org/apache/hadoop/hive/ql/ReExecOverlayDriver.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/ReOptimizeDriver.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 32fc257b03 
  ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java 199b181290 
  ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java 395a5f450f 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/reducesink/VectorReduceSinkCommonOperator.java
 8dd7cfe58c 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/reducesink/VectorReduceSinkEmptyKeyOperator.java
 134fc0ff0b 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/reducesink/VectorReduceSinkObjectHashOperator.java
 1eb72ce4d9 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/reducesink/VectorReduceSinkUniformHashOperator.java
 384bd74686 
  ql/src/java/org/apache/hadoop/hive/ql/hooks/PrivateHookContext.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveException.java b75850760f 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/SharedWorkOptimizer.java 
b0cf3bd94e 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java 
783a672c47 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java
 9a3f81c98f 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g 78cbf25c43 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 733ec79ce1 
  ql/src/java/org/apache/hadoop/hive/ql/plan/Statistics.java 0057f0c2c6 
  ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/EmptyStatsSource.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/GroupTransformer.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/PlanMapper.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/PlanMapperProcess.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/RuntimeStatsSource.java 
PRE-CREATION 
  
ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/SimpleRuntimeStatsSource.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/StatsSource.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/refs/OperatorRef.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/stats/OperatorStats.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/stats/OperatorStatsReaderHook.java 
PRE-CREATION 
  
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAssertTrueOOM.java 
PRE-CREATION 
  
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorGroupByOperator.java
 fa3f89fda1 
  ql/src/test/org/apache/hadoop/hive/ql/plan/mapping/TestCounterMapping.java 
PRE-CREATION 
  ql/src/test/org/apache/hadoop/hive/ql/plan/mapping/TestOperatorCmp.java 
PRE-CREATION 
  ql/src/test/org/apache/hadoop/hive/ql/plan/mapping/TestReOptimization.java 
PRE-CREATION 
  ql/src/test/queries/clientpositive/retry_failure.q PRE-CREATION 
  ql/src/test/queries/clientpositive/retry_failure_oom.q PRE-CREATION 
  ql/src/test/queries/clientpositive/retry_failure_stat_changes.q PRE-CREATION 
  ql/src/test/results/clientpositive/llap/dp_counter_mm.q.out 869916078b 
  ql/src/test/results/clientpositive/llap/dp_counter_non_mm.q.out 9f6426ccd1 
  ql/src/test/results/clientpositive/llap/orc_llap_counters.q.out aa2dcc7f68 
  ql/src/test/results/clientpositive/llap/orc_llap_counters1.q.out eb54a81d1a 
  ql/src/test/results/clientpositive/llap/orc_ppd_basic.q.out 590437c708

Re: Review Request 65422: HIVE-17626

2018-02-27 Thread Zoltan Haindrich 


> On Feb. 16, 2018, 4:50 a.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java
> > Lines 139 (patched)
> > 
> >
> > Should also compare with parent to check rowcount and other stats are 
> > not greater than parent.

ok


> On Feb. 16, 2018, 4:50 a.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java
> > Lines 310 (patched)
> > 
> >
> > Should also compare with parent to check rowcount and other stats are 
> > not greater than parent and normalize to parent stats in that case.

ok


- Zoltan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65422/#review197649
---


On Feb. 26, 2018, 6:19 p.m., Zoltan Haindrich wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65422/
> ---
> 
> (Updated Feb. 26, 2018, 6:19 p.m.)
> 
> 
> Review request for hive and Ashutosh Chauhan.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> preview
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 0880a960e6 
>   data/conf/llap/hive-site.xml c4c299c5de 
>   itests/src/test/resources/testconfiguration.properties 2776fe95f1 
>   itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java 
> 6cd7a136ae 
>   ql/src/java/org/apache/hadoop/hive/ql/AbstractReExecDriver.java 
> PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/Context.java dba2dbb15b 
>   ql/src/java/org/apache/hadoop/hive/ql/Driver.java 94999fed93 
>   ql/src/java/org/apache/hadoop/hive/ql/DriverFactory.java 60e8de8fd4 
>   ql/src/java/org/apache/hadoop/hive/ql/HookRunner.java 2a32a51588 
>   ql/src/java/org/apache/hadoop/hive/ql/IDriver.java 9f13fa8e88 
>   ql/src/java/org/apache/hadoop/hive/ql/ReExecOverlayDriver.java PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/ReOptimizeDriver.java PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 32fc257b03 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java 199b181290 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java 
> 395a5f450f 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/reducesink/VectorReduceSinkCommonOperator.java
>  8dd7cfe58c 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/reducesink/VectorReduceSinkEmptyKeyOperator.java
>  134fc0ff0b 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/reducesink/VectorReduceSinkObjectHashOperator.java
>  1eb72ce4d9 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/reducesink/VectorReduceSinkUniformHashOperator.java
>  384bd74686 
>   ql/src/java/org/apache/hadoop/hive/ql/hooks/PrivateHookContext.java 
> PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/SharedWorkOptimizer.java 
> b0cf3bd94e 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java 
> 783a672c47 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java
>  9a3f81c98f 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g 78cbf25c43 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 733ec79ce1 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java f9a6386ecf 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/Statistics.java 0057f0c2c6 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/EmptyStatsSource.java 
> PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/GroupTransformer.java 
> PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/PlanMapper.java 
> PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/PlanMapperProcess.java 
> PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/RuntimeStatsSource.java 
> PRE-CREATION 
>   
> ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/SimpleRuntimeStatsSource.java
>  PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/StatsSource.java 
> PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/refs/OperatorRef.java 
> PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/stats/OperatorStats.java PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/stats/OperatorStatsReaderHook.java 
> PRE-CREATION 
>   
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAssertTrueOOM.java
>  PRE-CREATION 
>   
> ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorGroupByOperator.java
>  fa3f89fda1 
>   ql/src/test/org/apache/hadoop/hive/ql/plan/mapping/TestCounterMapping.java 
>

[jira] [Created] (HIVE-18812) Hive Runtime Error while processing row

2018-02-27 Thread Xin shiyou (JIRA) 
Xin shiyou created HIVE-18812:
-

 Summary: Hive Runtime Error while processing row
 Key: HIVE-18812
 URL: https://issues.apache.org/jira/browse/HIVE-18812
 Project: Hive
  Issue Type: Bug
Affects Versions: 1.1.0
 Environment: 1、JAVA: 1.8.0_151
2、Hive :1.1.0-cdh5.12.1
Reporter: Xin shiyou


Error: java.lang.RuntimeException: 
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while 
processing row 

 at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:179)

 at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)

 at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459)

 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)

 at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)

 at java.security.AccessController.doPrivileged(Native Method)

 at javax.security.auth.Subject.doAs(Subject.java:422)

 at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)

 at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)

Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error 
while processing row 

 at 
org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:52)

 at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:170)

 ... 8 more

Caused by: java.lang.ArrayIndexOutOfBoundsException: 72

 at 
org.apache.hadoop.hive.ql.exec.vector.expressions.ConstantVectorExpression.evaluateLong(ConstantVectorExpression.java:99)

 at 
org.apache.hadoop.hive.ql.exec.vector.expressions.ConstantVectorExpression.evaluate(ConstantVectorExpression.java:147)

 at 
org.apache.hadoop.hive.ql.exec.vector.expressions.aggregates.VectorUDAFCount.aggregateInput(VectorUDAFCount.java:170)

 at 
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeGlobalAggregate.processBatch(VectorGroupByOperator.java:193)

 at 
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.processOp(VectorGroupByOperator.java:866)

 at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)

 at 
org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.processOp(VectorSelectOperator.java:138)

 at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)

 at 
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:98)

 at 
org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:157)

 at 
org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:45)

 ... 9 more



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)