[GitHub] hive pull request #237: HIVE-16886: HMS log notifications may have duplicate...

2018-07-26 Thread anishek
Github user anishek closed the pull request at:

https://github.com/apache/hive/pull/237


---


[GitHub] hive pull request #350: HIVE-19485 : dump directory for non native tables sh...

2018-07-26 Thread anishek
Github user anishek closed the pull request at:

https://github.com/apache/hive/pull/350


---


[GitHub] hive pull request #240: HIVE-17410 : repl load task during subsequent DAG ge...

2018-07-26 Thread anishek
GitHub user anishek reopened a pull request:

https://github.com/apache/hive/pull/240

HIVE-17410 : repl load task during subsequent DAG generation does notstart 
from the last partition processed



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/anishek/hive HIVE-17410

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/hive/pull/240.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #240


commit a0f95051ff16e07272fd1b8cd3f12d386535c0ef
Author: Anishek Agarwal 
Date:   2017-08-30T00:03:39Z

HIVE-17410 : repl load task during subsequent DAG generation does not start 
from the last partition processed

commit d4dcadcb48b727e291a7cf43bc3380f40264e4d3
Author: Anishek Agarwal 
Date:   2017-09-08T05:54:09Z

HIVE-17410 : repl load task during subsequent DAG generation does not start 
from the last partition processed

setting up the replicationState Correctly.




---


[GitHub] hive pull request #240: HIVE-17410 : repl load task during subsequent DAG ge...

2018-07-26 Thread anishek
Github user anishek closed the pull request at:

https://github.com/apache/hive/pull/240


---


[GitHub] hive pull request #262: HIVE-17825: Socket not closed when trying to read fi...

2018-07-26 Thread anishek
Github user anishek closed the pull request at:

https://github.com/apache/hive/pull/262


---


[GitHub] hive pull request #246: HIVE-17426: Execution framework in hive to run tasks...

2018-07-26 Thread anishek
Github user anishek closed the pull request at:

https://github.com/apache/hive/pull/246


---


[GitHub] hive pull request #283: HIVE-17829: ArrayIndexOutOfBoundsException - HBASE-b...

2018-07-26 Thread anishek
Github user anishek closed the pull request at:

https://github.com/apache/hive/pull/283


---


[GitHub] hive pull request #240: HIVE-17410 : repl load task during subsequent DAG ge...

2018-07-26 Thread anishek
Github user anishek closed the pull request at:

https://github.com/apache/hive/pull/240


---


[GitHub] hive pull request #259: HIVE-17615 : Task.executeTask has to be thread safe ...

2018-07-26 Thread anishek
Github user anishek closed the pull request at:

https://github.com/apache/hive/pull/259


---


[GitHub] hive pull request #286: HIVE-18352: introduce a METADATAONLY option while do...

2018-07-26 Thread anishek
Github user anishek closed the pull request at:

https://github.com/apache/hive/pull/286


---


[GitHub] hive pull request #289: HIVE-18341: Add repl load support for adding "raw" n...

2018-07-26 Thread anishek
Github user anishek closed the pull request at:

https://github.com/apache/hive/pull/289


---


[GitHub] hive pull request #263: HIVE-17830: dbnotification fails to work with rdbms ...

2018-07-26 Thread anishek
Github user anishek closed the pull request at:

https://github.com/apache/hive/pull/263


---


[GitHub] hive pull request #399: HIVE-20152: reset db state, when repl dump fails, so...

2018-07-26 Thread anishek
Github user anishek closed the pull request at:

https://github.com/apache/hive/pull/399


---


[GitHub] hive pull request #300: HIVE-18467: support whole warehouse dump / load + cr...

2018-07-26 Thread anishek
Github user anishek closed the pull request at:

https://github.com/apache/hive/pull/300


---


[jira] [Created] (HIVE-20253) nativetask cann't working in hive

2018-07-26 Thread gehaijiang (JIRA)
gehaijiang created HIVE-20253:
-

 Summary: nativetask cann't working in  hive
 Key: HIVE-20253
 URL: https://issues.apache.org/jira/browse/HIVE-20253
 Project: Hive
  Issue Type: Wish
Affects Versions: 1.2.1
 Environment: hadoop  3.0.3  

 hive sql: 

set 
mapreduce.job.map.output.collector.class=org.apache.hadoop.mapred.nativetask.NativeMapOutputCollectorDelegator;
select count(*) from test_cold;
Reporter: gehaijiang


hadoop  3.0.3, Support nativetask.  

mapred-site.xml: 


 mapreduce.job.map.output.collector.class
 
org.apache.hadoop.mapred.nativetask.NativeMapOutputCollectorDelegator
 

 

hive sql: 

set 
mapreduce.job.map.output.collector.class=org.apache.hadoop.mapred.nativetask.NativeMapOutputCollectorDelegator;
select count(*) from test_cold;

 

URL:
 
http://0.0.0.0:8088/taskdetails.jsp?jobid=job_1532646043398_0019=task_1532646043398_0019_m_00
-
Diagnostic Messages for this Task:
Error: java.io.IOException: Initialization of all the collectors failed. Error 
in last collector was:java.io.IOException: Cannot find serializer for 
org.apache.hadoop.hive.ql.io.HiveKey
 at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:423)
 at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:454)
 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:349)
 at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:422)
 at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1686)
 at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
Caused by: java.io.IOException: Cannot find serializer for 
org.apache.hadoop.hive.ql.io.HiveKey
 at 
org.apache.hadoop.mapred.nativetask.NativeMapOutputCollectorDelegator.init(NativeMapOutputCollectorDelegator.java:127)
 at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:408)
 ... 7 more

 

2018-07-27 10:08:25,391 ERROR operation.Operation (SQLOperation.java:run(209)) 
- Error running hive query:
org.apache.hive.service.cli.HiveSQLException: Error while processing statement: 
FAILED: Execution Error, return code 2 from 
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
 at 
org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:316)
 at 
org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:156)
 at 
org.apache.hive.service.cli.operation.SQLOperation.access$100(SQLOperation.java:71)
 at 
org.apache.hive.service.cli.operation.SQLOperation$1$1.run(SQLOperation.java:206)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:422)
 at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
 at 
org.apache.hive.service.cli.operation.SQLOperation$1.run(SQLOperation.java:218)
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
 at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
 at java.lang.Thread.run(Thread.java:745)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: Review Request 68072: HIVE-20247 cleanup issues in LLAP IO after cache OOM

2018-07-26 Thread Sergey Shelukhin

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68072/
---

(Updated July 27, 2018, 1:38 a.m.)


Review request for hive and Prasanth_J.


Repository: hive-git


Description
---

.


Diffs (updated)
-

  llap-server/src/java/org/apache/hadoop/hive/llap/cache/BuddyAllocator.java 
fcfc22a712 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedReaderImpl.java 
348f9df773 


Diff: https://reviews.apache.org/r/68072/diff/2/

Changes: https://reviews.apache.org/r/68072/diff/1-2/


Testing
---


Thanks,

Sergey Shelukhin



[jira] [Created] (HIVE-20252) Semijoin Reduction : Cycles due to semi join branch may remain undetected if small table side has a map join upstream.

2018-07-26 Thread Deepak Jaiswal (JIRA)
Deepak Jaiswal created HIVE-20252:
-

 Summary: Semijoin Reduction : Cycles due to semi join branch may 
remain undetected if small table side has a map join upstream.
 Key: HIVE-20252
 URL: https://issues.apache.org/jira/browse/HIVE-20252
 Project: Hive
  Issue Type: Bug
Reporter: Deepak Jaiswal
Assignee: Deepak Jaiswal


For eg,

 
 # 2018-07-26T17:22:14,664 DEBUG [51377701-dc98-424f-82e0-bbb5d6c84316 main] 
optimizer.SharedWorkOptimizer: Before SharedWorkOptimizer:
 # 
TS[0]-FIL[96]-SEL[2]-MAPJOIN[156]-MAPJOIN[157]-MAPJOIN[161]-MAPJOIN[162]-FIL[47]-SEL[48]-MAPJOIN[163]-FIL[66]-SEL[67]-TNK[105]-GBY[68]-RS[69]-GBY[70]-SEL[71]-RS[72]-SEL[73]-LIM[74]-FS[75]
 #                                                           
-SEL[142]-GBY[143]-RS[144]-GBY[145]-RS[155]
 # TS[3]-FIL[97]-SEL[5]-RS[34]-MAPJOIN[156]
 # TS[6]-FIL[98]-SEL[8]-RS[37]-MAPJOIN[157]
 # TS[9]-FIL[99]-SEL[11]-MAPJOIN[158]-GBY[40]-RS[42]-MAPJOIN[161]
 # TS[12]-FIL[100]-SEL[14]-RS[16]-MAPJOIN[158]
 #                       -SEL[131]-GBY[132]-EVENT[133]
 # 
TS[19]-FIL[101]-SEL[21]-MAPJOIN[159]-GBY[29]-RS[30]-GBY[31]-SEL[32]-RS[45]-MAPJOIN[162]
 # TS[22]-FIL[102]-SEL[24]-RS[26]-MAPJOIN[159]
 #                       -SEL[139]-GBY[140]-EVENT[141]
 # 
TS[49]-FIL[103]-SEL[51]-MAPJOIN[160]-GBY[59]-RS[60]-GBY[61]-SEL[62]-RS[64]-MAPJOIN[163]
 # TS[52]-FIL[104]-SEL[54]-RS[56]-MAPJOIN[160]
 #                       -SEL[147]-GBY[148]-EVENT[149]
 # 
 # 
 # DPP information stored in the cache: \{TS[19]=[EVENT[141]], 
TS[9]=[EVENT[133]], TS[49]=[RS[155], EVENT[149]]}

 

The semi join branch in line 3 feeds into TS[49] in line 12 which feeds to 
MAPJOIN[163] going back to parent of the semi join branch at line 2.


The logic to detect cycle may fail as there is a MAPJOIN[160] at line 12 which 
could cause the logic to look for wrong TS. The logic to find TS operator 
upstream must use findOperatorsUpstream() and examine each TS Op for complete 
coverage.

 

cc [~jcamachorodriguez]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: Review Request 68075: HIVE-20249: LLAP IO: NPE during refCount decrement

2018-07-26 Thread j . prasanth . j

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68075/
---

(Updated July 27, 2018, 12:21 a.m.)


Review request for hive and Sergey Shelukhin.


Changes
---

moved locked buffers dump to trace log


Bugs: HIVE-20249
https://issues.apache.org/jira/browse/HIVE-20249


Repository: hive-git


Description
---

HIVE-20249: LLAP IO: NPE during refCount decrement


Diffs (updated)
-

  llap-server/src/java/org/apache/hadoop/hive/llap/cache/BuddyAllocator.java 
fcfc22a 
  llap-server/src/java/org/apache/hadoop/hive/llap/cache/LowLevelCacheImpl.java 
53bdc2a 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedReaderImpl.java 
348f9df 


Diff: https://reviews.apache.org/r/68075/diff/2/

Changes: https://reviews.apache.org/r/68075/diff/1-2/


Testing
---


Thanks,

Prasanth_J



[jira] [Created] (HIVE-20251) Improve message in SharedWorkOptimizer when cycles are found in the plan

2018-07-26 Thread Jesus Camacho Rodriguez (JIRA)
Jesus Camacho Rodriguez created HIVE-20251:
--

 Summary: Improve message in SharedWorkOptimizer when cycles are 
found in the plan
 Key: HIVE-20251
 URL: https://issues.apache.org/jira/browse/HIVE-20251
 Project: Hive
  Issue Type: Improvement
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez


Currently, if there is a cycle, e.g., due to semijoin, which should not happen, 
SharedWorkOptimizer will just loop infinitely. It would be better to throw an 
Exception in those cases.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (HIVE-20250) Option to allow external tables to use query results cache

2018-07-26 Thread Jason Dere (JIRA)
Jason Dere created HIVE-20250:
-

 Summary: Option to allow external tables to use query results cache
 Key: HIVE-20250
 URL: https://issues.apache.org/jira/browse/HIVE-20250
 Project: Hive
  Issue Type: Sub-task
Reporter: Jason Dere






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Review Request 68075: HIVE-20249: LLAP IO: NPE during refCount decrement

2018-07-26 Thread j . prasanth . j

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68075/
---

Review request for hive and Sergey Shelukhin.


Bugs: HIVE-20249
https://issues.apache.org/jira/browse/HIVE-20249


Repository: hive-git


Description
---

HIVE-20249: LLAP IO: NPE during refCount decrement


Diffs
-

  llap-server/src/java/org/apache/hadoop/hive/llap/cache/BuddyAllocator.java 
fcfc22a7128f3d17af24c538a452e7b91a5e21f0 
  llap-server/src/java/org/apache/hadoop/hive/llap/cache/LowLevelCacheImpl.java 
53bdc2a49838e76f507ec6c20eb3d01ed78f17a9 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedReaderImpl.java 
348f9df773ff26b42d377e19947afaa5b1f51255 


Diff: https://reviews.apache.org/r/68075/diff/1/


Testing
---


Thanks,

Prasanth_J



[jira] [Created] (HIVE-20249) LLAP IO: NPE during refCount decrement

2018-07-26 Thread Prasanth Jayachandran (JIRA)
Prasanth Jayachandran created HIVE-20249:


 Summary: LLAP IO: NPE during refCount decrement
 Key: HIVE-20249
 URL: https://issues.apache.org/jira/browse/HIVE-20249
 Project: Hive
  Issue Type: New Feature
  Components: llap
Affects Versions: 4.0.0, 3.2.0
Reporter: Prasanth Jayachandran
Assignee: Prasanth Jayachandran


This was observed on one of the old build which was digesting the exception 
root cause.
{code:java}
Ignoring exception when closing input calls(cleanup). Exception 
class=java.lang.NullPointerException

java.lang.NullPointerException: null
at 
org.apache.hadoop.hive.llap.cache.BuddyAllocator$Arena.deallocate(BuddyAllocator.java:1355)
 ~[hive-llap-server-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at 
org.apache.hadoop.hive.llap.cache.BuddyAllocator.deallocate(BuddyAllocator.java:685)
 ~[hive-llap-server-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.io.orc.encoded.EncodedReaderImpl.releaseInitialRefcounts(EncodedReaderImpl.java:676)
 ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.io.orc.encoded.EncodedReaderImpl.readEncodedColumns(EncodedReaderImpl.java:543)
 ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at 
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.performDataRead(OrcEncodedDataReader.java:404)
 ~[hive-llap-server-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at 
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader$4.run(OrcEncodedDataReader.java:263)
 ~[hive-llap-server-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at 
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader$4.run(OrcEncodedDataReader.java:260)
 ~[hive-llap-server-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]
at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_112]
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
 ~[hadoop-common-3.0.0.3.0.0.0-SNAPSHOT.jar:?]
at 
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:260)
 ~[hive-llap-server-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at 
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:109)
 ~[hive-llap-server-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) 
~[tez-common-0.9.2-SNAPSHOT.jar:0.9.2-SNAPSHOT]
at 
org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:110)
 ~[hive-llap-server-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_112]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
~[?:1.8.0_112]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
~[?:1.8.0_112]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (HIVE-20248) clean up some TODOs after txn stats merge

2018-07-26 Thread Sergey Shelukhin (JIRA)
Sergey Shelukhin created HIVE-20248:
---

 Summary: clean up some TODOs after txn stats merge
 Key: HIVE-20248
 URL: https://issues.apache.org/jira/browse/HIVE-20248
 Project: Hive
  Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Review Request 68072: HIVE-20247 cleanup issues in LLAP IO after cache OOM

2018-07-26 Thread Sergey Shelukhin

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68072/
---

Review request for hive and Prasanth_J.


Repository: hive-git


Description
---

.


Diffs
-

  llap-server/src/java/org/apache/hadoop/hive/llap/cache/BuddyAllocator.java 
fcfc22a712 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedReaderImpl.java 
348f9df773 


Diff: https://reviews.apache.org/r/68072/diff/1/


Testing
---


Thanks,

Sergey Shelukhin



[jira] [Created] (HIVE-20247) cleanup issues in LLAP IO after cache OOM

2018-07-26 Thread Sergey Shelukhin (JIRA)
Sergey Shelukhin created HIVE-20247:
---

 Summary: cleanup issues in LLAP IO after cache OOM
 Key: HIVE-20247
 URL: https://issues.apache.org/jira/browse/HIVE-20247
 Project: Hive
  Issue Type: Bug
Reporter: Prasanth Jayachandran
Assignee: Sergey Shelukhin


LLAP IO creates unallocated buffer objects inside the read-related data 
structures, then allocates them in bulk, then decompresses into them and 
increfs them.
If allocate or decompress steps fail, it's hard for the higher-level cleanup to 
tell what the state of the buffers in the read-related structures is - they may 
be unallocated, allocated but not incref-ed, or incref-ed.
Some cleanup paths only deal with the latter case, resulting in bugs.

This only happens on one paths, others allocate and use buffers in a single 
place.




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (HIVE-20246) Make some collect stats flags be user configurable

2018-07-26 Thread Alice Fan (JIRA)
Alice Fan created HIVE-20246:


 Summary: Make some collect stats flags be user configurable
 Key: HIVE-20246
 URL: https://issues.apache.org/jira/browse/HIVE-20246
 Project: Hive
  Issue Type: Improvement
  Components: Metastore
Reporter: Alice Fan
Assignee: Alice Fan
 Fix For: 4.0.0


When hive.stats.autogather=true then the Metastore lists all files under the 
table directory to populate basic stats like file counts and sizes. This file 
listing operation can be very expensive particularly on filesystems like S3.
One way to address this issue is to reconfigure hive.stats.autogather=false.
However, set metaconf:hive.stats.autogather=false will not be taken by 
HiveMetaStore when user set this in session.




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Review Request 68070: HIVE-20244 forward port HIVE-19704 to master

2018-07-26 Thread Sergey Shelukhin

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68070/
---

Review request for hive and Prasanth_J.


Repository: hive-git


Description
---

see jira


Diffs
-

  llap-server/src/java/org/apache/hadoop/hive/llap/cache/BuddyAllocator.java 
fcfc22a712 
  
llap-server/src/java/org/apache/hadoop/hive/llap/cache/LowLevelCacheMemoryManager.java
 4297cfc61d 
  llap-server/src/java/org/apache/hadoop/hive/llap/cache/MemoryManager.java 
542041d133 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/OrcEncodedDataReader.java
 b76b0ded98 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/SerDeEncodedDataReader.java
 5b54af5d3b 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/metadata/MetadataCache.java 
426d599b29 
  
llap-server/src/test/org/apache/hadoop/hive/llap/cache/TestBuddyAllocator.java 
1e6f3ac96d 
  
llap-server/src/test/org/apache/hadoop/hive/llap/cache/TestLowLevelLrfuCachePolicy.java
 6eb2eb5089 
  
llap-server/src/test/org/apache/hadoop/hive/llap/cache/TestOrcMetadataCache.java
 df20f20c8f 
  ql/src/java/org/apache/hadoop/hive/llap/LlapCacheAwareFs.java dcb24b8018 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedReader.java 
f6b949e51b 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedReaderImpl.java 
348f9df773 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/StoppableAllocator.java 
PRE-CREATION 
  
ql/src/java/org/apache/hadoop/hive/ql/io/parquet/vector/VectorizedParquetRecordReader.java
 f64efe26f5 
  storage-api/src/java/org/apache/hadoop/hive/common/io/FileMetadataCache.java 
d1da7f5de8 


Diff: https://reviews.apache.org/r/68070/diff/1/


Testing
---


Thanks,

Sergey Shelukhin



Review Request 68069: HIVE-20240

2018-07-26 Thread Deepak Jaiswal

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68069/
---

Review request for hive and Jason Dere.


Bugs: HIVE-20240
https://issues.apache.org/jira/browse/HIVE-20240


Repository: hive-git


Description
---

See Jira


Diffs
-

  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java
 caec2c08e9 
  ql/src/test/queries/clientpositive/dynamic_semijoin_reduction_4.q a04ab666e0 
  ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction_4.q.out 
0feb362023 


Diff: https://reviews.apache.org/r/68069/diff/1/


Testing
---


Thanks,

Deepak Jaiswal



Re: Review Request 68034: HIVE-19568 Active/Passive HS2 HA: Disallow direct connection to passive HS2 instance

2018-07-26 Thread Sergey Shelukhin

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68034/
---

(Updated July 26, 2018, 7:03 p.m.)


Review request for hive and Prasanth_J.


Repository: hive-git


Description
---

see jira


Diffs (updated)
-

  itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestActivePassiveHA.java 
4055f13fd0 
  
itests/hive-unit/src/test/java/org/apache/hive/service/cli/session/TestQueryDisplay.java
 8b28e2dec0 
  itests/util/src/main/java/org/apache/hive/jdbc/miniHS2/MiniHS2.java 
a78dd739b1 
  service/src/java/org/apache/hive/service/cli/CLIService.java dbfaf7154e 
  service/src/java/org/apache/hive/service/cli/session/SessionManager.java 
e9649824f1 
  
service/src/java/org/apache/hive/service/cli/thrift/EmbeddedThriftBinaryCLIService.java
 8b61874f93 
  service/src/java/org/apache/hive/service/server/HiveServer2.java 432a341bef 
  service/src/test/org/apache/hive/service/auth/TestPlainSaslHelper.java 
14e28323e0 
  
service/src/test/org/apache/hive/service/cli/TestCLIServiceConnectionLimits.java
 5ecea9a08b 
  service/src/test/org/apache/hive/service/cli/TestCLIServiceRestore.java 
6b69d4d0af 
  
service/src/test/org/apache/hive/service/cli/TestRetryingThriftCLIServiceClient.java
 7bae62d977 
  
service/src/test/org/apache/hive/service/cli/session/TestPluggableHiveSessionImpl.java
 90237c088f 
  
service/src/test/org/apache/hive/service/cli/session/TestSessionGlobalInitFile.java
 af7a72e70f 
  
service/src/test/org/apache/hive/service/cli/session/TestSessionManagerMetrics.java
 d954692e98 


Diff: https://reviews.apache.org/r/68034/diff/5/

Changes: https://reviews.apache.org/r/68034/diff/4-5/


Testing
---


Thanks,

Sergey Shelukhin



Re: Review Request 68034: HIVE-19568 Active/Passive HS2 HA: Disallow direct connection to passive HS2 instance

2018-07-26 Thread Sergey Shelukhin

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68034/
---

(Updated July 26, 2018, 7 p.m.)


Review request for hive and Prasanth_J.


Repository: hive-git


Description
---

see jira


Diffs (updated)
-

  itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestActivePassiveHA.java 
4055f13fd0 
  
itests/hive-unit/src/test/java/org/apache/hive/service/cli/session/TestQueryDisplay.java
 8b28e2dec0 
  itests/util/src/main/java/org/apache/hive/jdbc/miniHS2/MiniHS2.java 
a78dd739b1 
  service/src/java/org/apache/hive/service/cli/CLIService.java dbfaf7154e 
  service/src/java/org/apache/hive/service/cli/session/SessionManager.java 
e9649824f1 
  
service/src/java/org/apache/hive/service/cli/thrift/EmbeddedThriftBinaryCLIService.java
 8b61874f93 
  service/src/java/org/apache/hive/service/server/HiveServer2.java 432a341bef 
  service/src/test/org/apache/hive/service/auth/TestPlainSaslHelper.java 
14e28323e0 
  
service/src/test/org/apache/hive/service/cli/TestCLIServiceConnectionLimits.java
 5ecea9a08b 
  service/src/test/org/apache/hive/service/cli/TestCLIServiceRestore.java 
6b69d4d0af 
  
service/src/test/org/apache/hive/service/cli/TestRetryingThriftCLIServiceClient.java
 7bae62d977 
  
service/src/test/org/apache/hive/service/cli/session/TestPluggableHiveSessionImpl.java
 90237c088f 
  
service/src/test/org/apache/hive/service/cli/session/TestSessionGlobalInitFile.java
 af7a72e70f 
  
service/src/test/org/apache/hive/service/cli/session/TestSessionManagerMetrics.java
 d954692e98 


Diff: https://reviews.apache.org/r/68034/diff/4/

Changes: https://reviews.apache.org/r/68034/diff/3-4/


Testing
---


Thanks,

Sergey Shelukhin



[ANNOUNCE] New PMC Member : Sahil Takiar

2018-07-26 Thread Ashutosh Chauhan
On behalf of the Hive PMC I am delighted to announce Sahil Takiar is
joining Hive PMC.
Thanks Sahil for all your contributions till now. Looking forward to many
more.

Welcome, Sahil!

Thanks,
Ashutosh


[ANNOUNCE] New PMC Member : Vihang Karajgaonkar

2018-07-26 Thread Ashutosh Chauhan
On behalf of the Hive PMC I am delighted to announce Vihang  Karajgaonkar
is joining Hive PMC.
Thanks Vihang for all your contributions till now. Looking forward to many
more.

Welcome, Vihang!

Thanks,
Ashutosh


[ANNOUNCE] New PMC Member : Peter Vary

2018-07-26 Thread Ashutosh Chauhan
On behalf of the Hive PMC I am delighted to announce Peter Vary is joining
Hive PMC.
Thanks Peter for all your contributions till now. Looking forward to many
more.

Welcome, Peter!

Thanks,
Ashutosh


Re: [VOTE] Apache Hive 3.1.0 Release Candidate 1

2018-07-26 Thread Jesus Camacho Rodriguez
+1

* Built from sources and ran tests
* Checked signatures

-Jesús

On 7/26/18, 1:41 AM, "Peter Vary"  wrote:

Never done this before, but here is what I did:
- Built from source, added my usual configs, and run some basic query tests
- Downloaded the artifacts from the URL Vineet provided
- Checked sha and asc signatures
- Checked that the source tar.gz contains the same files that my repository

If this considered enough, then +1 from me too.

> On Jul 26, 2018, at 06:57, Vineet Garg  wrote:
> 
> Thanks for voting Vihang and Ashutosh. 
> 
> I need 2 more votes for the release. Hive PMC members please test and 
vote.
> 
> Thanks,
> Vineet
> 
>> On Jul 24, 2018, at 4:05 PM, Ashutosh Chauhan  
wrote:
>> 
>> Built from sources.
>> Ran some unit tests.
>> Checked sha checksums.
>> 
>> Everything looks good.
>> +1
>> 
>> 
>> On Tue, Jul 24, 2018 at 2:27 PM Vihang Karajgaonkar
>>  wrote:
>> 
>>>  - Built the source and ran some basic hive commands
>>>  - Built standalone-metastore and deployed it with a non-hive
>>>  application. Tested basic metastore operations like create, alter, drop
>>> on
>>>  tables and partitions
>>>  - Verified the signature for the tar files
>>> 
>>> RC1 looks good to me.
>>> 
>>> +1 (non-binding)
>>> 
>>> On Mon, Jul 23, 2018 at 4:28 PM, Vineet Garg 
>>> wrote:
>>> 
 Apache Hive 3.1.0 Release Candidate 1 is available here:
 
 http://people.apache.org/~vgarg/apache-hive-3.1.0-rc-1
 
 Maven artifacts are available here:
 
 https://repository.apache.org/content/repositories/orgapachehive-1090/
 
 Source tag: https://github.com/apache/hive/tree/release-3.1.0-rc1
 
 Voting will conclude in 72 hours.
 
 Hive PMC Members: Please test and vote.
 
 Thanks.
 
>>> 
> 





Re: [VOTE] Apache Hive 3.1.0 Release Candidate 1

2018-07-26 Thread Peter Vary
Never done this before, but here is what I did:
- Built from source, added my usual configs, and run some basic query tests
- Downloaded the artifacts from the URL Vineet provided
- Checked sha and asc signatures
- Checked that the source tar.gz contains the same files that my repository

If this considered enough, then +1 from me too.

> On Jul 26, 2018, at 06:57, Vineet Garg  wrote:
> 
> Thanks for voting Vihang and Ashutosh. 
> 
> I need 2 more votes for the release. Hive PMC members please test and vote.
> 
> Thanks,
> Vineet
> 
>> On Jul 24, 2018, at 4:05 PM, Ashutosh Chauhan  wrote:
>> 
>> Built from sources.
>> Ran some unit tests.
>> Checked sha checksums.
>> 
>> Everything looks good.
>> +1
>> 
>> 
>> On Tue, Jul 24, 2018 at 2:27 PM Vihang Karajgaonkar
>>  wrote:
>> 
>>>  - Built the source and ran some basic hive commands
>>>  - Built standalone-metastore and deployed it with a non-hive
>>>  application. Tested basic metastore operations like create, alter, drop
>>> on
>>>  tables and partitions
>>>  - Verified the signature for the tar files
>>> 
>>> RC1 looks good to me.
>>> 
>>> +1 (non-binding)
>>> 
>>> On Mon, Jul 23, 2018 at 4:28 PM, Vineet Garg 
>>> wrote:
>>> 
 Apache Hive 3.1.0 Release Candidate 1 is available here:
 
 http://people.apache.org/~vgarg/apache-hive-3.1.0-rc-1
 
 Maven artifacts are available here:
 
 https://repository.apache.org/content/repositories/orgapachehive-1090/
 
 Source tag: https://github.com/apache/hive/tree/release-3.1.0-rc1
 
 Voting will conclude in 72 hours.
 
 Hive PMC Members: Please test and vote.
 
 Thanks.
 
>>> 
> 



[jira] [Created] (HIVE-20245) Vectorization: Fix NULL / Wrong Results issues in BETWEEN / IN

2018-07-26 Thread Matt McCline (JIRA)
Matt McCline created HIVE-20245:
---

 Summary: Vectorization: Fix NULL / Wrong Results issues in BETWEEN 
/ IN
 Key: HIVE-20245
 URL: https://issues.apache.org/jira/browse/HIVE-20245
 Project: Hive
  Issue Type: Bug
  Components: Hive
Reporter: Matt McCline
Assignee: Matt McCline


Write new UT tests that use random data and intentional isRepeating batches to 
checks for NULL and Wrong Results for vectorized BETWEEN and IN.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)