Re: Review Request 63458: HIVE-16917: HiveServer2 guard rails - Limit concurrent connections from user

2017-11-01 Thread j . prasanth . j 


> On Nov. 2, 2017, 3:19 a.m., Alexander Kolbasov wrote:
> > service/src/java/org/apache/hive/service/cli/session/SessionManager.java
> > Lines 477 (patched)
> > 
> >
> > This potentially leak IP addresses of clients in the log file - would 
> > be better to avoid it.

removed it.


- Prasanth_J


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63458/#review189871
---


On Nov. 1, 2017, 2:04 a.m., Prasanth_J wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63458/
> ---
> 
> (Updated Nov. 1, 2017, 2:04 a.m.)
> 
> 
> Review request for hive, Sergey Shelukhin and Thejas Nair.
> 
> 
> Bugs: HIVE-16917
> https://issues.apache.org/jira/browse/HIVE-16917
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-16917: HiveServer2 guard rails - Limit concurrent connections from user
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
> a3c853a5c5a2a3e72ae0141d0ad836ec95516280 
>   service/src/java/org/apache/hive/service/cli/session/SessionManager.java 
> 9b2ae57ee498e9b712cc87b16ba223995519611c 
>   service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java 
> 6354c8c5450eaa8ab91dd59387f2eddf0eb84389 
>   
> service/src/test/org/apache/hive/service/cli/TestCLIServiceConnectionLimits.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/63458/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Prasanth_J
> 
>

Re: Review Request 63458: HIVE-16917: HiveServer2 guard rails - Limit concurrent connections from user

2017-11-01 Thread j . prasanth . j 


> On Nov. 2, 2017, 1:34 a.m., Sergey Shelukhin wrote:
> > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
> > Lines 2475 (patched)
> > 
> >
> > hmm... would it make sense to have 3 limits instead? per user, per ip, 
> > and per user+ip (if needed). All would be enforced. Or is it going to be 
> > extensible?

Yeah. Makes sense to have 3 limits. Updated patch. All 3 are tracked now.


> On Nov. 2, 2017, 1:34 a.m., Sergey Shelukhin wrote:
> > service/src/java/org/apache/hive/service/cli/session/SessionManager.java
> > Lines 472 (patched)
> > 
> >
> > nit: map is concurrent, it should be possible to use putIfAbsent and 
> > replace w/o synchronize

update with java 8 one liner.


- Prasanth_J


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63458/#review189860
---


On Nov. 1, 2017, 2:04 a.m., Prasanth_J wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63458/
> ---
> 
> (Updated Nov. 1, 2017, 2:04 a.m.)
> 
> 
> Review request for hive, Sergey Shelukhin and Thejas Nair.
> 
> 
> Bugs: HIVE-16917
> https://issues.apache.org/jira/browse/HIVE-16917
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-16917: HiveServer2 guard rails - Limit concurrent connections from user
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
> a3c853a5c5a2a3e72ae0141d0ad836ec95516280 
>   service/src/java/org/apache/hive/service/cli/session/SessionManager.java 
> 9b2ae57ee498e9b712cc87b16ba223995519611c 
>   service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java 
> 6354c8c5450eaa8ab91dd59387f2eddf0eb84389 
>   
> service/src/test/org/apache/hive/service/cli/TestCLIServiceConnectionLimits.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/63458/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Prasanth_J
> 
>

Re: Review Request 63470: HIVE-17767 Rewrite correlated EXISTS/IN subqueries into LEFT SEMI JOIN

2017-11-01 Thread Vineet Garg 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63470/
---

(Updated Nov. 2, 2017, 4:51 a.m.)


Review request for hive and Ashutosh Chauhan.


Changes
---

Addressed review comments


Bugs: HIVE-17767
https://issues.apache.org/jira/browse/HIVE-17767


Repository: hive-git


Description
---

This patch does the following:
* Added back earlier patch to generate value generator
* Added logic to rewrite EXISTS/IN correlated subqueries into LEFT SEMI JOIN
* Remove SemiJoinTransposeRule (This rule pushes semi join underneath its left 
join which might not be semantically correct thing to do)


Diffs (updated)
-

  itests/src/test/resources/testconfiguration.properties 462f332e99 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveSubQRemoveRelBuilder.java
 3a1897f4aa 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRelDecorrelator.java
 62125f0fb7 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSubQueryRemoveRule.java
 2dca6a25ac 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/PlanModifierForASTConv.java
 5e8a994873 
  ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 80351bef87 
  ql/src/test/queries/clientpositive/subquery_exists.q 19c42f0c29 
  ql/src/test/queries/clientpositive/subquery_in.q 4ba170a706 
  ql/src/test/results/clientpositive/constprog_partitioner.q.out 87618df902 
  ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction_2.q.out 
87e08fbcde 
  ql/src/test/results/clientpositive/llap/explainuser_1.q.out 6e55acf0d8 
  ql/src/test/results/clientpositive/llap/subquery_exists.q.out e206f0851e 
  ql/src/test/results/clientpositive/llap/subquery_in.q.out af42131bc2 
  ql/src/test/results/clientpositive/llap/subquery_multi.q.out 96fe17a05a 
  ql/src/test/results/clientpositive/llap/subquery_notin.q.out 8e2ca937af 
  ql/src/test/results/clientpositive/llap/subquery_scalar.q.out c89d053b4a 
  ql/src/test/results/clientpositive/llap/subquery_select.q.out 118f6ebccf 
  ql/src/test/results/clientpositive/llap/subquery_views.q.out a9a81133b5 
  ql/src/test/results/clientpositive/llap/vector_mapjoin_reduce.q.out 
4e6f00f6b7 
  ql/src/test/results/clientpositive/masking_12.q.out 540c53e825 
  ql/src/test/results/clientpositive/masking_3.q.out 1114c80676 
  ql/src/test/results/clientpositive/masking_4.q.out 527da21610 
  ql/src/test/results/clientpositive/perf/spark/query10.q.out eb3a2f6699 
  ql/src/test/results/clientpositive/perf/spark/query16.q.out b74d721d41 
  ql/src/test/results/clientpositive/perf/spark/query35.q.out 8759b71b8c 
  ql/src/test/results/clientpositive/perf/spark/query69.q.out e4430beaac 
  ql/src/test/results/clientpositive/perf/spark/query94.q.out 43b8c77bdc 
  ql/src/test/results/clientpositive/perf/tez/query10.q.out cf3651b35b 
  ql/src/test/results/clientpositive/perf/tez/query14.q.out b2a45f155a 
  ql/src/test/results/clientpositive/perf/tez/query16.q.out a7b710d6e1 
  ql/src/test/results/clientpositive/perf/tez/query23.q.out 7112de61d9 
  ql/src/test/results/clientpositive/perf/tez/query35.q.out a72f57816e 
  ql/src/test/results/clientpositive/perf/tez/query69.q.out 591f3fcdb0 
  ql/src/test/results/clientpositive/perf/tez/query94.q.out 7674aa7f7c 
  ql/src/test/results/clientpositive/semijoin5.q.out 533c077f58 
  ql/src/test/results/clientpositive/spark/constprog_partitioner.q.out 
b89f9f5905 
  ql/src/test/results/clientpositive/spark/spark_explainuser_1.q.out 76c74d9ab7 
  ql/src/test/results/clientpositive/spark/subquery_exists.q.out dafe5b6d5b 
  ql/src/test/results/clientpositive/spark/subquery_in.q.out 471c2ccd94 
  ql/src/test/results/clientpositive/spark/subquery_multi.q.out ff519fda09 
  ql/src/test/results/clientpositive/spark/subquery_notin.q.out 1b2c0880ae 
  ql/src/test/results/clientpositive/spark/subquery_scalar.q.out de005ada82 
  ql/src/test/results/clientpositive/spark/subquery_select.q.out 7d3a16b6ee 
  ql/src/test/results/clientpositive/spark/subquery_views.q.out 91e39913a7 
  ql/src/test/results/clientpositive/spark/vector_mapjoin_reduce.q.out 
81af937e97 
  ql/src/test/results/clientpositive/subquery_exists.q.out c9f2a79041 
  ql/src/test/results/clientpositive/subquery_exists_having.q.out 2c41ff6c33 
  ql/src/test/results/clientpositive/subquery_in_having.q.out 6893442b61 
  ql/src/test/results/clientpositive/subquery_notexists.q.out 329573e8e1 
  ql/src/test/results/clientpositive/subquery_notexists_having.q.out 4d2b2fc873 
  ql/src/test/results/clientpositive/subquery_notin_having.q.out c321fe69ed 
  ql/src/test/results/clientpositive/subquery_unqualcolumnrefs.q.out 5c306f6b47 
  ql/src/test/results/clientpositive/vector_mapjoin_reduce.q.out ddea584990 


Diff: https://reviews.apache.org/r/63470/diff/2/

Changes: https://reviews.apache.org/r/63470/diff/1-2/


Testing
---

Re: ptest not picking patches

2017-11-01 Thread Vineet Garg 
Same here. I have submitted/re-submitted patches for two jiras and they haven’t 
been picked up for few days now. I saw one of them in queue today but I no 
longer see it.

Vineet

> On Nov 1, 2017, at 9:27 PM, Prasanth Jayachandran 
>  wrote:
> 
> There is definitely something going on. I submitted multiple patches for 
> precommit today and all of them are gone from the queue. 
> 
> Thanks
> Prasanth
> 
> 
>> On Nov 1, 2017, at 11:28 AM, Vihang Karajgaonkar  wrote:
>> 
>> Is there a way to find out if Jenkins was restarted since we submitted the
>> patch? Not sure if we can fix this in our precommit job. There have been
>> far too many instances recently when Jenkins was either restarted or was
>> non-responsive for long durations which delayed testing for our patches.
>> 
>> On Wed, Nov 1, 2017 at 11:22 AM, Ashutosh Chauhan 
>> wrote:
>> 
>>> There was an apache infra issue which was making jenkins to not submit
>>> patches to ptest server. https://issues.apache.org/jira/browse/INFRA-15358
>>> Also jenkins was restarted overnight.
>>> 
>>> On Wed, Nov 1, 2017 at 11:15 AM, Chris Drome 
>>> wrote:
>>> 
 Me as well for HIVE-17853.
 
 Attempting to cancel/resubmit patches.
 
 On Wed, Nov 1, 2017 at 10:50 AM, Prasanth Jayachandran <
 pjayachand...@hortonworks.com> wrote:
 
> Me too. Seen this yesterday with HIVE-17834.
> 
> Thanks
> Prasanth
> 
> 
> 
> On Wed, Nov 1, 2017 at 10:26 AM -0700, "Deepak Jaiswal" <
> djais...@hortonworks.com> wrote:
> 
> 
> Hi,
> 
> I uploaded couple of patches but they don't appear in the Pre-commit
 tests
> queue.
> Is anyone else facing this?
> 
> Regards,
> Deepak
> 
> 
 
>>> 
>

Re: ptest not picking patches

2017-11-01 Thread Prasanth Jayachandran 
There is definitely something going on. I submitted multiple patches for 
precommit today and all of them are gone from the queue. 

Thanks
Prasanth


> On Nov 1, 2017, at 11:28 AM, Vihang Karajgaonkar  wrote:
> 
> Is there a way to find out if Jenkins was restarted since we submitted the
> patch? Not sure if we can fix this in our precommit job. There have been
> far too many instances recently when Jenkins was either restarted or was
> non-responsive for long durations which delayed testing for our patches.
> 
> On Wed, Nov 1, 2017 at 11:22 AM, Ashutosh Chauhan 
> wrote:
> 
>> There was an apache infra issue which was making jenkins to not submit
>> patches to ptest server. https://issues.apache.org/jira/browse/INFRA-15358
>> Also jenkins was restarted overnight.
>> 
>> On Wed, Nov 1, 2017 at 11:15 AM, Chris Drome 
>> wrote:
>> 
>>> Me as well for HIVE-17853.
>>> 
>>> Attempting to cancel/resubmit patches.
>>> 
>>> On Wed, Nov 1, 2017 at 10:50 AM, Prasanth Jayachandran <
>>> pjayachand...@hortonworks.com> wrote:
>>> 
 Me too. Seen this yesterday with HIVE-17834.
 
 Thanks
 Prasanth
 
 
 
 On Wed, Nov 1, 2017 at 10:26 AM -0700, "Deepak Jaiswal" <
 djais...@hortonworks.com> wrote:
 
 
 Hi,
 
 I uploaded couple of patches but they don't appear in the Pre-commit
>>> tests
 queue.
 Is anyone else facing this?
 
 Regards,
 Deepak
 
 
>>> 
>>

Re: Review Request 63458: HIVE-16917: HiveServer2 guard rails - Limit concurrent connections from user

2017-11-01 Thread Alexander Kolbasov 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63458/#review189871
---




common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
Lines 2477 (patched)


nit: missing space between words



common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
Lines 2479 (patched)


nit: missing space between words



service/src/java/org/apache/hive/service/cli/session/SessionManager.java
Lines 472 (patched)


Right, you may actually use a combination of concurrent hashmap with atomics



service/src/java/org/apache/hive/service/cli/session/SessionManager.java
Lines 477 (patched)


This potentially leak IP addresses of clients in the log file - would be 
better to avoid it.



service/src/java/org/apache/hive/service/cli/session/SessionManager.java
Lines 497 (patched)


why is this <= 0 rather then == 0?



service/src/java/org/apache/hive/service/cli/session/SessionManager.java
Lines 503 (patched)


This is leaking IP addresses in the log file


- Alexander Kolbasov


On Nov. 1, 2017, 2:04 a.m., Prasanth_J wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63458/
> ---
> 
> (Updated Nov. 1, 2017, 2:04 a.m.)
> 
> 
> Review request for hive, Sergey Shelukhin and Thejas Nair.
> 
> 
> Bugs: HIVE-16917
> https://issues.apache.org/jira/browse/HIVE-16917
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-16917: HiveServer2 guard rails - Limit concurrent connections from user
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
> a3c853a5c5a2a3e72ae0141d0ad836ec95516280 
>   service/src/java/org/apache/hive/service/cli/session/SessionManager.java 
> 9b2ae57ee498e9b712cc87b16ba223995519611c 
>   service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java 
> 6354c8c5450eaa8ab91dd59387f2eddf0eb84389 
>   
> service/src/test/org/apache/hive/service/cli/TestCLIServiceConnectionLimits.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/63458/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Prasanth_J
> 
>

[jira] [Created] (HIVE-17963) Fix for HIVE-17113 can be improved for non-blobstore filesystems

2017-11-01 Thread Jason Dere (JIRA) 
Jason Dere created HIVE-17963:
-

 Summary: Fix for HIVE-17113 can be improved for non-blobstore 
filesystems
 Key: HIVE-17963
 URL: https://issues.apache.org/jira/browse/HIVE-17963
 Project: Hive
  Issue Type: Bug
Reporter: Jason Dere
Assignee: Jason Dere
Priority: Major


HIVE-17113/HIVE-17813 fix the duplicate file issue by performing file moves on 
a file-by-file basis. For non-blobstore filesystems this results in many more 
filesystem/namenode operations compared to the previous 
Utilities.mvFileToFinalPath() behavior (dedup files in src dir, rename src dir 
to final dir).
For non-blobstore filesystems, a better solution would be the one described 
[here|https://issues.apache.org/jira/browse/HIVE-17113?focusedCommentId=16100564=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16100564]:

1) Move the temp directory to a new directory name, to prevent additional files 
from being added by any runaway processes.
2) Run removeTempOrDuplicateFiles() on this renamed temp directory
3) Run renameOrMoveFiles() to move the renamed temp directory to the final 
location.

This results in only one additional file operation in non-blobstore FSes 
compared to the original Utilities.mvFileToFinalPath() behavior.

The proposal is to do away with the config setting 
hive.exec.move.files.from.source.dir and always have behavior that should take 
care of the duplicate file issue described in HIVE-17113. For non-blobstore 
filesystems we will do steps 1-3 described above. For blobstore filesystems we 
will do the solution done in HIVE-17113/HIVE-17813 which does the file-by-file 
copy - this should have the same number of file operations as doing a rename 
directory on blobstore, which effectively results in file moves on a 
file-by-file basis.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 63490: HIVE-17907 enable and apply resource plan commands in HS2

2017-11-01 Thread Sergey Shelukhin 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63490/
---

(Updated Nov. 2, 2017, 1:51 a.m.)


Review request for hive, Harish Jaiprakash and Prasanth_J.


Repository: hive-git


Description
---

see jira


Diffs (updated)
-

  
itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
 5a627ce6bc173d390086534d73156dfc3f3674b4 
  
itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestTriggersWorkloadManager.java
 fdb660af03305225918a46373617c16e2ea8c02f 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 
7337c1ca6cea782d052b40a04a61d8ad4a6c0e25 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 
dfb0a6d6aaf9d9c6b61efc2ab76707c65074d055 
  metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 
5aa20c582b857568b75905613a0389970bb3c2f5 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
 610b9fafd5d8ec3c42e099bde5235ca507b0101b 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
 84b70d88b00557a217952f4da74e1e839ac3b7a6 
  ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 
591be49af127b5301a53bd3ae0ca18fc9af640c8 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/UserPoolMapping.java 
81d6b859a6892eba6ae3c29e8209f10a547d4690 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/WorkloadManager.java 
35e5710629324f7350bf07a9b2372d11dbfc5474 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 
70656feea75a702a18328bbb557eb7e6a27a5045 
  ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 
6a2ff75c8468e7b1dd714046bc2a35abf74c95e6 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 
306559c7229f5df57687b460feaad3240a3674bd 
  ql/src/java/org/apache/hadoop/hive/ql/plan/AlterResourcePlanDesc.java 
f0658a665e7d13b21a0fa4cb96de1c85c6797b91 
  ql/src/java/org/apache/hadoop/hive/ql/plan/DDLWork.java 
8152cfeb5967a1c020bc2018fd99d2cbc4dbbcd3 
  ql/src/test/org/apache/hadoop/hive/ql/exec/tez/TestWorkloadManager.java 
258a865fefa967e84db381493566b706f891abcf 
  ql/src/test/results/clientpositive/llap/resourceplan.q.out 
5cdfc9de3e7f0ca9c7c9902aeeef83a06b345c90 
  service/src/java/org/apache/hive/service/server/HiveServer2.java 
2c4fe7fc2ba5dd58b62d6b2f3df44de124a381b9 
  
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java
 62801c53853dbafb7c425cff943ec819dcee4800 
  
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/RawStore.java
 a99b8e123ebb700c3754d7c7b8e5881e7781a8bd 
  
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java
 e13dcd20a30dd129ce01a45d886e819378a748a7 
  standalone-metastore/src/main/thrift/hive_metastore.thrift 
60531457b7a37424d0bdb6d39a15fc7d6d89de66 


Diff: https://reviews.apache.org/r/63490/diff/2/

Changes: https://reviews.apache.org/r/63490/diff/1-2/


Testing
---


Thanks,

Sergey Shelukhin

Re: Review Request 63381: HIVE-17926: Support triggers for non-pool sessions

2017-11-01 Thread j . prasanth . j 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63381/#review189864
---




service/src/java/org/apache/hive/service/server/HiveServer2.java
Line 140 (original), 142 (patched)


It was old code. When initialize default sessions is not set (or if queue 
names are empty), hasInitialSession will default to false and will get new 
session on every invocation of tezTask.getSession().


- Prasanth_J


On Oct. 31, 2017, 7:27 p.m., Prasanth_J wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63381/
> ---
> 
> (Updated Oct. 31, 2017, 7:27 p.m.)
> 
> 
> Review request for hive and Sergey Shelukhin.
> 
> 
> Bugs: HIVE-17926
> https://issues.apache.org/jira/browse/HIVE-17926
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-17926: Support triggers for non-pool sessions
> 
> 
> Diffs
> -
> 
>   
> itests/hive-unit/src/test/java/org/apache/hive/jdbc/AbstractJdbcTriggersTest.java
>  PRE-CREATION 
>   
> itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestTriggersNoTezSessionPool.java
>  PRE-CREATION 
>   
> itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestTriggersTezSessionPoolManager.java
>  aeca663 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPool.java b67c933 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java 
> 9b4714f 
>   service/src/java/org/apache/hive/service/server/HiveServer2.java 2c4fe7f 
> 
> 
> Diff: https://reviews.apache.org/r/63381/diff/4/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Prasanth_J
> 
>

Re: Review Request 63381: HIVE-17926: Support triggers for non-pool sessions

2017-11-01 Thread Sergey Shelukhin 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63381/#review189863
---




service/src/java/org/apache/hive/service/server/HiveServer2.java
Line 140 (original), 142 (patched)


hmm.. without calling setupPool, some fields like initConf, maybe others, 
that are used in various places, will not be initialized. Is it safe?


- Sergey Shelukhin


On Oct. 31, 2017, 7:27 p.m., Prasanth_J wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63381/
> ---
> 
> (Updated Oct. 31, 2017, 7:27 p.m.)
> 
> 
> Review request for hive and Sergey Shelukhin.
> 
> 
> Bugs: HIVE-17926
> https://issues.apache.org/jira/browse/HIVE-17926
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-17926: Support triggers for non-pool sessions
> 
> 
> Diffs
> -
> 
>   
> itests/hive-unit/src/test/java/org/apache/hive/jdbc/AbstractJdbcTriggersTest.java
>  PRE-CREATION 
>   
> itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestTriggersNoTezSessionPool.java
>  PRE-CREATION 
>   
> itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestTriggersTezSessionPoolManager.java
>  aeca663 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPool.java b67c933 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java 
> 9b4714f 
>   service/src/java/org/apache/hive/service/server/HiveServer2.java 2c4fe7f 
> 
> 
> Diff: https://reviews.apache.org/r/63381/diff/4/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Prasanth_J
> 
>

Re: Review Request 63458: HIVE-16917: HiveServer2 guard rails - Limit concurrent connections from user

2017-11-01 Thread Sergey Shelukhin 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63458/#review189860
---




common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
Lines 2475 (patched)


hmm... would it make sense to have 3 limits instead? per user, per ip, and 
per user+ip (if needed). All would be enforced. Or is it going to be extensible?



service/src/java/org/apache/hive/service/cli/session/SessionManager.java
Lines 472 (patched)


nit: map is concurrent, it should be possible to use putIfAbsent and 
replace w/o synchronize



service/src/java/org/apache/hive/service/cli/session/SessionManager.java
Lines 493 (patched)


nit: the null check is outside for increment but inside for decrement


- Sergey Shelukhin


On Nov. 1, 2017, 2:04 a.m., Prasanth_J wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63458/
> ---
> 
> (Updated Nov. 1, 2017, 2:04 a.m.)
> 
> 
> Review request for hive, Sergey Shelukhin and Thejas Nair.
> 
> 
> Bugs: HIVE-16917
> https://issues.apache.org/jira/browse/HIVE-16917
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-16917: HiveServer2 guard rails - Limit concurrent connections from user
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
> a3c853a5c5a2a3e72ae0141d0ad836ec95516280 
>   service/src/java/org/apache/hive/service/cli/session/SessionManager.java 
> 9b2ae57ee498e9b712cc87b16ba223995519611c 
>   service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java 
> 6354c8c5450eaa8ab91dd59387f2eddf0eb84389 
>   
> service/src/test/org/apache/hive/service/cli/TestCLIServiceConnectionLimits.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/63458/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Prasanth_J
> 
>

[jira] [Created] (HIVE-17962) org.apache.hadoop.hive.metastore.security.MemoryTokenStore - Parameterize Logging

2017-11-01 Thread BELUGA BEHR (JIRA) 
BELUGA BEHR created HIVE-17962:
--

 Summary: 
org.apache.hadoop.hive.metastore.security.MemoryTokenStore - Parameterize 
Logging
 Key: HIVE-17962
 URL: https://issues.apache.org/jira/browse/HIVE-17962
 Project: Hive
  Issue Type: Improvement
  Components: HiveServer2
Affects Versions: 3.0.0
Reporter: BELUGA BEHR


* Parameterize logging
* Small simplification



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Review Request 63490: HIVE-17907 enable and apply resource plan commands in HS2

2017-11-01 Thread Sergey Shelukhin 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63490/
---

Review request for hive, Harish Jaiprakash and Prasanth_J.


Repository: hive-git


Description
---

see jira


Diffs
-

  
itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
 5a627ce6bc173d390086534d73156dfc3f3674b4 
  
itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestTriggersWorkloadManager.java
 fdb660af03305225918a46373617c16e2ea8c02f 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 
7337c1ca6cea782d052b40a04a61d8ad4a6c0e25 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 
dfb0a6d6aaf9d9c6b61efc2ab76707c65074d055 
  metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 
5aa20c582b857568b75905613a0389970bb3c2f5 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
 610b9fafd5d8ec3c42e099bde5235ca507b0101b 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
 84b70d88b00557a217952f4da74e1e839ac3b7a6 
  ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 
591be49af127b5301a53bd3ae0ca18fc9af640c8 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/UserPoolMapping.java 
81d6b859a6892eba6ae3c29e8209f10a547d4690 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/WorkloadManager.java 
35e5710629324f7350bf07a9b2372d11dbfc5474 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 
70656feea75a702a18328bbb557eb7e6a27a5045 
  ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 
6a2ff75c8468e7b1dd714046bc2a35abf74c95e6 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 
306559c7229f5df57687b460feaad3240a3674bd 
  ql/src/java/org/apache/hadoop/hive/ql/plan/AlterResourcePlanDesc.java 
f0658a665e7d13b21a0fa4cb96de1c85c6797b91 
  ql/src/java/org/apache/hadoop/hive/ql/plan/DDLWork.java 
8152cfeb5967a1c020bc2018fd99d2cbc4dbbcd3 
  ql/src/test/org/apache/hadoop/hive/ql/exec/tez/TestWorkloadManager.java 
258a865fefa967e84db381493566b706f891abcf 
  service/src/java/org/apache/hive/service/server/HiveServer2.java 
2c4fe7fc2ba5dd58b62d6b2f3df44de124a381b9 
  
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java
 62801c53853dbafb7c425cff943ec819dcee4800 
  
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/RawStore.java
 a99b8e123ebb700c3754d7c7b8e5881e7781a8bd 
  
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java
 e13dcd20a30dd129ce01a45d886e819378a748a7 
  standalone-metastore/src/main/thrift/hive_metastore.thrift 
60531457b7a37424d0bdb6d39a15fc7d6d89de66 


Diff: https://reviews.apache.org/r/63490/diff/1/


Testing
---


Thanks,

Sergey Shelukhin

[jira] [Created] (HIVE-17961) NPE during initialization of VectorizedParquetRecordReader when input split is null

2017-11-01 Thread Vihang Karajgaonkar (JIRA) 
Vihang Karajgaonkar created HIVE-17961:
--

 Summary: NPE during initialization of 
VectorizedParquetRecordReader when input split is null
 Key: HIVE-17961
 URL: https://issues.apache.org/jira/browse/HIVE-17961
 Project: Hive
  Issue Type: Sub-task
  Components: Hive
Affects Versions: 3.0.0, 2.4.0
Reporter: Vihang Karajgaonkar
Assignee: Vihang Karajgaonkar
Priority: Major


HIVE-16465 introduces the regression which causes a NPE during initialize of 
the vectorized reader when input split is null. This was already fixed in 
HIVE-15718 but got exposed again we refactored for HIVE-16465. We should also 
add a test case to catch such regressions in the future.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Review Request 63486: HIVE-17953 Metrics should move to destination atomically

2017-11-01 Thread Alexander Kolbasov 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63486/
---

Review request for hive, Sahil Takiar and Vihang Karajgaonkar.


Bugs: HIVE-17953
https://issues.apache.org/jira/browse/HIVE-17953


Repository: hive-git


Description
---

HIVE-17953 Metrics should move to destination atomically


Diffs
-

  
common/src/java/org/apache/hadoop/hive/common/metrics/metrics2/JsonFileMetricsReporter.java
 b9be8bd18285df42a82385f7ce73d9e1924fee4c 
  
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/metrics/JsonReporter.java
 04a5f02a91b76ed3775f7c1c8e0122a135564c70 


Diff: https://reviews.apache.org/r/63486/diff/1/


Testing
---


Thanks,

Alexander Kolbasov

Re: Review Request 63394: HIVE-15016: Run tests with Hadoop 3.0.0-beta1

2017-11-01 Thread Aihua Xu via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63394/
---

(Updated Nov. 1, 2017, 10:20 p.m.)


Review request for hive.


Repository: hive-git


Description
---

HIVE-15016: Run tests with Hadoop 3.0.0-beta1


Diffs (updated)
-

  common/pom.xml 868e14dbc3 
  hbase-handler/pom.xml 7f57b77168 
  hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseMetaHook.java 
PRE-CREATION 
  hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseRowSerializer.java 
c6f3b0ff3a 
  hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java 
9cad97ad4b 
  
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseInputFormatUtil.java
 6054d533f0 
  
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java
 1ef454572c 
  
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableOutputFormat.java
 4b8f62c913 
  
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHFileOutputFormat.java 
a25a96faa9 
  hbase-handler/src/java/org/apache/hadoop/hive/hbase/ResultWritable.java 
b35aea9f60 
  hbase-handler/src/test/org/apache/hadoop/hive/hbase/TestHBaseSerDe.java 
f244ed6562 
  hbase-handler/src/test/org/apache/hadoop/hive/hbase/TestLazyHBaseObject.java 
b2bdd19964 
  
hbase-handler/src/test/org/apache/hadoop/hive/hbase/TestPutResultWritable.java 
561b0a8952 
  hbase-handler/src/test/queries/positive/hbase_bulk.q 475aafc1ce 
  hbase-handler/src/test/queries/positive/hbase_handler_bulk.q 85581ecdac 
  hbase-handler/src/test/results/positive/hbase_handler_bulk.q.out 1f42567a4b 
  hcatalog/core/src/main/java/org/apache/hive/hcatalog/common/HCatUtil.java 
4ace16ca6f 
  
hcatalog/core/src/test/java/org/apache/hive/hcatalog/rcfile/TestRCFileMapReduceInputFormat.java
 bb6c582d28 
  
hcatalog/webhcat/svr/src/test/java/org/apache/hive/hcatalog/templeton/TestWebHCatE2e.java
 22d2cc6bff 
  
hcatalog/webhcat/svr/src/test/java/org/apache/hive/hcatalog/templeton/mock/MockUriInfo.java
 d69a7339c1 
  itests/hcatalog-unit/pom.xml bb6b10554f 
  
itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/hbase/ManyMiniCluster.java
 745aa999f9 
  
itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/hbase/SkeletonHBaseTest.java
 4e1384a634 
  
itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/hbase/TestPigHBaseStorageHandler.java
 f8f18b3514 
  itests/hive-minikdc/pom.xml 95d2614901 
  itests/hive-unit-hadoop2/pom.xml 339a194f2d 
  itests/hive-unit/pom.xml 14409832c8 
  itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/TestAcidOnTez.java 
dbfc23510c 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/txn/compactor/TestCompactor.java
 707bcd10b7 
  
itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcWithLocalClusterSpark.java
 cabddea5c6 
  
itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestMultiSessionsHS2WithLocalClusterSpark.java
 e3f9646b59 
  itests/qtest-accumulo/pom.xml 40d0a749c8 
  itests/qtest-spark/pom.xml a506f7f081 
  itests/qtest/pom.xml 02664f3792 
  itests/util/pom.xml e6dc09fb8f 
  itests/util/src/main/java/org/apache/hadoop/hive/hbase/HBaseQTestUtil.java 
0cc9a89085 
  itests/util/src/main/java/org/apache/hadoop/hive/hbase/HBaseTestSetup.java 
4f8fa05925 
  llap-server/pom.xml 47a04cc310 
  
llap-server/src/java/org/apache/hadoop/hive/llap/shufflehandler/ShuffleHandler.java
 6b08da66b6 
  metastore/pom.xml 04c6f47879 
  pom.xml 006e8f8611 
  ql/src/test/org/apache/hadoop/hive/ql/io/TestRCFile.java 19b97e4cf2 
  ql/src/test/results/clientnegative/exim_00_unsupported_schema.q.out 
b582471afd 
  ql/src/test/results/clientnegative/external1.q.out 661d669206 
  ql/src/test/results/clientnegative/external2.q.out eb5518c017 
  serde/pom.xml 7419cfb17a 
  shims/0.23/pom.xml 3ff1d38776 
  shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java 
e9445eb11d 
  shims/common/src/main/java/org/apache/hadoop/fs/ProxyFileSystem.java 
2c37a51cf4 
  shims/scheduler/pom.xml 0eadb69435 
  spark-client/pom.xml 784d908e4a 
  standalone-metastore/pom.xml 8df622f179 
  
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/utils/HdfsUtils.java
 c10e36f94a 
  testutils/ptest2/src/main/resources/batch-exec.vm 2d16ca3e8a 


Diff: https://reviews.apache.org/r/63394/diff/3/

Changes: https://reviews.apache.org/r/63394/diff/2-3/


Testing
---


Thanks,

Aihua Xu

[jira] [Created] (HIVE-17960) LLAP UDF download should not fail on errors

2017-11-01 Thread Sergey Shelukhin (JIRA) 
Sergey Shelukhin created HIVE-17960:
---

 Summary: LLAP UDF download should not fail on errors
 Key: HIVE-17960
 URL: https://issues.apache.org/jira/browse/HIVE-17960
 Project: Hive
  Issue Type: Bug
Reporter: Sergey Shelukhin
Priority: Major


many people have a zoo of random outdated UDF jars that were deleted or have 
wrong permissions. When LLAP setup checks for permanent UDFs, it should log the 
errors related to those but not fail the entire startup.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HIVE-17959) SparkPartitionPruningSinkOperator buffers all writes in memory

2017-11-01 Thread Sahil Takiar (JIRA) 
Sahil Takiar created HIVE-17959:
---

 Summary: SparkPartitionPruningSinkOperator buffers all writes in 
memory
 Key: HIVE-17959
 URL: https://issues.apache.org/jira/browse/HIVE-17959
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Sahil Takiar
Priority: Major


{{SparkPartitionPruningSinkOperator}} buffers everything in memory and only 
flushes to the output file with the operator is closed. This can cause memory 
issues.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HIVE-17958) spark_dynamic_partition_pruning.q fails when hive.tez.dynamic.semijoin.reduction is false

2017-11-01 Thread Sahil Takiar (JIRA) 
Sahil Takiar created HIVE-17958:
---

 Summary: spark_dynamic_partition_pruning.q fails when 
hive.tez.dynamic.semijoin.reduction is false
 Key: HIVE-17958
 URL: https://issues.apache.org/jira/browse/HIVE-17958
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Sahil Takiar
Assignee: Sahil Takiar


Looks like {{RedundantDynamicPruningConditionsRemoval}} causes DPP to be 
disabled in a few cases (not sure why). When 
{{hive.tez.dynamic.semijoin.reduction}} is {{true}} (the default), then this 
rule is disabled so the normal tests don't hit this issue.

But when I disable {{hive.tez.dynamic.semijoin.reduction}} then the following 
query no longer fully triggers DPP:

{code}
EXPLAIN select count(*) from srcpart join srcpart_date on (srcpart.ds = 
srcpart_date.ds) join srcpart_hour on (srcpart.hr = srcpart_hour.hr)
5777 where srcpart_date.`date` = '2008-04-08' and srcpart_hour.hour = 11 and 
srcpart.hr = 11
{code}

There should be two DPP sinks, but when the config is set to false, there is 
only one.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 63470: HIVE-17767 Rewrite correlated EXISTS/IN subqueries into LEFT SEMI JOIN

2017-11-01 Thread Ashutosh Chauhan 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63470/#review189825
---




ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRelDecorrelator.java
Lines 345 (patched)


We shall initialize valuegen to true during visit. That will be safer.



ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRelDecorrelator.java
Lines 1177 (patched)


Shall leave a TODO to remove this restriction.



ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java
Lines 1790-1797 (original), 1790 (patched)


Any reason to drop this rule.



ql/src/test/results/clientpositive/constprog_partitioner.q.out
Lines 84 (patched)


Extra Gby is not needed, correct? Is this where we need rule to eliminate 
Gby beneath Left semi join?



ql/src/test/results/clientpositive/semijoin5.q.out
Line 122 (original), 122 (patched)


seems like we are not able to propagate constants over semi-join. Shall 
track this in seperate jira.



ql/src/test/results/clientpositive/subquery_in_having.q.out
Line 1819 (original), 1826 (patched)


Lets move this test to minilllaplocal only.


- Ashutosh Chauhan


On Nov. 1, 2017, 6:23 p.m., Vineet Garg wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63470/
> ---
> 
> (Updated Nov. 1, 2017, 6:23 p.m.)
> 
> 
> Review request for hive and Ashutosh Chauhan.
> 
> 
> Bugs: HIVE-17767
> https://issues.apache.org/jira/browse/HIVE-17767
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> This patch does the following:
> * Added back earlier patch to generate value generator
> * Added logic to rewrite EXISTS/IN correlated subqueries into LEFT SEMI JOIN
> * Remove SemiJoinTransposeRule (This rule pushes semi join underneath its 
> left join which might not be semantically correct thing to do)
> 
> 
> Diffs
> -
> 
>   itests/src/test/resources/testconfiguration.properties 462f332e99 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveSubQRemoveRelBuilder.java
>  3a1897f4aa 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRelDecorrelator.java
>  62125f0fb7 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSubQueryRemoveRule.java
>  2dca6a25ac 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 80351bef87 
>   ql/src/test/queries/clientpositive/subquery_exists.q 19c42f0c29 
>   ql/src/test/queries/clientpositive/subquery_in.q 4ba170a706 
>   ql/src/test/results/clientpositive/constprog_partitioner.q.out 87618df902 
>   ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction_2.q.out 
> 87e08fbcde 
>   ql/src/test/results/clientpositive/llap/explainuser_1.q.out 6e55acf0d8 
>   ql/src/test/results/clientpositive/llap/lineage3.q.out 66cc6ad5a0 
>   ql/src/test/results/clientpositive/llap/subquery_exists.q.out e206f0851e 
>   ql/src/test/results/clientpositive/llap/subquery_in.q.out af42131bc2 
>   ql/src/test/results/clientpositive/llap/subquery_multi.q.out 96fe17a05a 
>   ql/src/test/results/clientpositive/llap/subquery_notin.q.out 8e2ca937af 
>   ql/src/test/results/clientpositive/llap/subquery_scalar.q.out c89d053b4a 
>   ql/src/test/results/clientpositive/llap/subquery_select.q.out 118f6ebccf 
>   ql/src/test/results/clientpositive/llap/subquery_views.q.out a9a81133b5 
>   ql/src/test/results/clientpositive/llap/vector_mapjoin_reduce.q.out 
> 4e6f00f6b7 
>   ql/src/test/results/clientpositive/masking_12.q.out 540c53e825 
>   ql/src/test/results/clientpositive/masking_3.q.out 1114c80676 
>   ql/src/test/results/clientpositive/masking_4.q.out 527da21610 
>   ql/src/test/results/clientpositive/perf/spark/query10.q.out eb3a2f6699 
>   ql/src/test/results/clientpositive/perf/spark/query16.q.out b74d721d41 
>   ql/src/test/results/clientpositive/perf/spark/query35.q.out 8759b71b8c 
>   ql/src/test/results/clientpositive/perf/spark/query69.q.out e4430beaac 
>   ql/src/test/results/clientpositive/perf/spark/query94.q.out 43b8c77bdc 
>   ql/src/test/results/clientpositive/perf/tez/query10.q.out cf3651b35b 
>   ql/src/test/results/clientpositive/perf/tez/query14.q.out b2a45f155a 
>   ql/src/test/results/clientpositive/perf/tez/query16.q.out a7b710d6e1 
>   ql/src/test/results/clientpositive/perf/tez/query23.q.out 7112de61d9 
>   ql/src/test/results/clientpositive/perf/tez/query35.q.out a72f57816e 
>

Re: [Announce] New committer: Tao Li

2017-11-01 Thread Peter Vary 
Congratulations Tao! :)

> On Nov 1, 2017, at 6:49 PM, Ashutosh Chauhan  wrote:
> 
> The Project Management Committee (PMC) for Apache Hive has invited Tao Li to
> become a committer and we are pleased to announce that he has accepted.
> 
> Welcome, Tao!
> 
> Thanks,
> Ashutosh

Re: ptest not picking patches

2017-11-01 Thread Vihang Karajgaonkar 
Is there a way to find out if Jenkins was restarted since we submitted the
patch? Not sure if we can fix this in our precommit job. There have been
far too many instances recently when Jenkins was either restarted or was
non-responsive for long durations which delayed testing for our patches.

On Wed, Nov 1, 2017 at 11:22 AM, Ashutosh Chauhan 
wrote:

> There was an apache infra issue which was making jenkins to not submit
> patches to ptest server. https://issues.apache.org/jira/browse/INFRA-15358
> Also jenkins was restarted overnight.
>
> On Wed, Nov 1, 2017 at 11:15 AM, Chris Drome 
> wrote:
>
> > Me as well for HIVE-17853.
> >
> > Attempting to cancel/resubmit patches.
> >
> > On Wed, Nov 1, 2017 at 10:50 AM, Prasanth Jayachandran <
> > pjayachand...@hortonworks.com> wrote:
> >
> > > Me too. Seen this yesterday with HIVE-17834.
> > >
> > > Thanks
> > > Prasanth
> > >
> > >
> > >
> > > On Wed, Nov 1, 2017 at 10:26 AM -0700, "Deepak Jaiswal" <
> > > djais...@hortonworks.com> wrote:
> > >
> > >
> > > Hi,
> > >
> > > I uploaded couple of patches but they don't appear in the Pre-commit
> > tests
> > > queue.
> > > Is anyone else facing this?
> > >
> > > Regards,
> > > Deepak
> > >
> > >
> >
>

Review Request 63470: HIVE-17767 Rewrite correlated EXISTS/IN subqueries into LEFT SEMI JOIN

2017-11-01 Thread Vineet Garg 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63470/
---

Review request for hive and Ashutosh Chauhan.


Bugs: HIVE-17767
https://issues.apache.org/jira/browse/HIVE-17767


Repository: hive-git


Description
---

This patch does the following:
* Added back earlier patch to generate value generator
* Added logic to rewrite EXISTS/IN correlated subqueries into LEFT SEMI JOIN
* Remove SemiJoinTransposeRule (This rule pushes semi join underneath its left 
join which might not be semantically correct thing to do)


Diffs
-

  itests/src/test/resources/testconfiguration.properties 462f332e99 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveSubQRemoveRelBuilder.java
 3a1897f4aa 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRelDecorrelator.java
 62125f0fb7 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSubQueryRemoveRule.java
 2dca6a25ac 
  ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 80351bef87 
  ql/src/test/queries/clientpositive/subquery_exists.q 19c42f0c29 
  ql/src/test/queries/clientpositive/subquery_in.q 4ba170a706 
  ql/src/test/results/clientpositive/constprog_partitioner.q.out 87618df902 
  ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction_2.q.out 
87e08fbcde 
  ql/src/test/results/clientpositive/llap/explainuser_1.q.out 6e55acf0d8 
  ql/src/test/results/clientpositive/llap/lineage3.q.out 66cc6ad5a0 
  ql/src/test/results/clientpositive/llap/subquery_exists.q.out e206f0851e 
  ql/src/test/results/clientpositive/llap/subquery_in.q.out af42131bc2 
  ql/src/test/results/clientpositive/llap/subquery_multi.q.out 96fe17a05a 
  ql/src/test/results/clientpositive/llap/subquery_notin.q.out 8e2ca937af 
  ql/src/test/results/clientpositive/llap/subquery_scalar.q.out c89d053b4a 
  ql/src/test/results/clientpositive/llap/subquery_select.q.out 118f6ebccf 
  ql/src/test/results/clientpositive/llap/subquery_views.q.out a9a81133b5 
  ql/src/test/results/clientpositive/llap/vector_mapjoin_reduce.q.out 
4e6f00f6b7 
  ql/src/test/results/clientpositive/masking_12.q.out 540c53e825 
  ql/src/test/results/clientpositive/masking_3.q.out 1114c80676 
  ql/src/test/results/clientpositive/masking_4.q.out 527da21610 
  ql/src/test/results/clientpositive/perf/spark/query10.q.out eb3a2f6699 
  ql/src/test/results/clientpositive/perf/spark/query16.q.out b74d721d41 
  ql/src/test/results/clientpositive/perf/spark/query35.q.out 8759b71b8c 
  ql/src/test/results/clientpositive/perf/spark/query69.q.out e4430beaac 
  ql/src/test/results/clientpositive/perf/spark/query94.q.out 43b8c77bdc 
  ql/src/test/results/clientpositive/perf/tez/query10.q.out cf3651b35b 
  ql/src/test/results/clientpositive/perf/tez/query14.q.out b2a45f155a 
  ql/src/test/results/clientpositive/perf/tez/query16.q.out a7b710d6e1 
  ql/src/test/results/clientpositive/perf/tez/query23.q.out 7112de61d9 
  ql/src/test/results/clientpositive/perf/tez/query35.q.out a72f57816e 
  ql/src/test/results/clientpositive/perf/tez/query69.q.out 591f3fcdb0 
  ql/src/test/results/clientpositive/perf/tez/query94.q.out 7674aa7f7c 
  ql/src/test/results/clientpositive/semijoin5.q.out 533c077f58 
  ql/src/test/results/clientpositive/spark/constprog_partitioner.q.out 
b89f9f5905 
  ql/src/test/results/clientpositive/spark/spark_explainuser_1.q.out 76c74d9ab7 
  ql/src/test/results/clientpositive/spark/subquery_exists.q.out dafe5b6d5b 
  ql/src/test/results/clientpositive/spark/subquery_in.q.out 471c2ccd94 
  ql/src/test/results/clientpositive/spark/subquery_multi.q.out ff519fda09 
  ql/src/test/results/clientpositive/spark/subquery_notin.q.out 1b2c0880ae 
  ql/src/test/results/clientpositive/spark/subquery_scalar.q.out de005ada82 
  ql/src/test/results/clientpositive/spark/subquery_select.q.out 7d3a16b6ee 
  ql/src/test/results/clientpositive/spark/subquery_views.q.out 91e39913a7 
  ql/src/test/results/clientpositive/spark/vector_mapjoin_reduce.q.out 
81af937e97 
  ql/src/test/results/clientpositive/subquery_exists.q.out c9f2a79041 
  ql/src/test/results/clientpositive/subquery_exists_having.q.out 2c41ff6c33 
  ql/src/test/results/clientpositive/subquery_in_having.q.out 6893442b61 
  ql/src/test/results/clientpositive/subquery_notexists.q.out 329573e8e1 
  ql/src/test/results/clientpositive/subquery_notexists_having.q.out 4d2b2fc873 
  ql/src/test/results/clientpositive/subquery_notin_having.q.out c321fe69ed 
  ql/src/test/results/clientpositive/subquery_unqualcolumnrefs.q.out 5c306f6b47 
  ql/src/test/results/clientpositive/vector_mapjoin_reduce.q.out ddea584990 


Diff: https://reviews.apache.org/r/63470/diff/1/


Testing
---


Thanks,

Vineet Garg

Re: ptest not picking patches

2017-11-01 Thread Ashutosh Chauhan 
There was an apache infra issue which was making jenkins to not submit
patches to ptest server. https://issues.apache.org/jira/browse/INFRA-15358
Also jenkins was restarted overnight.

On Wed, Nov 1, 2017 at 11:15 AM, Chris Drome 
wrote:

> Me as well for HIVE-17853.
>
> Attempting to cancel/resubmit patches.
>
> On Wed, Nov 1, 2017 at 10:50 AM, Prasanth Jayachandran <
> pjayachand...@hortonworks.com> wrote:
>
> > Me too. Seen this yesterday with HIVE-17834.
> >
> > Thanks
> > Prasanth
> >
> >
> >
> > On Wed, Nov 1, 2017 at 10:26 AM -0700, "Deepak Jaiswal" <
> > djais...@hortonworks.com> wrote:
> >
> >
> > Hi,
> >
> > I uploaded couple of patches but they don't appear in the Pre-commit
> tests
> > queue.
> > Is anyone else facing this?
> >
> > Regards,
> > Deepak
> >
> >
>

Re: ptest not picking patches

2017-11-01 Thread Eugene Koifman 
Jenkings was restarted overnight.  That usually loses all queued up patches.  
Was OK this morning.

On 11/1/17, 11:15 AM, "Chris Drome"  wrote:

Me as well for HIVE-17853.

Attempting to cancel/resubmit patches.

On Wed, Nov 1, 2017 at 10:50 AM, Prasanth Jayachandran <
pjayachand...@hortonworks.com> wrote:

> Me too. Seen this yesterday with HIVE-17834.
>
> Thanks
> Prasanth
>
>
>
> On Wed, Nov 1, 2017 at 10:26 AM -0700, "Deepak Jaiswal" <
> djais...@hortonworks.com> wrote:
>
>
> Hi,
>
> I uploaded couple of patches but they don't appear in the Pre-commit tests
> queue.
> Is anyone else facing this?
>
> Regards,
> Deepak
>
>

Re: ptest not picking patches

2017-11-01 Thread Chris Drome 
Me as well for HIVE-17853.

Attempting to cancel/resubmit patches.

On Wed, Nov 1, 2017 at 10:50 AM, Prasanth Jayachandran <
pjayachand...@hortonworks.com> wrote:

> Me too. Seen this yesterday with HIVE-17834.
>
> Thanks
> Prasanth
>
>
>
> On Wed, Nov 1, 2017 at 10:26 AM -0700, "Deepak Jaiswal" <
> djais...@hortonworks.com> wrote:
>
>
> Hi,
>
> I uploaded couple of patches but they don't appear in the Pre-commit tests
> queue.
> Is anyone else facing this?
>
> Regards,
> Deepak
>
>

Re: ptest not picking patches

2017-11-01 Thread Prasanth Jayachandran 
Me too. Seen this yesterday with HIVE-17834.

Thanks
Prasanth



On Wed, Nov 1, 2017 at 10:26 AM -0700, "Deepak Jaiswal" 
> wrote:


Hi,

I uploaded couple of patches but they don't appear in the Pre-commit tests 
queue.
Is anyone else facing this?

Regards,
Deepak

[Announce] New committer: Tao Li

2017-11-01 Thread Ashutosh Chauhan 
The Project Management Committee (PMC) for Apache Hive has invited Tao Li to
become a committer and we are pleased to announce that he has accepted.

Welcome, Tao!

Thanks,
 Ashutosh

ptest not picking patches

2017-11-01 Thread Deepak Jaiswal 
Hi,

I uploaded couple of patches but they don’t appear in the Pre-commit tests 
queue.
Is anyone else facing this?

Regards,
Deepak

Re: Hive metastore TLP

2017-11-01 Thread Vihang Karajgaonkar 
Its great to hear that you are considering integrating Metastore to store
metadata for Drill. Couple of things to note regarding the timeline though.
As pointed by Alan the work being done in HIVE-17159 is on the master
branch which currently points to Hive 3.0 The latest release of Hive is 2.3
which is forked from branch-2 where this work is not available at the
moment. So unless we have a Hive 3.0 release in December you may not be
able to consume the standalone metastore right away. That said if you
decide to develop/design against existing Metastore public API, it should
not be too hard to rebase to latest Hive libraries when it is released.
Also, it may be important to note that there will be a few metastore APIs
which may still need Hive jars in the runtime, but based on our
understanding these APIs are exclusively needed by Hive and other
components like Drill may not need them.

Hope that helps.

Thanks,
Vihang

On Wed, Nov 1, 2017 at 7:10 AM, Alan Gates  wrote:

> Excellent, I am glad to hear that this could be useful for you.
>
> After discussing the proposal in the community [1] we agreed that the first
> step should be to break out the metastore as a separately releasable module
> inside Hive.  You can see the progress of that work at HIVE-17159[2].  With
> this change Drill we be able to use the metastore without needing all of
> Hive.  Once that is complete we will take up the discussion of whether and
> how to make it a separate TLP.  Open source communities being what they
> are, I cannot give you precise dates on when the separately releasable
> module will be available.  At the current rate of progress it should be
> done by the end of this year.
>
> Regarding compatibility, we are working hard to keep the thrift interface
> completely backwards compatible.  So if you develop a client against the
> current Hive metastore it should just work against the standalone
> metastore.  There will be small changes to metastore listeners (plugins
> that can be run inside the metastore) that may require changes to user code
> or recompilation of the listeners.  But even there we are trying to keep
> the changes to a minimum.
>
> Alan.
>
> 1. If you go to https://lists.apache.org/list.html?dev@hive.apache.org and
> do an advanced search over the last 6 months for messages with a subject of
> "[DISCUSS] Separating out the metastore as its own TLP” you will see all
> the mails.  Sorry, I can’t figure out how to get ponymail to give me a
> single link to the whole thread.
> 2. https://issues.apache.org/jira/browse/HIVE-17159
>
> On Tue, Oct 31, 2017 at 4:14 PM, Padma Penumarthy 
> wrote:
>
> > Hi,
> >
> > We are exploring the possibility of using hive metastore as metastore for
> > Apache Drill, to store and
> > manage our metadata. While investigating, I came across the metastore TLP
> > proposal,
> > https://cwiki.apache.org/confluence/display/Hive/Metastore+TLP+Proposal.
> >
> > I want to find out what is the timeline for this to be available to end
> > users ?
> > when will it be production ready ?
> > If we integrate with hive and design/develop using current metastore
> > client API, do we run
> > into risk of incompatibility in the future ? Our goal is to eventually
> > move to using hive metastore only for the
> > reasons correctly identified in your wiki i.e. users do not have to
> > install hive to be able to use Drill.
> >
> > Any input/feedback you can provide to us will be very helpful for us to
> > figure out our plans.
> >
> > Thanks
> > Padma
> >
> >
> >
>

[jira] [Created] (HIVE-17957) Hive 2.4.0 Release Planning

2017-11-01 Thread Sahil Takiar (JIRA) 
Sahil Takiar created HIVE-17957:
---

 Summary: Hive 2.4.0 Release Planning
 Key: HIVE-17957
 URL: https://issues.apache.org/jira/browse/HIVE-17957
 Project: Hive
  Issue Type: Task
Affects Versions: 2.4.0
Reporter: Sahil Takiar
Priority: Major
 Fix For: 2.4.0






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HIVE-17956) Retrieve "latest" partition from Hive Metastore

2017-11-01 Thread Micah Whitacre (JIRA) 
Micah Whitacre created HIVE-17956:
-

 Summary: Retrieve "latest" partition from Hive Metastore
 Key: HIVE-17956
 URL: https://issues.apache.org/jira/browse/HIVE-17956
 Project: Hive
  Issue Type: New Feature
  Components: Metastore
Reporter: Micah Whitacre


We are trying to utilize the Hive Metastore for our processing needs, 
specifically focusing on consuming through the HCatalog APIs.  One use case we 
have is that we want to consume the "latest" partition.  In researching there 
are a number of posts[1][2] that talk about using queries through Hive Server2 
to find that information.  It would be more ideal if this was a first class API 
offered from the Hive Metastore without requiring a query to be executed.

The other option would be to retrieve all of the partitions and sort client 
side.  There is a concern about the efficiency and memory requirements of this 
especially without the "iterator" concept implemented from HIVE-7195.

[1] - 
https://community.hortonworks.com/questions/85330/how-to-optimize-hive-access-to-the-latest-partitio.html
[2] - 
https://stackoverflow.com/questions/36095790/how-to-find-the-most-recent-partition-in-hive-table



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 62816: HIVE-17729 Add Database & Explain related blobstore tests

2017-11-01 Thread Sergio Pena via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62816/#review189798
---


Ship it!




Ship It!

- Sergio Pena


On Oct. 6, 2017, 7:13 p.m., Rentao Wu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62816/
> ---
> 
> (Updated Oct. 6, 2017, 7:13 p.m.)
> 
> 
> Review request for hive and Sergio Pena.
> 
> 
> Bugs: HIVE-17729
> https://issues.apache.org/jira/browse/HIVE-17729
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-17729 Add Database & Explain related blobstore tests
> 
> 
> This patch introduces the following regression tests into the hive-blobstore 
> qtest module:
> create_database.q -> tests tables with location inherited from database
> multiple_db.q -> tests query spanning multiple databases
> explain.q -> tests EXPLAIN INSERT OVERWRITE command
> 
> 
> Diffs
> -
> 
>   data/files/single_int.txt PRE-CREATION 
>   itests/hive-blobstore/src/test/queries/clientpositive/create_database.q 
> PRE-CREATION 
>   itests/hive-blobstore/src/test/queries/clientpositive/explain.q 
> PRE-CREATION 
>   itests/hive-blobstore/src/test/queries/clientpositive/multiple_db.q 
> PRE-CREATION 
>   itests/hive-blobstore/src/test/results/clientpositive/create_database.q.out 
> PRE-CREATION 
>   itests/hive-blobstore/src/test/results/clientpositive/explain.q.out 
> PRE-CREATION 
>   itests/hive-blobstore/src/test/results/clientpositive/multiple_db.q.out 
> PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/62816/diff/1/
> 
> 
> Testing
> ---
> 
> under /itests/hive-blobstore/
> 
> $ mvn clean test -Dtest=TestBlobstoreCliDriver 
> -Dqfile=create_database.q,explain.q,multiple_db.q
> 
> ---
>  T E S T S
> ---
> Running org.apache.hadoop.hive.cli.TestBlobstoreCliDriver
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 48.4 sec - in 
> org.apache.hadoop.hive.cli.TestBlobstoreCliDriver
> 
> Results :
> 
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 01:12 min
> [INFO] Finished at: 2017-10-06T11:29:41-07:00
> [INFO] Final Memory: 88M/1003M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Rentao Wu
> 
>

Re: Hive metastore TLP

2017-11-01 Thread Alan Gates 
Excellent, I am glad to hear that this could be useful for you.

After discussing the proposal in the community [1] we agreed that the first
step should be to break out the metastore as a separately releasable module
inside Hive.  You can see the progress of that work at HIVE-17159[2].  With
this change Drill we be able to use the metastore without needing all of
Hive.  Once that is complete we will take up the discussion of whether and
how to make it a separate TLP.  Open source communities being what they
are, I cannot give you precise dates on when the separately releasable
module will be available.  At the current rate of progress it should be
done by the end of this year.

Regarding compatibility, we are working hard to keep the thrift interface
completely backwards compatible.  So if you develop a client against the
current Hive metastore it should just work against the standalone
metastore.  There will be small changes to metastore listeners (plugins
that can be run inside the metastore) that may require changes to user code
or recompilation of the listeners.  But even there we are trying to keep
the changes to a minimum.

Alan.

1. If you go to https://lists.apache.org/list.html?dev@hive.apache.org and
do an advanced search over the last 6 months for messages with a subject of
"[DISCUSS] Separating out the metastore as its own TLP” you will see all
the mails.  Sorry, I can’t figure out how to get ponymail to give me a
single link to the whole thread.
2. https://issues.apache.org/jira/browse/HIVE-17159

On Tue, Oct 31, 2017 at 4:14 PM, Padma Penumarthy 
wrote:

> Hi,
>
> We are exploring the possibility of using hive metastore as metastore for
> Apache Drill, to store and
> manage our metadata. While investigating, I came across the metastore TLP
> proposal,
> https://cwiki.apache.org/confluence/display/Hive/Metastore+TLP+Proposal.
>
> I want to find out what is the timeline for this to be available to end
> users ?
> when will it be production ready ?
> If we integrate with hive and design/develop using current metastore
> client API, do we run
> into risk of incompatibility in the future ? Our goal is to eventually
> move to using hive metastore only for the
> reasons correctly identified in your wiki i.e. users do not have to
> install hive to be able to use Drill.
>
> Any input/feedback you can provide to us will be very helpful for us to
> figure out our plans.
>
> Thanks
> Padma
>
>
>

Re: Review Request 63427: HIVE-17396

2017-11-01 Thread Deepak Jaiswal 


> On Nov. 1, 2017, 3:48 a.m., Lefty Leverenz wrote:
> > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
> > Lines 3017-3018 (patched)
> > 
> >
> > Please spell out the acronyms TS, DPP, and nDVs in the parameter 
> > description.
> > 
> > Also, the description isn't clear to me -- "to control if ..." what?  
> > Something about reduction, or feeding?  (Forgive my ignorance, I'm just 
> > looking at the grammatical structure of the sentence along with the 
> > parameter name.)

Thanks for the feedback. How about this below?

The factor to decide if semijoin branch feeding into a TableScan which has an 
outgoing Dynamic Partition pruning(DPP) branch based on number of distinct 
values.


- Deepak


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63427/#review189778
---


On Oct. 31, 2017, 1:19 a.m., Deepak Jaiswal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63427/
> ---
> 
> (Updated Oct. 31, 2017, 1:19 a.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan and Jason Dere.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Dynamic Semijoin Reduction : markSemiJoinForDPP marks unwanted semijoin 
> branches
> 
> In method markSemiJoinForDPP (HIVE-17399), the nDVs comparison should not 
> have equality as there is a chance that the values are same on both sides and 
> the branch is still marked as good when it shouldn't be.
> Add a configurable factor to see how useful this is if nDVs on smaller side 
> are only slightly less than that on TS side.
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 6631a6e45d 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java da30c3b642 
>   ql/src/test/queries/clientpositive/dynamic_semijoin_reduction.q 6cc0a7f7a9 
>   ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction.q.out 
> 1a1a4d9b2d 
> 
> 
> Diff: https://reviews.apache.org/r/63427/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Deepak Jaiswal
> 
>