Re: Review Request 60349: Scalar sub-query: sq_count_check not required if gby keys are constant

2017-07-12 Thread Vineet Garg 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60349/
---

(Updated July 12, 2017, 9:57 p.m.)


Review request for hive and Ashutosh Chauhan.


Changes
---

Addressed review comments


Bugs: HIVE-16793
https://issues.apache.org/jira/browse/HIVE-16793


Repository: hive-git


Description
---

For scalar subqueries we skip adding UDF sq_count_check which have aggregates 
but we add it if there is a explicit group by present. We can skip it if group 
by is present but group by keys are constant. This patch adds a rule to do this 
optimization.


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 5700fb9325 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRemoveSqCountCheck.java
 PRE-CREATION 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSubQueryRemoveRule.java
 83d3f7436d 
  ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 88054e7379 
  ql/src/test/queries/clientpositive/subquery_scalar.q 876a1e98f8 
  ql/src/test/results/clientpositive/llap/subquery_scalar.q.out 3a0d1464c5 
  ql/src/test/results/clientpositive/perf/query14.q.out 42bad8da14 
  ql/src/test/results/clientpositive/perf/query23.q.out ebd2271108 


Diff: https://reviews.apache.org/r/60349/diff/3/

Changes: https://reviews.apache.org/r/60349/diff/2-3/


Testing
---

-Pre-commit testing
- new tests


Thanks,

Vineet Garg

Re: Review Request 60349: Scalar sub-query: sq_count_check not required if gby keys are constant

2017-07-07 Thread Ashutosh Chauhan 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60349/#review179973
---




ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRemoveSqCountCheck.java
Lines 42 (patched)


Add some comment on what rule does.



ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRemoveSqCountCheck.java
Lines 74 (patched)


Comment seems misplaced.



ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSubQueryRemoveRule.java
Lines 178 (patched)


Lets add a flag (with default off) hive.optimize.remove.sq_count_check 
which doesnt add this project, that way we avoid problem of both join order as 
well as join merging.



ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSubQueryRemoveRule.java
Lines 202 (patched)


can be deleted.



ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java
Lines 1533-1536 (patched)


Rule will fire only when flag is on.


- Ashutosh Chauhan


On June 26, 2017, 9:06 p.m., Vineet Garg wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60349/
> ---
> 
> (Updated June 26, 2017, 9:06 p.m.)
> 
> 
> Review request for hive and Ashutosh Chauhan.
> 
> 
> Bugs: HIVE-16793
> https://issues.apache.org/jira/browse/HIVE-16793
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> For scalar subqueries we skip adding UDF sq_count_check which have aggregates 
> but we add it if there is a explicit group by present. We can skip it if 
> group by is present but group by keys are constant. This patch adds a rule to 
> do this optimization.
> 
> 
> Diffs
> -
> 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRemoveSqCountCheck.java
>  PRE-CREATION 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSubQueryRemoveRule.java
>  83d3f7436d 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 88054e7379 
>   ql/src/test/results/clientnegative/subquery_scalar_multi_rows.q.out 
> 0a780db7ef 
>   ql/src/test/results/clientpositive/llap/subquery_scalar.q.out ceccc88b2e 
>   ql/src/test/results/clientpositive/llap/subquery_select.q.out 440e9d7b55 
>   ql/src/test/results/clientpositive/perf/query23.q.out ebd2271108 
>   ql/src/test/results/clientpositive/perf/query44.q.out 07187b14d4 
>   ql/src/test/results/clientpositive/perf/query54.q.out b9d0b8b376 
>   ql/src/test/results/clientpositive/perf/query58.q.out 5b37f36386 
>   ql/src/test/results/clientpositive/perf/query6.q.out 03fff79951 
> 
> 
> Diff: https://reviews.apache.org/r/60349/diff/2/
> 
> 
> Testing
> ---
> 
> -Pre-commit testing
> - new tests
> 
> 
> Thanks,
> 
> Vineet Garg
> 
>

Re: Review Request 60349: Scalar sub-query: sq_count_check not required if gby keys are constant

2017-06-26 Thread Vineet Garg 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60349/
---

(Updated June 26, 2017, 9:06 p.m.)


Review request for hive and Ashutosh Chauhan.


Changes
---

Updated golden files


Bugs: HIVE-16793
https://issues.apache.org/jira/browse/HIVE-16793


Repository: hive-git


Description
---

For scalar subqueries we skip adding UDF sq_count_check which have aggregates 
but we add it if there is a explicit group by present. We can skip it if group 
by is present but group by keys are constant. This patch adds a rule to do this 
optimization.


Diffs (updated)
-

  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRemoveSqCountCheck.java
 PRE-CREATION 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSubQueryRemoveRule.java
 83d3f7436d 
  ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 88054e7379 
  ql/src/test/results/clientnegative/subquery_scalar_multi_rows.q.out 
0a780db7ef 
  ql/src/test/results/clientpositive/llap/subquery_scalar.q.out ceccc88b2e 
  ql/src/test/results/clientpositive/llap/subquery_select.q.out 440e9d7b55 
  ql/src/test/results/clientpositive/perf/query23.q.out ebd2271108 
  ql/src/test/results/clientpositive/perf/query44.q.out 07187b14d4 
  ql/src/test/results/clientpositive/perf/query54.q.out b9d0b8b376 
  ql/src/test/results/clientpositive/perf/query58.q.out 5b37f36386 
  ql/src/test/results/clientpositive/perf/query6.q.out 03fff79951 


Diff: https://reviews.apache.org/r/60349/diff/2/

Changes: https://reviews.apache.org/r/60349/diff/1-2/


Testing
---

-Pre-commit testing
- new tests


Thanks,

Vineet Garg

Review Request 60349: Scalar sub-query: sq_count_check not required if gby keys are constant

2017-06-21 Thread Vineet Garg 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60349/
---

Review request for hive and Ashutosh Chauhan.


Bugs: HIVE-16793
https://issues.apache.org/jira/browse/HIVE-16793


Repository: hive-git


Description
---

For scalar subqueries we skip adding UDF sq_count_check which have aggregates 
but we add it if there is a explicit group by present. We can skip it if group 
by is present but group by keys are constant. This patch adds a rule to do this 
optimization.


Diffs
-

  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRemoveSqCountCheck.java
 PRE-CREATION 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSubQueryRemoveRule.java
 83d3f7436d 
  ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 348331e052 
  ql/src/test/results/clientnegative/subquery_scalar_multi_rows.q.out 
0a780db7ef 
  ql/src/test/results/clientpositive/llap/subquery_scalar.q.out e94edff262 
  ql/src/test/results/clientpositive/llap/subquery_select.q.out 202980e975 
  ql/src/test/results/clientpositive/perf/query23.q.out 1fd8cb4f25 
  ql/src/test/results/clientpositive/perf/query44.q.out 566548089c 
  ql/src/test/results/clientpositive/perf/query54.q.out 3cbcbe33f9 
  ql/src/test/results/clientpositive/perf/query58.q.out acdfc07718 
  ql/src/test/results/clientpositive/perf/query6.q.out 03fff79951 


Diff: https://reviews.apache.org/r/60349/diff/1/


Testing
---

-Pre-commit testing
- new tests


Thanks,

Vineet Garg