[jira] [Created] (HIVE-17432) Enable join and aggregate materialized view rewriting

2017-09-01 Thread Jesus Camacho Rodriguez (JIRA) 
Jesus Camacho Rodriguez created HIVE-17432:
--

 Summary: Enable join and aggregate materialized view rewriting
 Key: HIVE-17432
 URL: https://issues.apache.org/jira/browse/HIVE-17432
 Project: Hive
  Issue Type: New Feature
  Components: Hive
Affects Versions: 3.0.0
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez


Enable Calcite materialized view based rewriting for queries containing joins 
and aggregates.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Review Request 62048: HIVE-17431 change configuration handling in TezSessionState

2017-09-01 Thread Sergey Shelukhin 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62048/
---

Review request for hive and Siddharth Seth.


Repository: hive-git


Description
---

see jira


Diffs
-

  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPool.java 4f58565a4c 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java 
1f4705c083 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolSession.java 
005eeedc02 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionState.java 
fe5c6a1e45 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezTask.java f1f10286a3 
  ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java d7592bb966 
  ql/src/test/org/apache/hadoop/hive/ql/exec/tez/SampleTezSessionState.java 
973c0cc630 
  ql/src/test/org/apache/hadoop/hive/ql/exec/tez/TestTezSessionPool.java 
d2b98c46ca 
  ql/src/test/org/apache/hadoop/hive/ql/exec/tez/TestTezTask.java 176692b6e5 


Diff: https://reviews.apache.org/r/62048/diff/1/


Testing
---


Thanks,

Sergey Shelukhin

[jira] [Created] (HIVE-17431) change configuration handling in TezSessionState

2017-09-01 Thread Sergey Shelukhin (JIRA) 
Sergey Shelukhin created HIVE-17431:
---

 Summary: change configuration handling in TezSessionState
 Key: HIVE-17431
 URL: https://issues.apache.org/jira/browse/HIVE-17431
 Project: Hive
  Issue Type: Bug
Reporter: Sergey Shelukhin


The configuration is only set when opening the session; that seems unnecessary. 
E.g. when updating the session and localizing new resources we may 
theoretically open the session with a new config, but we don't update the 
config and only update the files if the session is already open, which seems to 
imply that it's ok to not update the config. 
In most cases, the session is opened only once or reopened without intending to 
change the config (e.g. if it times out).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 61165: HIVE-16811 Estimate statistics in absence of stats

2017-09-01 Thread Vineet Garg 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61165/
---

(Updated Sept. 1, 2017, 10:21 p.m.)


Review request for hive and Ashutosh Chauhan.


Changes
---

Q files update + review comments


Bugs: HIVE-16811
https://issues.apache.org/jira/browse/HIVE-16811


Repository: hive-git


Description
---

This patch introduces estimation of statistics if stats doesn't already exist.


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java e4b09a2cdd 
  itests/src/test/resources/testconfiguration.properties f452341d1c 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/RelOptHiveTable.java 
22790de209 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java
 ad29d65abb 
  ql/src/java/org/apache/hadoop/hive/ql/plan/ColStatistics.java f2d2e2dc0b 
  ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java 487a823d92 
  ql/src/test/queries/clientpositive/join_reordering_no_stats.q PRE-CREATION 
  ql/src/test/results/clientpositive/annotate_stats_filter.q.out e22c3ef0fc 
  ql/src/test/results/clientpositive/annotate_stats_groupby.q.out fccfabd5d1 
  ql/src/test/results/clientpositive/annotate_stats_part.q.out 866d30a8ea 
  ql/src/test/results/clientpositive/annotate_stats_select.q.out e3f08ea555 
  ql/src/test/results/clientpositive/annotate_stats_table.q.out efc3c1f123 
  ql/src/test/results/clientpositive/auto_join_reordering_values.q.out 
156be41502 
  ql/src/test/results/clientpositive/auto_join_stats.q.out e80af96fcb 
  ql/src/test/results/clientpositive/auto_join_stats2.q.out 6ea5afa920 
  ql/src/test/results/clientpositive/auto_sortmerge_join_12.q.out d129807f55 
  ql/src/test/results/clientpositive/cbo_rp_annotate_stats_groupby.q.out 
23f5fcfc76 
  ql/src/test/results/clientpositive/columnStatsUpdateForStatsOptimizer_2.q.out 
a7c9b3fc41 
  ql/src/test/results/clientpositive/explain_rearrange.q.out 2e5f9e4626 
  ql/src/test/results/clientpositive/join19.q.out dd167ecabf 
  ql/src/test/results/clientpositive/join42.q.out 0cd31c6d66 
  ql/src/test/results/clientpositive/join43.q.out 24168ca4fe 
  ql/src/test/results/clientpositive/join_cond_pushdown_unqual1.q.out 
f77c75c232 
  ql/src/test/results/clientpositive/join_cond_pushdown_unqual2.q.out 
f04519f8c5 
  ql/src/test/results/clientpositive/join_cond_pushdown_unqual3.q.out 
f15d536945 
  ql/src/test/results/clientpositive/join_cond_pushdown_unqual4.q.out 
1ec5dc415c 
  ql/src/test/results/clientpositive/join_hive_626.q.out 33a5fbd628 
  ql/src/test/results/clientpositive/join_star.q.out 261fb53bf9 
  ql/src/test/results/clientpositive/llap/acid_bucket_pruning.q.out 97f8d6b4cc 
  ql/src/test/results/clientpositive/llap/auto_smb_mapjoin_14.q.out 8bc280cedd 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_1.q.out 
4b646dcba5 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_10.q.out 
ab94c37fed 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_11.q.out 
7e3fe4c862 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_12.q.out 
054cf1af7e 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_13.q.out 
9c37848896 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_14.q.out 
21f1b3faff 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_15.q.out 
03ff5a6659 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_2.q.out 
42e4cb7a5c 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_3.q.out 
64c10ce953 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_4.q.out 
96854a9665 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_5.q.out 
b77a879b78 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_6.q.out 
9e6053675b 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_7.q.out 
13d3accf4c 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_8.q.out 
7fc90fd19f 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_9.q.out 
278c55b3e4 
  ql/src/test/results/clientpositive/llap/bucket_groupby.q.out 5913768a6d 
  ql/src/test/results/clientpositive/llap/bucket_map_join_tez1.q.out 649afb502a 
  ql/src/test/results/clientpositive/llap/bucket_map_join_tez2.q.out bac4ae086c 
  ql/src/test/results/clientpositive/llap/bucketmapjoin1.q.out feb3bbcfc4 
  ql/src/test/results/clientpositive/llap/bucketmapjoin2.q.out bda8070ba5 
  ql/src/test/results/clientpositive/llap/bucketmapjoin3.q.out ae17c1b707 
  ql/src/test/results/clientpositive/llap/bucketmapjoin4.q.out aab43431e6 
  ql/src/test/results/clientpositive/llap/bucketmapjoin7.q.out 3a2cbb2b14 
  ql/src/test/results/clientpositive/llap/bucketpruning1.q.out 922f23a29e 
  ql/src/test/results/clientpositive/llap/bucketsortoptimize_insert_2.q.out 
e4fed1124b

[jira] [Created] (HIVE-17430) Add LOAD DATA test for blobstores

2017-09-01 Thread Yuzhou Sun (JIRA) 
Yuzhou Sun created HIVE-17430:
-

 Summary: Add LOAD DATA test for blobstores
 Key: HIVE-17430
 URL: https://issues.apache.org/jira/browse/HIVE-17430
 Project: Hive
  Issue Type: Test
  Components: Tests
Affects Versions: 2.2.0
Reporter: Yuzhou Sun
Assignee: Yuzhou Sun


This patch introduces load_data.q regression tests into the hive-blobstore 
qtest module.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HIVE-17429) Hive JDBC doesn't return rows when querying Impala

2017-09-01 Thread Zach Amsden (JIRA) 
Zach Amsden created HIVE-17429:
--

 Summary: Hive JDBC doesn't return rows when querying Impala
 Key: HIVE-17429
 URL: https://issues.apache.org/jira/browse/HIVE-17429
 Project: Hive
  Issue Type: Bug
  Components: JDBC
Reporter: Zach Amsden
Assignee: Zach Amsden


The Hive JDBC driver used to return a result set when querying Impala.  Now, 
instead, it gets data back but interprets the data as query logs instead of a 
resultSet.  This causes many issues (we see complaints about beeline as well as 
test failures).

This appears to be a regression introduced with asynchronous operation against 
Hive.

Ideally, we could make both behaviors work.  I have a simple patch that should 
fix the problem.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 61165: HIVE-16811 Estimate statistics in absence of stats

2017-09-01 Thread Vineet Garg 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61165/
---

(Updated Sept. 1, 2017, 7 p.m.)


Review request for hive and Ashutosh Chauhan.


Changes
---

Addressed review comments


Bugs: HIVE-16811
https://issues.apache.org/jira/browse/HIVE-16811


Repository: hive-git


Description
---

This patch introduces estimation of statistics if stats doesn't already exist.


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java e4b09a2cdd 
  itests/src/test/resources/testconfiguration.properties f452341d1c 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/RelOptHiveTable.java 
22790de209 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java
 ad29d65abb 
  ql/src/java/org/apache/hadoop/hive/ql/plan/ColStatistics.java f2d2e2dc0b 
  ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java 487a823d92 
  ql/src/test/queries/clientpositive/join_reordering_no_stats.q PRE-CREATION 
  ql/src/test/results/clientpositive/annotate_stats_filter.q.out e22c3ef0fc 
  ql/src/test/results/clientpositive/annotate_stats_groupby.q.out fccfabd5d1 
  ql/src/test/results/clientpositive/annotate_stats_part.q.out 866d30a8ea 
  ql/src/test/results/clientpositive/annotate_stats_select.q.out e3f08ea555 
  ql/src/test/results/clientpositive/annotate_stats_table.q.out efc3c1f123 
  ql/src/test/results/clientpositive/auto_join_reordering_values.q.out 
156be41502 
  ql/src/test/results/clientpositive/auto_join_stats.q.out e80af96fcb 
  ql/src/test/results/clientpositive/auto_join_stats2.q.out 6ea5afa920 
  ql/src/test/results/clientpositive/auto_sortmerge_join_12.q.out d129807f55 
  ql/src/test/results/clientpositive/cbo_rp_annotate_stats_groupby.q.out 
23f5fcfc76 
  ql/src/test/results/clientpositive/columnStatsUpdateForStatsOptimizer_2.q.out 
a7c9b3fc41 
  ql/src/test/results/clientpositive/explain_rearrange.q.out 2e5f9e4626 
  ql/src/test/results/clientpositive/join19.q.out dd167ecabf 
  ql/src/test/results/clientpositive/join42.q.out 0cd31c6d66 
  ql/src/test/results/clientpositive/join43.q.out 24168ca4fe 
  ql/src/test/results/clientpositive/join_cond_pushdown_unqual1.q.out 
f77c75c232 
  ql/src/test/results/clientpositive/join_cond_pushdown_unqual2.q.out 
f04519f8c5 
  ql/src/test/results/clientpositive/join_cond_pushdown_unqual3.q.out 
f15d536945 
  ql/src/test/results/clientpositive/join_cond_pushdown_unqual4.q.out 
1ec5dc415c 
  ql/src/test/results/clientpositive/join_hive_626.q.out 33a5fbd628 
  ql/src/test/results/clientpositive/join_star.q.out 261fb53bf9 
  ql/src/test/results/clientpositive/llap/acid_bucket_pruning.q.out 97f8d6b4cc 
  ql/src/test/results/clientpositive/llap/auto_smb_mapjoin_14.q.out 8bc280cedd 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_1.q.out 
4b646dcba5 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_10.q.out 
ab94c37fed 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_11.q.out 
7e3fe4c862 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_12.q.out 
054cf1af7e 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_13.q.out 
9c37848896 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_14.q.out 
21f1b3faff 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_15.q.out 
03ff5a6659 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_2.q.out 
42e4cb7a5c 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_3.q.out 
64c10ce953 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_4.q.out 
96854a9665 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_5.q.out 
b77a879b78 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_6.q.out 
9e6053675b 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_7.q.out 
13d3accf4c 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_8.q.out 
7fc90fd19f 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_9.q.out 
278c55b3e4 
  ql/src/test/results/clientpositive/llap/bucket_groupby.q.out 5913768a6d 
  ql/src/test/results/clientpositive/llap/bucket_map_join_tez1.q.out 649afb502a 
  ql/src/test/results/clientpositive/llap/bucket_map_join_tez2.q.out bac4ae086c 
  ql/src/test/results/clientpositive/llap/bucketmapjoin1.q.out feb3bbcfc4 
  ql/src/test/results/clientpositive/llap/bucketmapjoin2.q.out bda8070ba5 
  ql/src/test/results/clientpositive/llap/bucketmapjoin3.q.out ae17c1b707 
  ql/src/test/results/clientpositive/llap/bucketmapjoin4.q.out aab43431e6 
  ql/src/test/results/clientpositive/llap/bucketmapjoin7.q.out 3a2cbb2b14 
  ql/src/test/results/clientpositive/llap/bucketpruning1.q.out 922f23a29e 
  ql/src/test/results/clientpositive/llap/bucketsortoptimize_insert_2.q.out 
e4fed1124b 
  ql/src/test/results/clientpositive/llap/bucketsortoptimize_insert_6.q.out

Re: Review Request 61165: HIVE-16811 Estimate statistics in absence of stats

2017-09-01 Thread Ashutosh Chauhan 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61165/#review184359
---




ql/src/test/queries/clientpositive/join_reordering_no_stats.q
Lines 52 (patched)


Need test for estimateStats=false.



ql/src/test/results/clientpositive/annotate_stats_table.q.out
Line 45 (original), 45 (patched)


This test is specifically for stats state. This change looks incorrect.



ql/src/test/results/clientpositive/columnStatsUpdateForStatsOptimizer_2.q.out
Line 325 (original), 325 (patched)


This doesn't look correct. Estimated stats with state: COMPLETE.


- Ashutosh Chauhan


On Sept. 1, 2017, 7:49 a.m., Vineet Garg wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61165/
> ---
> 
> (Updated Sept. 1, 2017, 7:49 a.m.)
> 
> 
> Review request for hive and Ashutosh Chauhan.
> 
> 
> Bugs: HIVE-16811
> https://issues.apache.org/jira/browse/HIVE-16811
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> This patch introduces estimation of statistics if stats doesn't already exist.
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java e4b09a2cdd 
>   itests/src/test/resources/testconfiguration.properties f452341d1c 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/RelOptHiveTable.java 
> 22790de209 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java
>  ad29d65abb 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/ColStatistics.java f2d2e2dc0b 
>   ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java 487a823d92 
>   ql/src/test/queries/clientpositive/join_reordering_no_stats.q PRE-CREATION 
>   ql/src/test/results/clientpositive/annotate_stats_filter.q.out e22c3ef0fc 
>   ql/src/test/results/clientpositive/annotate_stats_groupby.q.out fccfabd5d1 
>   ql/src/test/results/clientpositive/annotate_stats_part.q.out 866d30a8ea 
>   ql/src/test/results/clientpositive/annotate_stats_select.q.out e3f08ea555 
>   ql/src/test/results/clientpositive/annotate_stats_table.q.out efc3c1f123 
>   ql/src/test/results/clientpositive/auto_join_reordering_values.q.out 
> 156be41502 
>   ql/src/test/results/clientpositive/auto_join_stats.q.out e80af96fcb 
>   ql/src/test/results/clientpositive/auto_join_stats2.q.out 6ea5afa920 
>   ql/src/test/results/clientpositive/auto_sortmerge_join_12.q.out d129807f55 
>   ql/src/test/results/clientpositive/cbo_rp_annotate_stats_groupby.q.out 
> 23f5fcfc76 
>   
> ql/src/test/results/clientpositive/columnStatsUpdateForStatsOptimizer_2.q.out 
> a7c9b3fc41 
>   ql/src/test/results/clientpositive/explain_rearrange.q.out 2e5f9e4626 
>   ql/src/test/results/clientpositive/join19.q.out dd167ecabf 
>   ql/src/test/results/clientpositive/join42.q.out 0cd31c6d66 
>   ql/src/test/results/clientpositive/join43.q.out 24168ca4fe 
>   ql/src/test/results/clientpositive/join_cond_pushdown_unqual1.q.out 
> f77c75c232 
>   ql/src/test/results/clientpositive/join_cond_pushdown_unqual2.q.out 
> f04519f8c5 
>   ql/src/test/results/clientpositive/join_cond_pushdown_unqual3.q.out 
> f15d536945 
>   ql/src/test/results/clientpositive/join_cond_pushdown_unqual4.q.out 
> 1ec5dc415c 
>   ql/src/test/results/clientpositive/join_hive_626.q.out 33a5fbd628 
>   ql/src/test/results/clientpositive/join_star.q.out 261fb53bf9 
>   ql/src/test/results/clientpositive/llap/acid_bucket_pruning.q.out 
> 97f8d6b4cc 
>   ql/src/test/results/clientpositive/llap/auto_smb_mapjoin_14.q.out 
> 8bc280cedd 
>   ql/src/test/results/clientpositive/llap/auto_sortmerge_join_1.q.out 
> 4b646dcba5 
>   ql/src/test/results/clientpositive/llap/auto_sortmerge_join_10.q.out 
> ab94c37fed 
>   ql/src/test/results/clientpositive/llap/auto_sortmerge_join_11.q.out 
> 7e3fe4c862 
>   ql/src/test/results/clientpositive/llap/auto_sortmerge_join_12.q.out 
> 054cf1af7e 
>   ql/src/test/results/clientpositive/llap/auto_sortmerge_join_13.q.out 
> 9c37848896 
>   ql/src/test/results/clientpositive/llap/auto_sortmerge_join_14.q.out 
> 21f1b3faff 
>   ql/src/test/results/clientpositive/llap/auto_sortmerge_join_15.q.out 
> 03ff5a6659 
>   ql/src/test/results/clientpositive/llap/auto_sortmerge_join_2.q.out 
> 42e4cb7a5c 
>   ql/src/test/results/clientpositive/llap/auto_sortmerge_join_3.q.out 
> 64c10ce953 
>   ql/src/test/results/clientpositive/llap/auto_sortmerge_join_4.q.out 
> 96854a9665 
>   ql/src/test/results/clientpositive/llap/auto_sortmerge_join_5.q.out 
> b77a879b78 
>   ql/src/test/results/clientpositive/llap/auto_sortmerge_join_6.q.out 
> 9e6053675b 
>

Re: Review Request 61165: HIVE-16811 Estimate statistics in absence of stats

2017-09-01 Thread Ashutosh Chauhan 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61165/#review184356
---




ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java
Lines 203 (patched)


This function returns 0 even when stats are absent, making rest of logic to 
not work correctly.



ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java
Lines 222 (patched)


This function also returns 0 when stats are absent, making rest of logic to 
not work correctly.



ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java
Lines 325 (patched)


Its also possible that stats for few cols are present and absent for 
others. Seems like that case is not handled.



ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java
Lines 1006 (patched)


This wont be correct for tiny and small int.



ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java
Lines 1009 (patched)


Long.max and Long.min



ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java
Lines 1034 (patched)


nulls are trouble some. Lets arbitartily pick start of 1970 (unix epoch) 
and end of unix epoch (2038). Can set same for ts.



ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java
Lines 1036 (patched)


todo needed


- Ashutosh Chauhan


On Sept. 1, 2017, 7:49 a.m., Vineet Garg wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61165/
> ---
> 
> (Updated Sept. 1, 2017, 7:49 a.m.)
> 
> 
> Review request for hive and Ashutosh Chauhan.
> 
> 
> Bugs: HIVE-16811
> https://issues.apache.org/jira/browse/HIVE-16811
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> This patch introduces estimation of statistics if stats doesn't already exist.
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java e4b09a2cdd 
>   itests/src/test/resources/testconfiguration.properties f452341d1c 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/RelOptHiveTable.java 
> 22790de209 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java
>  ad29d65abb 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/ColStatistics.java f2d2e2dc0b 
>   ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java 487a823d92 
>   ql/src/test/queries/clientpositive/join_reordering_no_stats.q PRE-CREATION 
>   ql/src/test/results/clientpositive/annotate_stats_filter.q.out e22c3ef0fc 
>   ql/src/test/results/clientpositive/annotate_stats_groupby.q.out fccfabd5d1 
>   ql/src/test/results/clientpositive/annotate_stats_part.q.out 866d30a8ea 
>   ql/src/test/results/clientpositive/annotate_stats_select.q.out e3f08ea555 
>   ql/src/test/results/clientpositive/annotate_stats_table.q.out efc3c1f123 
>   ql/src/test/results/clientpositive/auto_join_reordering_values.q.out 
> 156be41502 
>   ql/src/test/results/clientpositive/auto_join_stats.q.out e80af96fcb 
>   ql/src/test/results/clientpositive/auto_join_stats2.q.out 6ea5afa920 
>   ql/src/test/results/clientpositive/auto_sortmerge_join_12.q.out d129807f55 
>   ql/src/test/results/clientpositive/cbo_rp_annotate_stats_groupby.q.out 
> 23f5fcfc76 
>   
> ql/src/test/results/clientpositive/columnStatsUpdateForStatsOptimizer_2.q.out 
> a7c9b3fc41 
>   ql/src/test/results/clientpositive/explain_rearrange.q.out 2e5f9e4626 
>   ql/src/test/results/clientpositive/join19.q.out dd167ecabf 
>   ql/src/test/results/clientpositive/join42.q.out 0cd31c6d66 
>   ql/src/test/results/clientpositive/join43.q.out 24168ca4fe 
>   ql/src/test/results/clientpositive/join_cond_pushdown_unqual1.q.out 
> f77c75c232 
>   ql/src/test/results/clientpositive/join_cond_pushdown_unqual2.q.out 
> f04519f8c5 
>   ql/src/test/results/clientpositive/join_cond_pushdown_unqual3.q.out 
> f15d536945 
>   ql/src/test/results/clientpositive/join_cond_pushdown_unqual4.q.out 
> 1ec5dc415c 
>   ql/src/test/results/clientpositive/join_hive_626.q.out 33a5fbd628 
>   ql/src/test/results/clientpositive/join_star.q.out 261fb53bf9 
>   ql/src/test/results/clientpositive/llap/acid_bucket_pruning.q.out 
> 97f8d6b4cc 
>   ql/src/test/results/clientpositive/llap/auto_smb_mapjoin_14.q.out 
> 8bc280cedd 
>   ql/src/test/results/clientpositive/llap/auto_sortmerge_join_1.q.out 
> 4b646dcba5 
>   ql/src/test/results/clientpositive/llap/auto_sortmerge_join_10.q.out 
> ab94c37fed 
>

Re: Review Request 61165: HIVE-16811 Estimate statistics in absence of stats

2017-09-01 Thread Vineet Garg 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61165/
---

(Updated Sept. 1, 2017, 7:49 a.m.)


Review request for hive and Ashutosh Chauhan.


Changes
---

Q files update


Bugs: HIVE-16811
https://issues.apache.org/jira/browse/HIVE-16811


Repository: hive-git


Description
---

This patch introduces estimation of statistics if stats doesn't already exist.


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java e4b09a2cdd 
  itests/src/test/resources/testconfiguration.properties f452341d1c 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/RelOptHiveTable.java 
22790de209 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java
 ad29d65abb 
  ql/src/java/org/apache/hadoop/hive/ql/plan/ColStatistics.java f2d2e2dc0b 
  ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java 487a823d92 
  ql/src/test/queries/clientpositive/join_reordering_no_stats.q PRE-CREATION 
  ql/src/test/results/clientpositive/annotate_stats_filter.q.out e22c3ef0fc 
  ql/src/test/results/clientpositive/annotate_stats_groupby.q.out fccfabd5d1 
  ql/src/test/results/clientpositive/annotate_stats_part.q.out 866d30a8ea 
  ql/src/test/results/clientpositive/annotate_stats_select.q.out e3f08ea555 
  ql/src/test/results/clientpositive/annotate_stats_table.q.out efc3c1f123 
  ql/src/test/results/clientpositive/auto_join_reordering_values.q.out 
156be41502 
  ql/src/test/results/clientpositive/auto_join_stats.q.out e80af96fcb 
  ql/src/test/results/clientpositive/auto_join_stats2.q.out 6ea5afa920 
  ql/src/test/results/clientpositive/auto_sortmerge_join_12.q.out d129807f55 
  ql/src/test/results/clientpositive/cbo_rp_annotate_stats_groupby.q.out 
23f5fcfc76 
  ql/src/test/results/clientpositive/columnStatsUpdateForStatsOptimizer_2.q.out 
a7c9b3fc41 
  ql/src/test/results/clientpositive/explain_rearrange.q.out 2e5f9e4626 
  ql/src/test/results/clientpositive/join19.q.out dd167ecabf 
  ql/src/test/results/clientpositive/join42.q.out 0cd31c6d66 
  ql/src/test/results/clientpositive/join43.q.out 24168ca4fe 
  ql/src/test/results/clientpositive/join_cond_pushdown_unqual1.q.out 
f77c75c232 
  ql/src/test/results/clientpositive/join_cond_pushdown_unqual2.q.out 
f04519f8c5 
  ql/src/test/results/clientpositive/join_cond_pushdown_unqual3.q.out 
f15d536945 
  ql/src/test/results/clientpositive/join_cond_pushdown_unqual4.q.out 
1ec5dc415c 
  ql/src/test/results/clientpositive/join_hive_626.q.out 33a5fbd628 
  ql/src/test/results/clientpositive/join_star.q.out 261fb53bf9 
  ql/src/test/results/clientpositive/llap/acid_bucket_pruning.q.out 97f8d6b4cc 
  ql/src/test/results/clientpositive/llap/auto_smb_mapjoin_14.q.out 8bc280cedd 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_1.q.out 
4b646dcba5 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_10.q.out 
ab94c37fed 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_11.q.out 
7e3fe4c862 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_12.q.out 
054cf1af7e 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_13.q.out 
9c37848896 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_14.q.out 
21f1b3faff 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_15.q.out 
03ff5a6659 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_2.q.out 
42e4cb7a5c 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_3.q.out 
64c10ce953 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_4.q.out 
96854a9665 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_5.q.out 
b77a879b78 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_6.q.out 
9e6053675b 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_7.q.out 
13d3accf4c 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_8.q.out 
7fc90fd19f 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_9.q.out 
278c55b3e4 
  ql/src/test/results/clientpositive/llap/bucket_groupby.q.out 5913768a6d 
  ql/src/test/results/clientpositive/llap/bucket_map_join_tez1.q.out 649afb502a 
  ql/src/test/results/clientpositive/llap/bucket_map_join_tez2.q.out bac4ae086c 
  ql/src/test/results/clientpositive/llap/bucketmapjoin1.q.out feb3bbcfc4 
  ql/src/test/results/clientpositive/llap/bucketmapjoin2.q.out bda8070ba5 
  ql/src/test/results/clientpositive/llap/bucketmapjoin3.q.out ae17c1b707 
  ql/src/test/results/clientpositive/llap/bucketmapjoin4.q.out aab43431e6 
  ql/src/test/results/clientpositive/llap/bucketmapjoin7.q.out 3a2cbb2b14 
  ql/src/test/results/clientpositive/llap/bucketpruning1.q.out 922f23a29e 
  ql/src/test/results/clientpositive/llap/bucketsortoptimize_insert_2.q.out 
e4fed1124b 
  ql/src/test/results/clientpositive/llap/bucketsortoptimize_insert_6.q.out 
67e925ad99