[jira] [Commented] (HIVE-1989) recognize transitivity of predicates on join keys
[
https://issues.apache.org/jira/browse/HIVE-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13096627#comment-13096627
]
Hudson commented on HIVE-1989:
--
Integrated in Hive-trunk-h0.21 #931 (See
[https://builds.apache.org/job/Hive-trunk-h0.21/931/])
HIVE-1989. Recognize transitivity of predicates on join keys
(Charles Chen via jvs)
jvs : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1164804
Files :
* /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
* /hive/trunk/conf/hive-default.xml
*
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java
*
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/ppd/ExprWalkerProcFactory.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java
* /hive/trunk/ql/src/test/results/clientpositive/auto_join16.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join8.q.out
* /hive/trunk/ql/src/test/results/clientpositive/cluster.q.out
* /hive/trunk/ql/src/test/results/clientpositive/index_auto_mult_tables.q.out
*
/hive/trunk/ql/src/test/results/clientpositive/index_auto_mult_tables_compact.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join16.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join38.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join8.q.out
* /hive/trunk/ql/src/test/results/clientpositive/louter_join_ppr.q.out
* /hive/trunk/ql/src/test/results/clientpositive/outer_join_ppr.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd_clusterby.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd_gby_join.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd_join.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd_join2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd_join3.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join3.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join4.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join5.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd_udf_case.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd_union.q.out
* /hive/trunk/ql/src/test/results/clientpositive/router_join_ppr.q.out
* /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin9.q.out
* /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_6.q.out
* /hive/trunk/ql/src/test/results/clientpositive/udf_named_struct.q.out
* /hive/trunk/ql/src/test/results/clientpositive/union22.q.out
* /hive/trunk/ql/src/test/results/compiler/plan/join1.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/join2.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/join3.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/join8.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/sample1.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/sample2.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/sample3.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/sample4.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/sample5.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/sample6.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/sample7.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/subq.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/udf1.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/udf4.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/udf6.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/udf_case.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/udf_when.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/union.q.xml
recognize transitivity of predicates on join keys
-
Key: HIVE-1989
URL: https://issues.apache.org/jira/browse/HIVE-1989
Project: Hive
Issue Type: Improvement
Components: Query Processor
Affects Versions: 0.6.0
Reporter: John Sichi
Assignee: Charles Chen
Fix For: 0.9.0
Attachments: HIVE-1989v1.patch, HIVE-1989v10.patch,
HIVE-1989v11.patch, HIVE-1989v12.patch, HIVE-1989v4.patch,
HIVE-1989v5-WITH-HIVE-2382v1.patch, HIVE-1989v6-WITH-HIVE-2383v1.patch,
HIVE-1989v8.patch, HIVE-1989v9.patch
Given
{noformat}
set hive.mapred.mode=strict;
create table invites (foo int, bar string) partitioned by (ds string);
create table invites2 (foo int, bar string) partitioned by (ds string);
select count(*) from invites join invites2 on invites.ds=invites2.ds where
invites.ds='2011-01-01';
{noformat}
currently an error occurs:
{noformat}
Error in semantic analysis: No Partition Predicate Found for Alias invites2
Table invites2
{noformat}
The optimizer should be able to infer a predicate on invites2
[jira] [Commented] (HIVE-1989) recognize transitivity of predicates on join keys
[ https://issues.apache.org/jira/browse/HIVE-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095797#comment-13095797 ] jirapos...@reviews.apache.org commented on HIVE-1989: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1228/ --- (Updated 2011-09-02 06:26:31.707450) Review request for hive. Changes --- Rebase to trunk Summary --- https://issues.apache.org/jira/browse/HIVE-1989 This addresses bug HIVE-1989. https://issues.apache.org/jira/browse/HIVE-1989 Diffs (updated) - http://svn.apache.org/repos/asf/hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1164353 http://svn.apache.org/repos/asf/hive/trunk/conf/hive-default.xml 1164353 http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java 1164353 http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/ppd/ExprWalkerProcFactory.java 1164353 http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java 1164353 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/auto_join16.q.out 1164353 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/auto_join8.q.out 1164353 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/cluster.q.out 1164353 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_mult_tables.q.out 1164353 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_mult_tables_compact.q.out 1164353 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join16.q.out 1164353 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join38.q.out 1164353 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join8.q.out 1164353 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/louter_join_ppr.q.out 1164353 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_clusterby.q.out 1164353 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_gby_join.q.out 1164353 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join.q.out 1164353 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join2.q.out 1164353 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join3.q.out 1164353 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join1.q.out 1164353 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join2.q.out 1164353 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join4.q.out 1164353 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_udf_case.q.out 1164353 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/router_join_ppr.q.out 1164353 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin9.q.out 1164353 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_6.q.out 1164353 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_named_struct.q.out 1164353 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/union22.q.out 1164353 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join1.q.xml 1164353 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join2.q.xml 1164353 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join3.q.xml 1164353 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join8.q.xml 1164353 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample1.q.xml 1164353 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample2.q.xml 1164353 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample3.q.xml 1164353 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample4.q.xml 1164353 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample5.q.xml 1164353 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample6.q.xml 1164353
[jira] [Commented] (HIVE-1989) recognize transitivity of predicates on join keys
[ https://issues.apache.org/jira/browse/HIVE-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13096118#comment-13096118 ] jirapos...@reviews.apache.org commented on HIVE-1989: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1228/ --- (Updated 2011-09-02 16:49:23.015066) Review request for hive. Changes --- Fixed unit tests Summary --- https://issues.apache.org/jira/browse/HIVE-1989 This addresses bug HIVE-1989. https://issues.apache.org/jira/browse/HIVE-1989 Diffs (updated) - http://svn.apache.org/repos/asf/hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1164358 http://svn.apache.org/repos/asf/hive/trunk/conf/hive-default.xml 1164358 http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java 1164358 http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/ppd/ExprWalkerProcFactory.java 1164358 http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java 1164358 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/auto_join16.q.out 1164358 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/auto_join8.q.out 1164358 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/cluster.q.out 1164358 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_mult_tables.q.out 1164358 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_mult_tables_compact.q.out 1164358 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join16.q.out 1164358 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join38.q.out 1164358 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join8.q.out 1164358 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/louter_join_ppr.q.out 1164358 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/outer_join_ppr.q.out 1164358 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_clusterby.q.out 1164358 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_gby_join.q.out 1164358 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join.q.out 1164358 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join2.q.out 1164358 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join3.q.out 1164358 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join1.q.out 1164358 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join2.q.out 1164358 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join3.q.out 1164358 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join4.q.out 1164358 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join5.q.out 1164358 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_udf_case.q.out 1164358 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_union.q.out 1164358 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/router_join_ppr.q.out 1164358 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin9.q.out 1164358 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_6.q.out 1164358 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_named_struct.q.out 1164358 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/union22.q.out 1164358 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join1.q.xml 1164358 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join2.q.xml 1164358 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join3.q.xml 1164358 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join8.q.xml 1164358 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample1.q.xml 1164358 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample2.q.xml 1164358
[jira] [Commented] (HIVE-1989) recognize transitivity of predicates on join keys
[
https://issues.apache.org/jira/browse/HIVE-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13096120#comment-13096120
]
Charles Chen commented on HIVE-1989:
All but ppd_outer_join5 are due to extractFinalCandidates now extracting each
predicate from an AND clause and trying to replicate each one of these instead
of the whole and expression, so there are parentheses changes and stuff and
some plan improvements. ppd_outer_join5 is improved because HIVE-2337 got
committed (ppd_outer_join5 is added in this patch)
recognize transitivity of predicates on join keys
-
Key: HIVE-1989
URL: https://issues.apache.org/jira/browse/HIVE-1989
Project: Hive
Issue Type: Improvement
Components: Query Processor
Affects Versions: 0.6.0
Reporter: John Sichi
Assignee: Charles Chen
Fix For: 0.8.0
Attachments: HIVE-1989v1.patch, HIVE-1989v10.patch,
HIVE-1989v11.patch, HIVE-1989v12.patch, HIVE-1989v4.patch,
HIVE-1989v5-WITH-HIVE-2382v1.patch, HIVE-1989v6-WITH-HIVE-2383v1.patch,
HIVE-1989v8.patch, HIVE-1989v9.patch
Given
{noformat}
set hive.mapred.mode=strict;
create table invites (foo int, bar string) partitioned by (ds string);
create table invites2 (foo int, bar string) partitioned by (ds string);
select count(*) from invites join invites2 on invites.ds=invites2.ds where
invites.ds='2011-01-01';
{noformat}
currently an error occurs:
{noformat}
Error in semantic analysis: No Partition Predicate Found for Alias invites2
Table invites2
{noformat}
The optimizer should be able to infer a predicate on invites2 via
transitivity. The current lack places a burden on the user to add a
redundant predicate, and makes impossible (at least in strict mode) join
views where both underlying tables are partitioned (the join select list has
to pick one of the tables arbitrarily).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1989) recognize transitivity of predicates on join keys
[
https://issues.apache.org/jira/browse/HIVE-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13096130#comment-13096130
]
Charles Chen commented on HIVE-1989:
wait actually ppd_outer_join5 was not added here...
recognize transitivity of predicates on join keys
-
Key: HIVE-1989
URL: https://issues.apache.org/jira/browse/HIVE-1989
Project: Hive
Issue Type: Improvement
Components: Query Processor
Affects Versions: 0.6.0
Reporter: John Sichi
Assignee: Charles Chen
Fix For: 0.8.0
Attachments: HIVE-1989v1.patch, HIVE-1989v10.patch,
HIVE-1989v11.patch, HIVE-1989v12.patch, HIVE-1989v4.patch,
HIVE-1989v5-WITH-HIVE-2382v1.patch, HIVE-1989v6-WITH-HIVE-2383v1.patch,
HIVE-1989v8.patch, HIVE-1989v9.patch
Given
{noformat}
set hive.mapred.mode=strict;
create table invites (foo int, bar string) partitioned by (ds string);
create table invites2 (foo int, bar string) partitioned by (ds string);
select count(*) from invites join invites2 on invites.ds=invites2.ds where
invites.ds='2011-01-01';
{noformat}
currently an error occurs:
{noformat}
Error in semantic analysis: No Partition Predicate Found for Alias invites2
Table invites2
{noformat}
The optimizer should be able to infer a predicate on invites2 via
transitivity. The current lack places a burden on the user to add a
redundant predicate, and makes impossible (at least in strict mode) join
views where both underlying tables are partitioned (the join select list has
to pick one of the tables arbitrarily).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1989) recognize transitivity of predicates on join keys
[
https://issues.apache.org/jira/browse/HIVE-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13096433#comment-13096433
]
John Sichi commented on HIVE-1989:
--
+1. Will commit when tests pass.
recognize transitivity of predicates on join keys
-
Key: HIVE-1989
URL: https://issues.apache.org/jira/browse/HIVE-1989
Project: Hive
Issue Type: Improvement
Components: Query Processor
Affects Versions: 0.6.0
Reporter: John Sichi
Assignee: Charles Chen
Fix For: 0.8.0
Attachments: HIVE-1989v1.patch, HIVE-1989v10.patch,
HIVE-1989v11.patch, HIVE-1989v12.patch, HIVE-1989v4.patch,
HIVE-1989v5-WITH-HIVE-2382v1.patch, HIVE-1989v6-WITH-HIVE-2383v1.patch,
HIVE-1989v8.patch, HIVE-1989v9.patch
Given
{noformat}
set hive.mapred.mode=strict;
create table invites (foo int, bar string) partitioned by (ds string);
create table invites2 (foo int, bar string) partitioned by (ds string);
select count(*) from invites join invites2 on invites.ds=invites2.ds where
invites.ds='2011-01-01';
{noformat}
currently an error occurs:
{noformat}
Error in semantic analysis: No Partition Predicate Found for Alias invites2
Table invites2
{noformat}
The optimizer should be able to infer a predicate on invites2 via
transitivity. The current lack places a burden on the user to add a
redundant predicate, and makes impossible (at least in strict mode) join
views where both underlying tables are partitioned (the join select list has
to pick one of the tables arbitrarily).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1989) recognize transitivity of predicates on join keys
[
https://issues.apache.org/jira/browse/HIVE-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13096586#comment-13096586
]
Charles Chen commented on HIVE-1989:
My internship at Facebook is over.
Please email jsi...@fb.com for Facebook-related inquiries, or c...@stanford.edu
for Charles-related inquiries.
recognize transitivity of predicates on join keys
-
Key: HIVE-1989
URL: https://issues.apache.org/jira/browse/HIVE-1989
Project: Hive
Issue Type: Improvement
Components: Query Processor
Affects Versions: 0.6.0
Reporter: John Sichi
Assignee: Charles Chen
Fix For: 0.9.0
Attachments: HIVE-1989v1.patch, HIVE-1989v10.patch,
HIVE-1989v11.patch, HIVE-1989v12.patch, HIVE-1989v4.patch,
HIVE-1989v5-WITH-HIVE-2382v1.patch, HIVE-1989v6-WITH-HIVE-2383v1.patch,
HIVE-1989v8.patch, HIVE-1989v9.patch
Given
{noformat}
set hive.mapred.mode=strict;
create table invites (foo int, bar string) partitioned by (ds string);
create table invites2 (foo int, bar string) partitioned by (ds string);
select count(*) from invites join invites2 on invites.ds=invites2.ds where
invites.ds='2011-01-01';
{noformat}
currently an error occurs:
{noformat}
Error in semantic analysis: No Partition Predicate Found for Alias invites2
Table invites2
{noformat}
The optimizer should be able to infer a predicate on invites2 via
transitivity. The current lack places a burden on the user to add a
redundant predicate, and makes impossible (at least in strict mode) join
views where both underlying tables are partitioned (the join select list has
to pick one of the tables arbitrarily).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1989) recognize transitivity of predicates on join keys
[ https://issues.apache.org/jira/browse/HIVE-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095473#comment-13095473 ] jirapos...@reviews.apache.org commented on HIVE-1989: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1228/ --- (Updated 2011-09-01 17:58:25.969663) Review request for hive. Changes --- Fix unit tests Summary --- https://issues.apache.org/jira/browse/HIVE-1989 This addresses bug HIVE-1989. https://issues.apache.org/jira/browse/HIVE-1989 Diffs (updated) - http://svn.apache.org/repos/asf/hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1163909 http://svn.apache.org/repos/asf/hive/trunk/conf/hive-default.xml 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/ppd_transitivity.q PRE-CREATION http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/auto_join16.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/auto_join8.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/cluster.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_mult_tables.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_mult_tables_compact.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join16.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join38.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join8.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/louter_join_ppr.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_clusterby.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_gby_join.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join2.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join3.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join1.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join2.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join4.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_transitivity.q.out PRE-CREATION http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_udf_case.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/router_join_ppr.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin9.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_6.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_named_struct.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/union22.q.out 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/input4.q.xml 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join1.q.xml 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join2.q.xml 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join3.q.xml 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join8.q.xml 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample1.q.xml 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample2.q.xml 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample3.q.xml 1163909 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample4.q.xml 1163909
[jira] [Commented] (HIVE-1989) recognize transitivity of predicates on join keys
[ https://issues.apache.org/jira/browse/HIVE-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095619#comment-13095619 ] jirapos...@reviews.apache.org commented on HIVE-1989: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1228/ --- (Updated 2011-09-01 21:31:00.970058) Review request for hive. Changes --- Changed extractFinalCandidates() to decompose AND expression into parts so the parts can be replicated over transitively even if the whole thing can't. Added comments. Summary --- https://issues.apache.org/jira/browse/HIVE-1989 This addresses bug HIVE-1989. https://issues.apache.org/jira/browse/HIVE-1989 Diffs (updated) - http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/union.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/udf_when.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/udf_case.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/udf1.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/udf4.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/udf6.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/subq.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample4.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample5.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample6.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample7.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample3.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample2.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample1.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/input4.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join1.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join2.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join3.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join8.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin9.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_6.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_named_struct.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/union22.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/router_join_ppr.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_udf_case.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join2.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join4.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_repeated_alias2.q.out PRE-CREATION http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_transitivity.q.out PRE-CREATION http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join1.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_gby_join.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join2.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join3.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_clusterby.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join16.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join38.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join8.q.out 1163905
[jira] [Commented] (HIVE-1989) recognize transitivity of predicates on join keys
[
https://issues.apache.org/jira/browse/HIVE-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095643#comment-13095643
]
John Sichi commented on HIVE-1989:
--
Charles, can you add a test case for the original partitioned join view use
case? Separate JIRA is fine.
recognize transitivity of predicates on join keys
-
Key: HIVE-1989
URL: https://issues.apache.org/jira/browse/HIVE-1989
Project: Hive
Issue Type: Improvement
Components: Query Processor
Affects Versions: 0.6.0
Reporter: John Sichi
Assignee: Charles Chen
Fix For: 0.8.0
Attachments: HIVE-1989v1.patch, HIVE-1989v4.patch,
HIVE-1989v5-WITH-HIVE-2382v1.patch, HIVE-1989v6-WITH-HIVE-2383v1.patch,
HIVE-1989v8.patch, HIVE-1989v9.patch
Given
{noformat}
set hive.mapred.mode=strict;
create table invites (foo int, bar string) partitioned by (ds string);
create table invites2 (foo int, bar string) partitioned by (ds string);
select count(*) from invites join invites2 on invites.ds=invites2.ds where
invites.ds='2011-01-01';
{noformat}
currently an error occurs:
{noformat}
Error in semantic analysis: No Partition Predicate Found for Alias invites2
Table invites2
{noformat}
The optimizer should be able to infer a predicate on invites2 via
transitivity. The current lack places a burden on the user to add a
redundant predicate, and makes impossible (at least in strict mode) join
views where both underlying tables are partitioned (the join select list has
to pick one of the tables arbitrarily).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1989) recognize transitivity of predicates on join keys
[ https://issues.apache.org/jira/browse/HIVE-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095784#comment-13095784 ] jirapos...@reviews.apache.org commented on HIVE-1989: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1228/ --- (Updated 2011-09-02 05:47:55.711939) Review request for hive. Changes --- Added unparseIdentifier for ASTNode handling Summary --- https://issues.apache.org/jira/browse/HIVE-1989 This addresses bug HIVE-1989. https://issues.apache.org/jira/browse/HIVE-1989 Diffs (updated) - http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample6.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample4.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample5.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample3.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join8.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample1.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample2.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join3.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join2.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join1.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/input4.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin9.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_6.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_named_struct.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/union22.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_transitivity.q.out PRE-CREATION http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_udf_case.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/router_join_ppr.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join1.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join2.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join4.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_repeated_alias2.q.out PRE-CREATION http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join2.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join3.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_gby_join.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_clusterby.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join8.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/louter_join_ppr.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join16.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join38.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_mult_tables.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_mult_tables_compact.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/ppd_repeated_alias2.q PRE-CREATION http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/ppd_transitivity.q PRE-CREATION http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/auto_join16.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/auto_join8.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/cluster.q.out 1163905
[jira] [Commented] (HIVE-1989) recognize transitivity of predicates on join keys
[ https://issues.apache.org/jira/browse/HIVE-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095134#comment-13095134 ] jirapos...@reviews.apache.org commented on HIVE-1989: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1228/ --- (Updated 2011-09-01 05:43:51.993258) Review request for hive. Changes --- Rebase to trunk Summary --- https://issues.apache.org/jira/browse/HIVE-1989 This addresses bug HIVE-1989. https://issues.apache.org/jira/browse/HIVE-1989 Diffs (updated) - http://svn.apache.org/repos/asf/hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1163905 http://svn.apache.org/repos/asf/hive/trunk/conf/hive-default.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/ppd_transitivity.q PRE-CREATION http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/auto_join16.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/cluster.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_mult_tables.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_mult_tables_compact.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join16.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join38.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/louter_join_ppr.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_clusterby.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_gby_join.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join2.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join3.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join1.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join2.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join4.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_transitivity.q.out PRE-CREATION http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_udf_case.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/router_join_ppr.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin9.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_6.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_named_struct.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/union22.q.out 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join1.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join2.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join3.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample1.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample2.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample3.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample4.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample5.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample6.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample7.q.xml 1163905 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/subq.q.xml 1163905
[jira] [Commented] (HIVE-1989) recognize transitivity of predicates on join keys
[ https://issues.apache.org/jira/browse/HIVE-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13087250#comment-13087250 ] jirapos...@reviews.apache.org commented on HIVE-1989: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1228/ --- (Updated 2011-08-18 20:18:26.581983) Review request for hive. Changes --- Updated unit test plan changes Summary --- https://issues.apache.org/jira/browse/HIVE-1989 This addresses bug HIVE-1989. https://issues.apache.org/jira/browse/HIVE-1989 Diffs (updated) - http://svn.apache.org/repos/asf/hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1158991 http://svn.apache.org/repos/asf/hive/trunk/conf/hive-default.xml 1158991 http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java 1158991 http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java 1158991 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/ppd_transitivity.q PRE-CREATION http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/auto_join16.q.out 1158991 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/cluster.q.out 1158991 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_mult_tables.q.out 1158991 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_mult_tables_compact.q.out 1158991 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join16.q.out 1158991 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join38.q.out 1158991 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/louter_join_ppr.q.out 1158991 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_clusterby.q.out 1158991 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_gby_join.q.out 1158991 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join.q.out 1158991 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join2.q.out 1158991 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join3.q.out 1158991 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join1.q.out 1158991 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join2.q.out 1158991 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join4.q.out 1158991 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_transitivity.q.out PRE-CREATION http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_udf_case.q.out 1158991 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/router_join_ppr.q.out 1158991 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin9.q.out 1158991 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_6.q.out 1158991 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/udf_named_struct.q.out 1158991 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/union22.q.out 1158991 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join1.q.xml 1158991 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join2.q.xml 1158991 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join3.q.xml 1158991 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample1.q.xml 1158991 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample2.q.xml 1158991 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample3.q.xml 1158991 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample4.q.xml 1158991 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample5.q.xml 1158991 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample6.q.xml 1158991 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/sample7.q.xml 1158991 http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/subq.q.xml 1158991
[jira] [Commented] (HIVE-1989) recognize transitivity of predicates on join keys
[
https://issues.apache.org/jira/browse/HIVE-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13086732#comment-13086732
]
jirapos...@reviews.apache.org commented on HIVE-1989:
-
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1228/
---
(Updated 2011-08-18 01:41:13.707664)
Review request for hive.
Changes
---
Temporarily merged HIVE-2383: Incorrect alias filtering for predicates pushdown
into this task for unit tests, fix deleted config sections
Summary
---
https://issues.apache.org/jira/browse/HIVE-1989
This addresses bug HIVE-1989.
https://issues.apache.org/jira/browse/HIVE-1989
Diffs (updated)
-
http://svn.apache.org/repos/asf/hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
1158048
http://svn.apache.org/repos/asf/hive/trunk/conf/hive-default.xml 1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/ppd_transitivity.q
PRE-CREATION
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/auto_join16.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/cluster.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_mult_tables.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_mult_tables_compact.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join16.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join38.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/louter_join_ppr.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_clusterby.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_gby_join.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join2.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join3.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join1.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join2.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join4.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_transitivity.q.out
PRE-CREATION
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_udf_case.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/router_join_ppr.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin9.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_6.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/union22.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join1.q.xml
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join2.q.xml
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join3.q.xml
1158048
Diff: https://reviews.apache.org/r/1228/diff
Testing
---
Thanks,
Charles
recognize transitivity of predicates on join keys
-
Key: HIVE-1989
URL: https://issues.apache.org/jira/browse/HIVE-1989
Project: Hive
Issue Type: Improvement
Components: Query Processor
Affects Versions: 0.6.0
Reporter: John Sichi
Assignee: Charles Chen
Attachments: HIVE-1989v1.patch, HIVE-1989v4.patch,
HIVE-1989v5-WITH-HIVE-2382v1.patch
Given
{noformat}
set hive.mapred.mode=strict;
create table invites (foo int, bar string) partitioned by (ds string);
create table invites2 (foo int, bar string) partitioned by (ds string);
select count(*) from invites join invites2 on invites.ds=invites2.ds where
invites.ds='2011-01-01';
{noformat}
currently an
[jira] [Commented] (HIVE-1989) recognize transitivity of predicates on join keys
[
https://issues.apache.org/jira/browse/HIVE-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13085415#comment-13085415
]
jirapos...@reviews.apache.org commented on HIVE-1989:
-
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1228/
---
(Updated 2011-08-15 22:57:22.762640)
Review request for hive.
Changes
---
For review
Summary
---
https://issues.apache.org/jira/browse/HIVE-1989
This addresses bug HIVE-1989.
https://issues.apache.org/jira/browse/HIVE-1989
Diffs (updated)
-
http://svn.apache.org/repos/asf/hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
1158048
http://svn.apache.org/repos/asf/hive/trunk/conf/hive-default.xml 1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/auto_join16.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/cluster.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_mult_tables.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_mult_tables_compact.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join16.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join38.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/louter_join_ppr.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_clusterby.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_gby_join.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join2.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join3.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join1.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join2.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join4.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_udf_case.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/router_join_ppr.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin9.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_6.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/union22.q.out
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join1.q.xml
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join2.q.xml
1158048
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/compiler/plan/join3.q.xml
1158048
Diff: https://reviews.apache.org/r/1228/diff
Testing
---
Thanks,
Charles
recognize transitivity of predicates on join keys
-
Key: HIVE-1989
URL: https://issues.apache.org/jira/browse/HIVE-1989
Project: Hive
Issue Type: Improvement
Components: Query Processor
Affects Versions: 0.6.0
Reporter: John Sichi
Assignee: Charles Chen
Attachments: HIVE-1989v1.patch, HIVE-1989v4.patch
Given
{noformat}
set hive.mapred.mode=strict;
create table invites (foo int, bar string) partitioned by (ds string);
create table invites2 (foo int, bar string) partitioned by (ds string);
select count(*) from invites join invites2 on invites.ds=invites2.ds where
invites.ds='2011-01-01';
{noformat}
currently an error occurs:
{noformat}
Error in semantic analysis: No Partition Predicate Found for Alias invites2
Table invites2
{noformat}
The optimizer should be able to infer a predicate on invites2 via
transitivity. The current lack places a burden on the user to add a
redundant predicate, and makes impossible (at least in strict mode) join
views where both underlying tables are partitioned
[jira] [Commented] (HIVE-1989) recognize transitivity of predicates on join keys
[
https://issues.apache.org/jira/browse/HIVE-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13083887#comment-13083887
]
John Sichi commented on HIVE-1989:
--
Usually when we add a new optimization, we add a corresponding conf parameter
so that we can disable it if it causes trouble. Add it to HiveConf.java and
conf/hive-default.xml
recognize transitivity of predicates on join keys
-
Key: HIVE-1989
URL: https://issues.apache.org/jira/browse/HIVE-1989
Project: Hive
Issue Type: Improvement
Components: Query Processor
Affects Versions: 0.6.0
Reporter: John Sichi
Assignee: Charles Chen
Attachments: HIVE-1989v1.patch
Given
{noformat}
set hive.mapred.mode=strict;
create table invites (foo int, bar string) partitioned by (ds string);
create table invites2 (foo int, bar string) partitioned by (ds string);
select count(*) from invites join invites2 on invites.ds=invites2.ds where
invites.ds='2011-01-01';
{noformat}
currently an error occurs:
{noformat}
Error in semantic analysis: No Partition Predicate Found for Alias invites2
Table invites2
{noformat}
The optimizer should be able to infer a predicate on invites2 via
transitivity. The current lack places a burden on the user to add a
redundant predicate, and makes impossible (at least in strict mode) join
views where both underlying tables are partitioned (the join select list has
to pick one of the tables arbitrarily).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
