[jira] [Commented] (HIVE-4020) Swap applying order of CP and PPD
[
https://issues.apache.org/jira/browse/HIVE-4020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13623074#comment-13623074
]
Hudson commented on HIVE-4020:
--
Integrated in Hive-trunk-hadoop2 #138 (See
[https://builds.apache.org/job/Hive-trunk-hadoop2/138/])
HIVE-4020: Swap applying order of CP and PPD (Navis via Ashutosh Chauhan)
(Revision 1452423)
Result = FAILURE
hashutosh :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1452423
Files :
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java
* /hive/trunk/ql/src/test/results/clientpositive/auto_join19.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join9.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin_negative.q.out
* /hive/trunk/ql/src/test/results/clientpositive/filter_join_breaktask.q.out
* /hive/trunk/ql/src/test/results/clientpositive/index_auto_mult_tables.q.out
*
/hive/trunk/ql/src/test/results/clientpositive/index_auto_mult_tables_compact.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input39_hadoop20.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join38.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join9.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join_map_ppr.q.out
* /hive/trunk/ql/src/test/results/clientpositive/lateral_view_ppd.q.out
* /hive/trunk/ql/src/test/results/clientpositive/louter_join_ppr.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd_repeated_alias.q.out
* /hive/trunk/ql/src/test/results/clientpositive/router_join_ppr.q.out
* /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin9.q.out
* /hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_3.q.out
* /hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_4.q.out
* /hive/trunk/ql/src/test/results/clientpositive/stats11.q.out
* /hive/trunk/ql/src/test/results/clientpositive/union26.q.out
* /hive/trunk/ql/src/test/results/compiler/plan/case_sensitivity.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/cast1.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input1.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input6.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input9.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input_part1.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input_testxpath2.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/join4.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/join5.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/join6.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/join7.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/join8.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/sample7.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/subq.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/udf1.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/union.q.xml
Swap applying order of CP and PPD
-
Key: HIVE-4020
URL: https://issues.apache.org/jira/browse/HIVE-4020
Project: Hive
Issue Type: Improvement
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Minor
Fix For: 0.11.0
Attachments: HIVE-4020.D8571.1.patch, HIVE-4020.D8571.2.patch
Doing Hive-2340, I've found CP removed some column mapping needed for
backtracking expression desc. By swapping order of CP and PPD, the problem
was solved.
After that I've realized that CP on earlier stage is possible after PPD is
applied cause some columns on filter predicate are not selected and can be
removed right after the new pushed-down filter. For example,
(bucketmapjoin1.q)
noformat
select /*+mapjoin(b)*/ a.key, a.value, b.value
from srcbucket_mapjoin_part a join srcbucket_mapjoin_part_2 b
on a.key=b.key where b.ds=2008-04-08
noformat
plan for hashtable sink operator is changed to
noformat
HashTable Sink Operator
condition expressions:
0 {key} {value}
1 {value}
noformat
which was
noformat
HashTable Sink Operator
condition expressions:
0 {key} {value}
1 {value} {ds}
noformat
HIVE-2340 seemed need more time for commit, so booked as an another issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4020) Swap applying order of CP and PPD
[
https://issues.apache.org/jira/browse/HIVE-4020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13593097#comment-13593097
]
Hudson commented on HIVE-4020:
--
Integrated in Hive-trunk-h0.21 #2000 (See
[https://builds.apache.org/job/Hive-trunk-h0.21/2000/])
HIVE-4020: Swap applying order of CP and PPD (Navis via Ashutosh Chauhan)
(Revision 1452423)
Result = SUCCESS
hashutosh :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1452423
Files :
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java
* /hive/trunk/ql/src/test/results/clientpositive/auto_join19.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join9.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/bucketmapjoin_negative.q.out
* /hive/trunk/ql/src/test/results/clientpositive/filter_join_breaktask.q.out
* /hive/trunk/ql/src/test/results/clientpositive/index_auto_mult_tables.q.out
*
/hive/trunk/ql/src/test/results/clientpositive/index_auto_mult_tables_compact.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input39_hadoop20.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join38.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join9.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join_map_ppr.q.out
* /hive/trunk/ql/src/test/results/clientpositive/lateral_view_ppd.q.out
* /hive/trunk/ql/src/test/results/clientpositive/louter_join_ppr.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd_repeated_alias.q.out
* /hive/trunk/ql/src/test/results/clientpositive/router_join_ppr.q.out
* /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin9.q.out
* /hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_3.q.out
* /hive/trunk/ql/src/test/results/clientpositive/sort_merge_join_desc_4.q.out
* /hive/trunk/ql/src/test/results/clientpositive/stats11.q.out
* /hive/trunk/ql/src/test/results/clientpositive/union26.q.out
* /hive/trunk/ql/src/test/results/compiler/plan/case_sensitivity.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/cast1.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input1.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input6.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input9.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input_part1.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input_testxpath2.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/join4.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/join5.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/join6.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/join7.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/join8.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/sample7.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/subq.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/udf1.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/union.q.xml
Swap applying order of CP and PPD
-
Key: HIVE-4020
URL: https://issues.apache.org/jira/browse/HIVE-4020
Project: Hive
Issue Type: Improvement
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Minor
Fix For: 0.11.0
Attachments: HIVE-4020.D8571.1.patch, HIVE-4020.D8571.2.patch
Doing Hive-2340, I've found CP removed some column mapping needed for
backtracking expression desc. By swapping order of CP and PPD, the problem
was solved.
After that I've realized that CP on earlier stage is possible after PPD is
applied cause some columns on filter predicate are not selected and can be
removed right after the new pushed-down filter. For example,
(bucketmapjoin1.q)
noformat
select /*+mapjoin(b)*/ a.key, a.value, b.value
from srcbucket_mapjoin_part a join srcbucket_mapjoin_part_2 b
on a.key=b.key where b.ds=2008-04-08
noformat
plan for hashtable sink operator is changed to
noformat
HashTable Sink Operator
condition expressions:
0 {key} {value}
1 {value}
noformat
which was
noformat
HashTable Sink Operator
condition expressions:
0 {key} {value}
1 {value} {ds}
noformat
HIVE-2340 seemed need more time for commit, so booked as an another issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4020) Swap applying order of CP and PPD
[
https://issues.apache.org/jira/browse/HIVE-4020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13591640#comment-13591640
]
Ashutosh Chauhan commented on HIVE-4020:
+1 Navis can you refresh the patch. I suspect .q.out files need to be updated
because of other recent commits. I will test and get in quickly before it goes
stale again.
Swap applying order of CP and PPD
-
Key: HIVE-4020
URL: https://issues.apache.org/jira/browse/HIVE-4020
Project: Hive
Issue Type: Improvement
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Minor
Attachments: HIVE-4020.D8571.1.patch
Doing Hive-2340, I've found CP removed some column mapping needed for
backtracking expression desc. By swapping order of CP and PPD, the problem
was solved.
After that I've realized that CP on earlier stage is possible after PPD is
applied cause some columns on filter predicate are not selected and can be
removed right after the new pushed-down filter. For example,
(bucketmapjoin1.q)
noformat
select /*+mapjoin(b)*/ a.key, a.value, b.value
from srcbucket_mapjoin_part a join srcbucket_mapjoin_part_2 b
on a.key=b.key where b.ds=2008-04-08
noformat
plan for hashtable sink operator is changed to
noformat
HashTable Sink Operator
condition expressions:
0 {key} {value}
1 {value}
noformat
which was
noformat
HashTable Sink Operator
condition expressions:
0 {key} {value}
1 {value} {ds}
noformat
HIVE-2340 seemed need more time for commit, so booked as an another issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
