[jira] [Updated] (HIVE-3464) Merging join tree may reorder joins which could be invalid
[ https://issues.apache.org/jira/browse/HIVE-3464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-3464: --- Resolution: Fixed Fix Version/s: 0.11.0 Status: Resolved (was: Patch Available) Committed to trunk. Thanks, Navis! > Merging join tree may reorder joins which could be invalid > -- > > Key: HIVE-3464 > URL: https://issues.apache.org/jira/browse/HIVE-3464 > Project: Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.10.0 >Reporter: Navis >Assignee: Navis > Fix For: 0.11.0 > > Attachments: HIVE-3464.D5409.2.patch, HIVE-3464.D5409.3.patch, > HIVE-3464.D5409.4.patch, HIVE-3464.D5409.5.patch, HIVE-3464.D5409.6.patch > > > Currently, hive merges join tree from right to left regardless of join types, > which may introduce join reordering. For example, > select * from a join a b on a.key=b.key join a c on b.key=c.key join a d on > a.key=d.key; > Hive tries to merge join tree in a-d=b-d, a-d=a-b, b-c=a-b order and a-d=a-b > and b-c=a-b will be merged. Final join tree is "a-(bdc)". > With this, ab-d join will be executed prior to ab-c. But if join type of -c > and -d is different, this is not valid. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3464) Merging join tree may reorder joins which could be invalid
[ https://issues.apache.org/jira/browse/HIVE-3464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-3464: -- Attachment: HIVE-3464.D5409.6.patch navis updated the revision "HIVE-3464 [jira] Merging join tree may reorder joins which could be invalid". Addressed comments Reviewers: JIRA REVISION DETAIL https://reviews.facebook.net/D5409 CHANGE SINCE LAST DIFF https://reviews.facebook.net/D5409?vs=30597&id=30849#toc AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java ql/src/test/queries/clientpositive/mergejoins_mixed.q ql/src/test/queries/clientpositive/smb_mapjoin_17.q ql/src/test/results/clientpositive/join_filters_overlap.q.out ql/src/test/results/clientpositive/mergejoins_mixed.q.out ql/src/test/results/clientpositive/smb_mapjoin_17.q.out To: JIRA, navis Cc: njain, vikram > Merging join tree may reorder joins which could be invalid > -- > > Key: HIVE-3464 > URL: https://issues.apache.org/jira/browse/HIVE-3464 > Project: Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.10.0 >Reporter: Navis >Assignee: Navis > Attachments: HIVE-3464.D5409.2.patch, HIVE-3464.D5409.3.patch, > HIVE-3464.D5409.4.patch, HIVE-3464.D5409.5.patch, HIVE-3464.D5409.6.patch > > > Currently, hive merges join tree from right to left regardless of join types, > which may introduce join reordering. For example, > select * from a join a b on a.key=b.key join a c on b.key=c.key join a d on > a.key=d.key; > Hive tries to merge join tree in a-d=b-d, a-d=a-b, b-c=a-b order and a-d=a-b > and b-c=a-b will be merged. Final join tree is "a-(bdc)". > With this, ab-d join will be executed prior to ab-c. But if join type of -c > and -d is different, this is not valid. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3464) Merging join tree may reorder joins which could be invalid
[ https://issues.apache.org/jira/browse/HIVE-3464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-3464: Status: Patch Available (was: Open) > Merging join tree may reorder joins which could be invalid > -- > > Key: HIVE-3464 > URL: https://issues.apache.org/jira/browse/HIVE-3464 > Project: Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.10.0 >Reporter: Navis >Assignee: Navis > Attachments: HIVE-3464.D5409.2.patch, HIVE-3464.D5409.3.patch, > HIVE-3464.D5409.4.patch, HIVE-3464.D5409.5.patch, HIVE-3464.D5409.6.patch > > > Currently, hive merges join tree from right to left regardless of join types, > which may introduce join reordering. For example, > select * from a join a b on a.key=b.key join a c on b.key=c.key join a d on > a.key=d.key; > Hive tries to merge join tree in a-d=b-d, a-d=a-b, b-c=a-b order and a-d=a-b > and b-c=a-b will be merged. Final join tree is "a-(bdc)". > With this, ab-d join will be executed prior to ab-c. But if join type of -c > and -d is different, this is not valid. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3464) Merging join tree may reorder joins which could be invalid
[ https://issues.apache.org/jira/browse/HIVE-3464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-3464: -- Attachment: HIVE-3464.D5409.5.patch navis updated the revision "HIVE-3464 [jira] Merging join tree may reorder joins which could be invalid". 1. Use largest one as big table if all tables are smaller than threshold 2. Fix bug on max aliases for outer join 3. Addressed comments & rebased to trunk Reviewers: JIRA REVISION DETAIL https://reviews.facebook.net/D5409 CHANGE SINCE LAST DIFF https://reviews.facebook.net/D5409?vs=30549&id=30597#toc AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/CommonJoinResolver.java ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java ql/src/test/queries/clientpositive/mergejoins_mixed.q ql/src/test/queries/clientpositive/smb_mapjoin_17.q ql/src/test/results/clientpositive/join_filters_overlap.q.out ql/src/test/results/clientpositive/mergejoins_mixed.q.out ql/src/test/results/clientpositive/smb_mapjoin_17.q.out To: JIRA, navis Cc: njain, vikram > Merging join tree may reorder joins which could be invalid > -- > > Key: HIVE-3464 > URL: https://issues.apache.org/jira/browse/HIVE-3464 > Project: Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.10.0 >Reporter: Navis >Assignee: Navis > Attachments: HIVE-3464.D5409.2.patch, HIVE-3464.D5409.3.patch, > HIVE-3464.D5409.4.patch, HIVE-3464.D5409.5.patch > > > Currently, hive merges join tree from right to left regardless of join types, > which may introduce join reordering. For example, > select * from a join a b on a.key=b.key join a c on b.key=c.key join a d on > a.key=d.key; > Hive tries to merge join tree in a-d=b-d, a-d=a-b, b-c=a-b order and a-d=a-b > and b-c=a-b will be merged. Final join tree is "a-(bdc)". > With this, ab-d join will be executed prior to ab-c. But if join type of -c > and -d is different, this is not valid. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3464) Merging join tree may reorder joins which could be invalid
[ https://issues.apache.org/jira/browse/HIVE-3464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-3464: -- Attachment: HIVE-3464.D5409.4.patch navis updated the revision "HIVE-3464 [jira] Merging join tree may reorder joins which could be invalid". Rebased to trunk Reviewers: JIRA REVISION DETAIL https://reviews.facebook.net/D5409 CHANGE SINCE LAST DIFF https://reviews.facebook.net/D5409?vs=23079&id=30549#toc AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java ql/src/test/queries/clientpositive/mergejoins_mixed.q ql/src/test/results/clientpositive/join_filters_overlap.q.out ql/src/test/results/clientpositive/mergejoins_mixed.q.out To: JIRA, navis Cc: njain > Merging join tree may reorder joins which could be invalid > -- > > Key: HIVE-3464 > URL: https://issues.apache.org/jira/browse/HIVE-3464 > Project: Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.10.0 >Reporter: Navis >Assignee: Navis > Attachments: HIVE-3464.D5409.2.patch, HIVE-3464.D5409.3.patch, > HIVE-3464.D5409.4.patch > > > Currently, hive merges join tree from right to left regardless of join types, > which may introduce join reordering. For example, > select * from a join a b on a.key=b.key join a c on b.key=c.key join a d on > a.key=d.key; > Hive tries to merge join tree in a-d=b-d, a-d=a-b, b-c=a-b order and a-d=a-b > and b-c=a-b will be merged. Final join tree is "a-(bdc)". > With this, ab-d join will be executed prior to ab-c. But if join type of -c > and -d is different, this is not valid. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3464) Merging join tree may reorder joins which could be invalid
[ https://issues.apache.org/jira/browse/HIVE-3464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3464: - Status: Open (was: Patch Available) Questions/Comments on phabricator > Merging join tree may reorder joins which could be invalid > -- > > Key: HIVE-3464 > URL: https://issues.apache.org/jira/browse/HIVE-3464 > Project: Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.10.0 >Reporter: Navis >Assignee: Navis > Attachments: HIVE-3464.D5409.2.patch, HIVE-3464.D5409.3.patch > > > Currently, hive merges join tree from right to left regardless of join types, > which may introduce join reordering. For example, > select * from a join a b on a.key=b.key join a c on b.key=c.key join a d on > a.key=d.key; > Hive tries to merge join tree in a-d=b-d, a-d=a-b, b-c=a-b order and a-d=a-b > and b-c=a-b will be merged. Final join tree is "a-(bdc)". > With this, ab-d join will be executed prior to ab-c. But if join type of -c > and -d is different, this is not valid. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3464) Merging join tree may reorder joins which could be invalid
[ https://issues.apache.org/jira/browse/HIVE-3464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-3464: -- Attachment: HIVE-3464.D5409.3.patch navis updated the revision "HIVE-3464 [jira] Merging join tree may reorder joins which could be invalid". Reviewers: JIRA Fixed test result of recently added tests Added comments, rebased on trunk REVISION DETAIL https://reviews.facebook.net/D5409 AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java ql/src/test/queries/clientpositive/mergejoins_mixed.q ql/src/test/results/clientpositive/join_filters_overlap.q.out ql/src/test/results/clientpositive/mergejoins_mixed.q.out To: JIRA, navis > Merging join tree may reorder joins which could be invalid > -- > > Key: HIVE-3464 > URL: https://issues.apache.org/jira/browse/HIVE-3464 > Project: Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.10.0 >Reporter: Navis >Assignee: Navis > Attachments: HIVE-3464.D5409.2.patch, HIVE-3464.D5409.3.patch > > > Currently, hive merges join tree from right to left regardless of join types, > which may introduce join reordering. For example, > select * from a join a b on a.key=b.key join a c on b.key=c.key join a d on > a.key=d.key; > Hive tries to merge join tree in a-d=b-d, a-d=a-b, b-c=a-b order and a-d=a-b > and b-c=a-b will be merged. Final join tree is "a-(bdc)". > With this, ab-d join will be executed prior to ab-c. But if join type of -c > and -d is different, this is not valid. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3464) Merging join tree may reorder joins which could be invalid
[ https://issues.apache.org/jira/browse/HIVE-3464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-3464: Status: Patch Available (was: Open) Running test > Merging join tree may reorder joins which could be invalid > -- > > Key: HIVE-3464 > URL: https://issues.apache.org/jira/browse/HIVE-3464 > Project: Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.10.0 >Reporter: Navis >Assignee: Navis > Attachments: HIVE-3464.D5409.2.patch > > > Currently, hive merges join tree from right to left regardless of join types, > which may introduce join reordering. For example, > select * from a join a b on a.key=b.key join a c on b.key=c.key join a d on > a.key=d.key; > Hive tries to merge join tree in a-d=b-d, a-d=a-b, b-c=a-b order and a-d=a-b > and b-c=a-b will be merged. Final join tree is "a-(bdc)". > With this, ab-d join will be executed prior to ab-c. But if join type of -c > and -d is different, this is not valid. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3464) Merging join tree may reorder joins which could be invalid
[ https://issues.apache.org/jira/browse/HIVE-3464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-3464: -- Attachment: HIVE-3464.D5409.2.patch navis updated the revision "HIVE-3464 [jira] Merging join tree may reorder joins which could be invalid". Reviewers: JIRA Rebase on trunk REVISION DETAIL https://reviews.facebook.net/D5409 AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java ql/src/test/queries/clientpositive/mergejoins_mixed.q ql/src/test/results/clientpositive/mergejoins_mixed.q.out To: JIRA, navis > Merging join tree may reorder joins which could be invalid > -- > > Key: HIVE-3464 > URL: https://issues.apache.org/jira/browse/HIVE-3464 > Project: Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.10.0 >Reporter: Navis >Assignee: Navis > Attachments: HIVE-3464.D5409.2.patch > > > Currently, hive merges join tree from right to left regardless of join types, > which may introduce join reordering. For example, > select * from a join a b on a.key=b.key join a c on b.key=c.key join a d on > a.key=d.key; > Hive tries to merge join tree in a-d=b-d, a-d=a-b, b-c=a-b order and a-d=a-b > and b-c=a-b will be merged. Final join tree is "a-(bdc)". > With this, ab-d join will be executed prior to ab-c. But if join type of -c > and -d is different, this is not valid. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
