[Impala-ASF-CR] IMPALA-11280: Join node incorrectly picks up unnest(array) predicates
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18614 ) Change subject: IMPALA-11280: Join node incorrectly picks up unnest(array) predicates .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/18614 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I89fed4eef220ca513b259f0e2649cdfbe43c797a Gerrit-Change-Number: 18614 Gerrit-PatchSet: 3 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 14 Jun 2022 15:37:52 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11280: Join node incorrectly picks up unnest(array) predicates
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/18614 ) Change subject: IMPALA-11280: Join node incorrectly picks up unnest(array) predicates .. IMPALA-11280: Join node incorrectly picks up unnest(array) predicates The expectation for predicates on unnested arrays is that they are either picked up by the SCAN node or the UNNEST node for evaluation. If there is only one array being unnested then the SCAN node, otherwise the UNNEST node will be responsible for the evaluation. However, if there is a JOIN node involved where the JOIN construction happens before creating the UNNEST node then the JOIN node incorrectly picks up the predicates for the unnested arrays as well. This patch is to fix this behaviour. Tests: - Added E2E tests to cover result correctness. - Added planner tests to verify that the desired node picks up the predicates for unnested arrays. Change-Id: I89fed4eef220ca513b259f0e2649cdfbe43c797a Reviewed-on: http://gerrit.cloudera.org:8080/18614 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M fe/src/main/java/org/apache/impala/planner/PlanNode.java M fe/src/main/java/org/apache/impala/planner/ScanNode.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M fe/src/main/java/org/apache/impala/planner/SingularRowSrcNode.java M fe/src/main/java/org/apache/impala/planner/UnnestNode.java M testdata/workloads/functional-planner/queries/PlannerTest/zipping-unnest.test M testdata/workloads/functional-query/queries/QueryTest/nested-array-in-select-list.test 7 files changed, 131 insertions(+), 6 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/18614 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I89fed4eef220ca513b259f0e2649cdfbe43c797a Gerrit-Change-Number: 18614 Gerrit-PatchSet: 4 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-11280: Join node incorrectly picks up unnest(array) predicates
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18614 ) Change subject: IMPALA-11280: Join node incorrectly picks up unnest(array) predicates .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8232/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/18614 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I89fed4eef220ca513b259f0e2649cdfbe43c797a Gerrit-Change-Number: 18614 Gerrit-PatchSet: 3 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 14 Jun 2022 10:54:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11280: Join node incorrectly picks up unnest(array) predicates
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18614 ) Change subject: IMPALA-11280: Join node incorrectly picks up unnest(array) predicates .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/18614 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I89fed4eef220ca513b259f0e2649cdfbe43c797a Gerrit-Change-Number: 18614 Gerrit-PatchSet: 3 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 14 Jun 2022 10:54:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11280: Join node incorrectly picks up unnest(array) predicates
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/18614 ) Change subject: IMPALA-11280: Join node incorrectly picks up unnest(array) predicates .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/18614 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I89fed4eef220ca513b259f0e2649cdfbe43c797a Gerrit-Change-Number: 18614 Gerrit-PatchSet: 2 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 14 Jun 2022 10:21:33 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11280: Join node incorrectly picks up unnest(array) predicates
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/18614 ) Change subject: IMPALA-11280: Join node incorrectly picks up unnest(array) predicates .. Patch Set 2: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/18614 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I89fed4eef220ca513b259f0e2649cdfbe43c797a Gerrit-Change-Number: 18614 Gerrit-PatchSet: 2 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 14 Jun 2022 10:21:22 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11280: Join node incorrectly picks up unnest(array) predicates
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18614 ) Change subject: IMPALA-11280: Join node incorrectly picks up unnest(array) predicates .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/10766/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/18614 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I89fed4eef220ca513b259f0e2649cdfbe43c797a Gerrit-Change-Number: 18614 Gerrit-PatchSet: 2 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 14 Jun 2022 09:07:00 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11280: Join node incorrectly picks up unnest(array) predicates
Gabor Kaszab has posted comments on this change. ( http://gerrit.cloudera.org:8080/18614 ) Change subject: IMPALA-11280: Join node incorrectly picks up unnest(array) predicates .. Patch Set 2: (2 comments) http://gerrit.cloudera.org:8080/#/c/18614/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/18614/1//COMMIT_MSG@12 PS1, Line 12: However, if : there is a JOIN node involved > In the test this comes from rewriting an IN clause to a join - can this hap This is reproducible by explicitly writing joins in the query. Added test coverage for that. http://gerrit.cloudera.org:8080/#/c/18614/1//COMMIT_MSG@14 PS1, Line 14: creating > typo Done -- To view, visit http://gerrit.cloudera.org:8080/18614 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I89fed4eef220ca513b259f0e2649cdfbe43c797a Gerrit-Change-Number: 18614 Gerrit-PatchSet: 2 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 14 Jun 2022 08:47:58 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11280: Join node incorrectly picks up unnest(array) predicates
Hello Csaba Ringhofer, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/18614 to look at the new patch set (#2). Change subject: IMPALA-11280: Join node incorrectly picks up unnest(array) predicates .. IMPALA-11280: Join node incorrectly picks up unnest(array) predicates The expectation for predicates on unnested arrays is that they are either picked up by the SCAN node or the UNNEST node for evaluation. If there is only one array being unnested then the SCAN node, otherwise the UNNEST node will be responsible for the evaluation. However, if there is a JOIN node involved where the JOIN construction happens before creating the UNNEST node then the JOIN node incorrectly picks up the predicates for the unnested arrays as well. This patch is to fix this behaviour. Tests: - Added E2E tests to cover result correctness. - Added planner tests to verify that the desired node picks up the predicates for unnested arrays. Change-Id: I89fed4eef220ca513b259f0e2649cdfbe43c797a --- M fe/src/main/java/org/apache/impala/planner/PlanNode.java M fe/src/main/java/org/apache/impala/planner/ScanNode.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M fe/src/main/java/org/apache/impala/planner/SingularRowSrcNode.java M fe/src/main/java/org/apache/impala/planner/UnnestNode.java M testdata/workloads/functional-planner/queries/PlannerTest/zipping-unnest.test M testdata/workloads/functional-query/queries/QueryTest/nested-array-in-select-list.test 7 files changed, 131 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/18614/2 -- To view, visit http://gerrit.cloudera.org:8080/18614 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I89fed4eef220ca513b259f0e2649cdfbe43c797a Gerrit-Change-Number: 18614 Gerrit-PatchSet: 2 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-11280: Join node incorrectly picks up unnest(array) predicates
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/18614 ) Change subject: IMPALA-11280: Join node incorrectly picks up unnest(array) predicates .. Patch Set 1: Code-Review+1 (2 comments) http://gerrit.cloudera.org:8080/#/c/18614/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/18614/1//COMMIT_MSG@12 PS1, Line 12: However, if : there is a JOIN node involved In the test this comes from rewriting an IN clause to a join - can this happen simply with joins, or only during statement rewrites? http://gerrit.cloudera.org:8080/#/c/18614/1//COMMIT_MSG@14 PS1, Line 14: creatiing typo -- To view, visit http://gerrit.cloudera.org:8080/18614 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I89fed4eef220ca513b259f0e2649cdfbe43c797a Gerrit-Change-Number: 18614 Gerrit-PatchSet: 1 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 13 Jun 2022 13:35:59 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11280: Join node incorrectly picks up unnest(array) predicates
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18614 ) Change subject: IMPALA-11280: Join node incorrectly picks up unnest(array) predicates .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/10759/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/18614 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I89fed4eef220ca513b259f0e2649cdfbe43c797a Gerrit-Change-Number: 18614 Gerrit-PatchSet: 1 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 13 Jun 2022 13:31:25 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11280: Join node incorrectly picks up unnest(array) predicates
Gabor Kaszab has uploaded this change for review. ( http://gerrit.cloudera.org:8080/18614 Change subject: IMPALA-11280: Join node incorrectly picks up unnest(array) predicates .. IMPALA-11280: Join node incorrectly picks up unnest(array) predicates The expectation for predicates on unnested arrays is that they are either picked up by the SCAN node or the UNNEST node for evaluation. If there is only one array being unnested then the SCAN node, otherwise the UNNEST node will be responsible for the evaluation. However, if there is a JOIN node involved where the JOIN construction happens before creatiing the UNNEST node then the JOIN node incorrectly picks up the predicates for the unnested arrays as well. This patch is to fix this behaviour. Tests: - Added an E2E test to cover result correctness. - Added a planner test to verify that the desired node picks up the predicates for unnested arrays. Change-Id: I89fed4eef220ca513b259f0e2649cdfbe43c797a --- M fe/src/main/java/org/apache/impala/planner/PlanNode.java M fe/src/main/java/org/apache/impala/planner/ScanNode.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M fe/src/main/java/org/apache/impala/planner/SingularRowSrcNode.java M fe/src/main/java/org/apache/impala/planner/UnnestNode.java M testdata/workloads/functional-planner/queries/PlannerTest/zipping-unnest.test M testdata/workloads/functional-query/queries/QueryTest/nested-array-in-select-list.test 7 files changed, 73 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/18614/1 -- To view, visit http://gerrit.cloudera.org:8080/18614 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I89fed4eef220ca513b259f0e2649cdfbe43c797a Gerrit-Change-Number: 18614 Gerrit-PatchSet: 1 Gerrit-Owner: Gabor Kaszab
