Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/18731 )
Change subject: IMPALA-11424: Support pushdown non-equi join predicate from OUTER/INNER JOIN to SCANNODE ...................................................................... Patch Set 17: (2 comments) http://gerrit.cloudera.org:8080/#/c/18731/11/testdata/workloads/functional-planner/queries/PlannerTest/none-equal-predicate-push-down.test File testdata/workloads/functional-planner/queries/PlannerTest/none-equal-predicate-push-down.test: http://gerrit.cloudera.org:8080/#/c/18731/11/testdata/workloads/functional-planner/queries/PlannerTest/none-equal-predicate-push-down.test@5 PS11, Line 5: alltype > Yes, you are right, it will be better. I added. About Parquet + ORC stat/dictionary filters: these predicates are only visible in the plan from explain_level=2 The Kudu predicates and partition pruning is visible at level 1. http://gerrit.cloudera.org:8080/#/c/18731/17/testdata/workloads/functional-planner/queries/PlannerTest/none-equal-predicate-push-down.test File testdata/workloads/functional-planner/queries/PlannerTest/none-equal-predicate-push-down.test: http://gerrit.cloudera.org:8080/#/c/18731/17/testdata/workloads/functional-planner/queries/PlannerTest/none-equal-predicate-push-down.test@587 PS17, Line 587: HDFS partitions=24/24 Partition pruning does not seem to work here - t2.`month` <= 2 should reduce the number of partitions. It would be great to investigate this in this patch or at least a TODO should be added to the test. AFAIK partition pruning happens here: https://github.com/apache/impala/blob/c38afdf50f652dcc454eb8f20da45224e3c9bfca/fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java#L1585 This is done before calling HashJoinNode.init(), so these new conjuncts do not exist at that point yet. I think that it would be the best to generate the new conjuncts earlier so that the scan nodes can pick them up automatically, but I am not yet sure about the right place to do this. -- To view, visit http://gerrit.cloudera.org:8080/18731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie3ce23cbd7522a209c830504f329b972d67bc263 Gerrit-Change-Number: 18731 Gerrit-PatchSet: 17 Gerrit-Owner: Baike Xia <[email protected]> Gerrit-Reviewer: Aman Sinha <[email protected]> Gerrit-Reviewer: Baike Xia <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Comment-Date: Mon, 02 Jan 2023 13:28:44 +0000 Gerrit-HasComments: Yes
