Qifan Chen has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19116 )
Change subject: IMPALA-11536: fix invalid predicates propagate for outer join simplification ...................................................................... IMPALA-11536: fix invalid predicates propagate for outer join simplification When set ENABLE_OUTER_JOIN_TO_INNER_TRANSFORMATION = true, the planner will simplify outer joins if the WHERE clause contains at least one null rejecting condition and then remove the outer-joined tuple id from the map of GlobalState#outerJoinedTupleIds. However, there may be false removals for right join simplification or full join simplification. This may lead to incorrect results since it is incorrect to propagate a non null-rejecting predicate into a plan subtree that is on the nullable side of an outer join. GlobalState#outerJoinedTupleIds indicates whether a table is on the nullable side of an outer join. E.g. SELECT COUNT(*) FROM functional.nullrows t1 FULL JOIN functional.nullrows t2 ON t1.id = t2.id FULL JOIN functional.nullrows t3 ON coalesce(t1.id, t2.id) = t3.id WHERE t1.group_str = 'a' AND coalesce(t2.group_str, 'f') = 'f' The predicate coalesce(t2.group_str, 'f') = 'f' will propagate into t2 if we remove t2 from GlobalState#outerJoinedTupleIds. Testing: - Add new plan tests in outer-to-inner-joins.test - Add new query tests to verify the correctness on transformation Change-Id: I6565c5bff0d2f24f30118ba47a2583383e83fff7 Reviewed-on: http://gerrit.cloudera.org:8080/19116 Reviewed-by: Qifan Chen <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M testdata/workloads/functional-planner/queries/PlannerTest/outer-to-inner-joins.test M testdata/workloads/functional-query/queries/QueryTest/outer-to-inner-joins.test 3 files changed, 307 insertions(+), 17 deletions(-) Approvals: Qifan Chen: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/19116 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I6565c5bff0d2f24f30118ba47a2583383e83fff7 Gerrit-Change-Number: 19116 Gerrit-PatchSet: 10 Gerrit-Owner: Xianqing He <[email protected]> Gerrit-Reviewer: Aman Sinha <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Qifan Chen <[email protected]> Gerrit-Reviewer: Xianqing He <[email protected]>
