Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16622 )
Change subject: IMPALA-10252: fix invalid runtime filters for outer joins ...................................................................... IMPALA-10252: fix invalid runtime filters for outer joins The planner generates runtime filters for non-join conjuncts assigned to LEFT OUTER and FULL OUTER JOIN nodes. This is correct in many cases where NULLs stemming from unmatched rows would result in the predicate evaluating to false. E.g. x = y is always false if y is NULL. However, it is incorrect if the NULL returned from the unmatched row can result in the predicate evaluating to true. E.g. x = isnull(y, 1) can return true even if y is NULL. The fix is to detect cases when the source expression from the left input of the join returns non-NULL for null inputs and then skip generating the filter. Examples of expressions that may be affected by this change are COALESCE and ISNULL. Testing: Added regression tests: * Planner tests for LEFT OUTER and FULL OUTER where the runtime filter was incorrectly generated before this patch. * Enabled end-to-end test that was previously failing. * Added a new runtime filter test that will execute on both Parquet and Kudu (which are subtly different because of nullability of slots). Ran exhaustive tests. Change-Id: I507af1cc8df15bca21e0d8555019997812087261 Reviewed-on: http://gerrit.cloudera.org:8080/16622 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java M testdata/workloads/functional-planner/queries/PlannerTest/runtime-filter-propagation.test M testdata/workloads/functional-query/queries/QueryTest/runtime_filters.test M testdata/workloads/functional-query/queries/QueryTest/subquery.test M tests/query_test/test_queries.py 6 files changed, 144 insertions(+), 9 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/16622 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I507af1cc8df15bca21e0d8555019997812087261 Gerrit-Change-Number: 16622 Gerrit-PatchSet: 8 Gerrit-Owner: Tim Armstrong <[email protected]> Gerrit-Reviewer: Aman Sinha <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Qifan Chen <[email protected]> Gerrit-Reviewer: Shant Hovsepian <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]>
