Attila Jeges has uploaded a new change for review. http://gerrit.cloudera.org:8080/7096
Change subject: IMPALA-3548: Prune runtime filters based on query options in the FE ...................................................................... IMPALA-3548: Prune runtime filters based on query options in the FE Currently, the FE generates a number of runtime filters and assigns them to plan nodes without taking the value of RUNTIME_FILTER_MODE and DISABLE_ROW_RUNTIME_FILTERING query options into account. The backend then removes filters from exec nodes, based on the following rules: 1. If DISABLE_ROW_RUNTIME_FILTERING is set, filters are removed from the exec nodes that are marked as targets not bound by partition columns. 2. If RUNTIME_FILTER_MODE is set to LOCAL, filters are removed from the exec nodes that are marked as remote targets. This may cause some confusion to users because they may see runtime filters in the output of explain that are not applied when the query is executed. This change moves the logic of runtime filter pruning to the planner in the FE. Pruning based on the value of DISABLE_ROW_RUNTIME_FILTERING query option takes place as early as when runtime filters are assigned to the single node plan. Pruning based on RUNTIME_FILTER_MODE query option on the other hand has to wait until the distributed plan is ready (since it requires information about the locality of runtime filter targets). Change-Id: I422e48847428cab9887aee899fed47a8de95c323 --- M be/src/exec/hash-join-node.cc M be/src/exec/hdfs-scan-node-base.cc M fe/src/main/java/org/apache/impala/planner/PlanNode.java M fe/src/main/java/org/apache/impala/planner/Planner.java M fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java A testdata/workloads/functional-query/queries/QueryTest/runtime_filters_distrib_pruning.test A testdata/workloads/functional-query/queries/QueryTest/runtime_filters_singlenode_pruning.test M tests/query_test/test_runtime_filters.py 8 files changed, 665 insertions(+), 76 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/96/7096/2 -- To view, visit http://gerrit.cloudera.org:8080/7096 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I422e48847428cab9887aee899fed47a8de95c323 Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Attila Jeges <[email protected]>
