Hello Jason Fehr, Surya Hebbar, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/24123
to look at the new patch set (#6).
Change subject: IMPALA-14796: Show effective runtime filter targets in profile
......................................................................
IMPALA-14796: Show effective runtime filter targets in profile
This patch adds an "Eff. Tgt. Node(s)" (Effective Target Node(s)) column
to the "Final filter table" in the query profile. This shows which scan
nodes actually had rows rejected by each runtime filter, distinguishing
filters that were effective from those that were applied but rejected no
data.
Implementation:
- Check filter effectiveness based on profiles gathered at coordinator
side. For each ScanNode, check its filter counters to see if any data
is rejected.
- In coordinator.cc, move the code of setting the "Final filter table"
from ReleaseExecResources() to ComputeQuerySummary() to ensure the
final status reports from backends all arrive.
- In FilterDebugString(), add a column to show the node ids where the
runtime filter is effective.
Additionally, removed temp_object_pool and temp_mem_tracker from
FilterDebugString() as they have been unused since commit a985e11.
Tests
- Added e2e test in test_runtime_filters.py
- Added test for the aggregated profile (gen_experimental_profile=true)
in custom_cluster/test_runtime_profile.py.
- Updated in_list_filters.test for the new column.
Generated-by: Claude Sonnet 4.5
Change-Id: Iccf4b87ac4579a70273f3306ec7b58850f06b17c
---
M be/src/runtime/coordinator.cc
M be/src/runtime/coordinator.h
M be/src/util/runtime-profile.cc
M be/src/util/runtime-profile.h
M be/src/util/string-util.cc
M be/src/util/string-util.h
M testdata/workloads/functional-query/queries/QueryTest/in_list_filters.test
A testdata/workloads/tpch/queries/effective-runtime-filter.test
M tests/custom_cluster/test_runtime_profile.py
M tests/query_test/test_runtime_filters.py
10 files changed, 234 insertions(+), 25 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/23/24123/6
--
To view, visit http://gerrit.cloudera.org:8080/24123
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iccf4b87ac4579a70273f3306ec7b58850f06b17c
Gerrit-Change-Number: 24123
Gerrit-PatchSet: 6
Gerrit-Owner: Quanlong Huang <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Jason Fehr <[email protected]>
Gerrit-Reviewer: Quanlong Huang <[email protected]>
Gerrit-Reviewer: Surya Hebbar <[email protected]>
