Hello Quanlong Huang, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/24451
to look at the new patch set (#3).
Change subject: IMPALA-15097: Show RF effectiveness in plan graph
......................................................................
IMPALA-15097: Show RF effectiveness in plan graph
This updates plan-graph.py to visualize runtime filter
effectiveness from scan-node runtime filter counters.
Runtime filter consumer edges are marked as dashed when the target
scan processed input but the filter rejected nothing. At higher
verbosity, the graph also shows compact rejected/total counters for
rows, row groups, files, or splits when those counters are available.
Per-instance scan-node counters are summed when present because the
graph should show total skipped input. Averaged Fragment counters are
used only as fallback for profiles without per-instance filter
counters. If no usable counters are available, runtime filter edges are
left unchanged.
The filter table parser now handles the current profile format with
the Eff. Tgt. Node(s) column while keeping compatibility with the older
format.
Testing:
- python3 -m py_compile bin/diagnostics/experimental/plan-graph.py \
tests/unittests/test_plan_graph.py
- python3 -m pytest --confcutdir=tests/unittests \
tests/unittests/test_plan_graph.py -q
- bin/diagnostics/experimental/plan-graph.py -v1 rf_profile.txt | \
dot -Tsvg -o query-plan-v1.svg
- bin/diagnostics/experimental/plan-graph.py -v2 rf_profile.txt | \
dot -Tsvg -o query-plan-v2.svg
- Smoke-tested --no-color, --no-rf, and an older profile-table format
Change-Id: Ic61ba09b9977da4545f3bfd1489e0856a7c0af40
---
M bin/diagnostics/experimental/plan-graph.py
A tests/unittests/test_plan_graph.py
2 files changed, 447 insertions(+), 10 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/51/24451/3
--
To view, visit http://gerrit.cloudera.org:8080/24451
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic61ba09b9977da4545f3bfd1489e0856a7c0af40
Gerrit-Change-Number: 24451
Gerrit-PatchSet: 3
Gerrit-Owner: Aleksandr Efimov <[email protected]>
Gerrit-Reviewer: Aleksandr Efimov <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Quanlong Huang <[email protected]>
