[
https://issues.apache.org/jira/browse/IMPALA-15097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18088529#comment-18088529
]
Aleksandr Efimov commented on IMPALA-15097:
-------------------------------------------
I’ve been working with Impala profile parsing/diagnostic tooling recently, and
runtime filters are an area where we’ve hit a few practical edge cases. I’d
like to make the first patch a bit more useful than only drawing dashed lines.
Proposed initial scope:
- mark runtime-filter consumer edges as dashed when the profile shows that the
filter processed input but rejected nothing;
- show compact effectiveness counters on the graph when available, e.g.
rows/files/row-groups rejected out of processed/total;
- read the counters from the scan-node runtime filter sections, while keeping
the graph unchanged when the profile does not contain enough information;
- make sure the parser handles the current runtime filter table format,
including the effective-targets column;
- add a focused regression test with a small profile snippet, and verify the
output on the attached profile.
I’ll keep broader profile-graph improvements as follow-ups so this issue stays
focused on runtime filter effectiveness.
> Show runtime filter effectiveness in plan-graph.py
> --------------------------------------------------
>
> Key: IMPALA-15097
> URL: https://issues.apache.org/jira/browse/IMPALA-15097
> Project: IMPALA
> Issue Type: Improvement
> Reporter: Quanlong Huang
> Assignee: Aleksandr Efimov
> Priority: Major
> Attachments: query-plan1.svg, rf_profile.txt
>
>
> When verbosity is larger than 0, plan-graph.py generates nodes and lines for
> runtime filters. It'd be helpful to show ineffective runtime filters, e.g.,
> by using dash lines.
> It'd be more helpful if we should how many files/rows, etc. are skipped by
> the filter.
> Uploaded an example profile and the generated svg file. The command is
> {code:java}
> bin/diagnostics/experimental/plan-graph.py -v1 rf_profile.txt | dot -T svg -o
> query-plan1.svg {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
