[
https://issues.apache.org/jira/browse/IMPALA-12018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17767048#comment-17767048
]
Csaba Ringhofer commented on IMPALA-12018:
------------------------------------------
> 1. Runtime filter arrived ontime, or it is guaranteed that scan node will
> need to wait for that runtime filter arrival (ie., join node right above the
> scan will not start pulling rows before its join build complete).
I was thinking about concept like "critical runtime filter" - we could
designate some filters as critical and wait for them indefinitely. Critical
filters could be calculated into the estimations. Meanwhile other filters would
be treated as optional and scanners nodes would have a timeout . Optional
filters would be ignored in resource estimates.
We could treat filters consumed by the probe side scanner as critical, as we
have to wait for all produces build nodes to complete anyway. Large build side
scanners could be also treated as critical if selectivity and fpp looks good.
> Consider runtime filters in resource estimates
> ----------------------------------------------
>
> Key: IMPALA-12018
> URL: https://issues.apache.org/jira/browse/IMPALA-12018
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
> Reporter: Csaba Ringhofer
> Assignee: Riza Suminto
> Priority: Major
>
> Currently Impala creates a plan first and looks for runtime filters bases on
> the complete plan.
> IMPALA-3573 is about considering runtime filters during join ordering which
> would be a major change. Meanwhile it could be also useful to consider
> selective looking runtime filters in resource estimates without changing the
> plan topology.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
