[
https://issues.apache.org/jira/browse/IMPALA-12018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17766904#comment-17766904
]
Riza Suminto commented on IMPALA-12018:
---------------------------------------
I have an idea that if a scan node receive a runtime filter from a join node,
then that join node selectivity can be applied to reduce the cardinality of the
scan node. However, this comes with 2 requirement:
# Runtime filter arrived ontime, or it is guaranteed that scan node will need
to wait for that runtime filter arrival (ie., join node right above the scan
will not start pulling rows before its join build complete).
# The runtime filter itself accurate selectivity.
The second point can be tricky given that Impala's default
RUNTIME_FILTER_ERROR_RATE == max_filter_error_rate == 0.75, and join build
cardinality itself can be underestimated, leading to undersize bloom filter
(IMPALA-12451).
> Consider runtime filters in resource estimates
> ----------------------------------------------
>
> Key: IMPALA-12018
> URL: https://issues.apache.org/jira/browse/IMPALA-12018
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
> Reporter: Csaba Ringhofer
> Assignee: Riza Suminto
> Priority: Major
>
> Currently Impala creates a plan first and looks for runtime filters bases on
> the complete plan.
> IMPALA-3573 is about considering runtime filters during join ordering which
> would be a major change. Meanwhile it could be also useful to consider
> selective looking runtime filters in resource estimates without changing the
> plan topology.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
