[
https://issues.apache.org/jira/browse/IMPALA-14275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18010990#comment-18010990
]
Joe McDonnell commented on IMPALA-14275:
----------------------------------------
Example from TPC-DS Q61:
{noformat}
| 24:HASH JOIN [INNER JOIN, BROADCAST]
| | hash predicates: ss_item_sk = i_item_sk
| | fk/pk conjuncts: assumed fk/pk
| | runtime filters: RF012[bloom] <- i_item_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB
thread-reservation=0
| | tuple-ids=9,10,11,12,13,14 row-size=80B cardinality=173.74M
| | in pipelines: 14(GETNEXT), 19(OPEN)
| | tuple cache key: 73afcb4d31e69e9432d3724f3711c318
| | tuple cache hash trace:
| | Child node 23: 4dffc2a2dbb3264f52e755e3cdc181ff
| | Child node 40: cf78922113804978cdcf8ac7422c4218
| | Child node 19 via runtime filter RF012:
c8507f7a30bc2c6ee78db2cc7cb523e4{noformat}
RF012 is produced by this node, but it is including it as a child. The
c8507f7a30bc2c6ee78db2cc7cb523e4 is actually pointing to a tuple cache node
below the exchange. This is fairly harmless, as that doesn't have any
information associated with it, but we should avoid it.
> Tuple cache needs to differentiate between consumed vs produced runtime
> filters
> -------------------------------------------------------------------------------
>
> Key: IMPALA-14275
> URL: https://issues.apache.org/jira/browse/IMPALA-14275
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 5.0.0
> Reporter: Joe McDonnell
> Priority: Major
>
> The tuple cache needs to include information about consumed runtime filters
> (e.g. a scan node receiving a runtime filter) to have a correct cache key.
> The code is currently considering all runtime filters whether they are
> produced or consumed. A hash join that produces a runtime filter will include
> that in the hash. We should only include this for consumed runtime filters.
> This only applies to mt_dop=0, because with mt_dop > 0 the runtime filter is
> produced by the join builder fragment, which doesn't enter into the
> calculations.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
