[
https://issues.apache.org/jira/browse/IMPALA-13660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-13660:
-----------------------------------
Description:
Enable intermediate result caching for broadcast hash joins. The cache key must
incorporate probe and build sides; since the build side will be part of the
cache key, we can rely on the broadcast results being stable.
This differs from a partition join, where we don't have the same guarantees;
partition joins would also incorporate an exchange on the probe side, so our
normal rules would preclude caching it right now.
The build side comes from another fragment, so we need to tolerate an exchange
on the build side, similar to what was done in IMPALA-13185. We also cancel the
build side fragment if a cache hit is identified on all join fragments, since
the build side results would no longer be needed.
was:
Enable intermediate result caching for broadcast hash joins. The cache key must
incorporate probe and build sides; since the build side will be part of the
cache key, we can rely on the broadcast results being stable.
This differs from a partition join, where we don't have the same guarantees;
partition joins would also incorporate an exchange on the probe side, so our
normal rules would preclude caching it right now.
The build side comes from another fragment, so we need to tolerate an exchange
on the build side, similar to what was done in IMPALA-13185.
> Cache broadcast hash joins
> --------------------------
>
> Key: IMPALA-13660
> URL: https://issues.apache.org/jira/browse/IMPALA-13660
> Project: IMPALA
> Issue Type: Improvement
> Reporter: Michael Smith
> Priority: Major
>
> Enable intermediate result caching for broadcast hash joins. The cache key
> must incorporate probe and build sides; since the build side will be part of
> the cache key, we can rely on the broadcast results being stable.
> This differs from a partition join, where we don't have the same guarantees;
> partition joins would also incorporate an exchange on the probe side, so our
> normal rules would preclude caching it right now.
> The build side comes from another fragment, so we need to tolerate an
> exchange on the build side, similar to what was done in IMPALA-13185. We also
> cancel the build side fragment if a cache hit is identified on all join
> fragments, since the build side results would no longer be needed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
