[
https://issues.apache.org/jira/browse/IMPALA-13660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18080479#comment-18080479
]
ASF subversion and git services commented on IMPALA-13660:
----------------------------------------------------------
Commit 53f5d74b9c905aad36c54f9251e0d25fc21d80bc in impala's branch
refs/heads/master from Joe McDonnell
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=53f5d74b9 ]
IMPALA-14951: Fix hang during Iceberg delete with tuple cache
With mt_dop, IcebergDeleteNode has the same independent
builder as PartitionedHashJoinNode. It needs the same logic
used for IMPALA-13660 to notify the build side when a probe
side thread closes before probing. Interestingly enough,
this is hard to demonstrate with a select. The coordinator
cancels the query when it receives all the rows, which posts
the builder out of its wait. However, for a delete, this is
not true, so it can hang indefinitely.
This centralizes the necessary logic to share it between
PartitionedHashJoinNode, NestedLoopJoinNode, and
IcebergDeleteNode. Applying it to IcebergDeleteNode fixes
the hang.
Testing:
- Added a test case that consistently reproduced the hang
before the fix
Change-Id: Iff9228446f69ce43ed303c96893a91b99474800d
Reviewed-on: http://gerrit.cloudera.org:8080/24279
Reviewed-by: Yida Wu <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Cache broadcast hash joins
> --------------------------
>
> Key: IMPALA-13660
> URL: https://issues.apache.org/jira/browse/IMPALA-13660
> Project: IMPALA
> Issue Type: Improvement
> Reporter: Michael Smith
> Assignee: Joe McDonnell
> Priority: Major
> Fix For: Impala 5.0.0
>
>
> Enable intermediate result caching for broadcast hash joins. The cache key
> must incorporate probe and build sides; since the build side will be part of
> the cache key, we can rely on the broadcast results being stable.
> This differs from a partition join, where we don't have the same guarantees;
> partition joins would also incorporate an exchange on the probe side, so our
> normal rules would preclude caching it right now.
> The build side comes from another fragment, so we need to tolerate an
> exchange on the build side, similar to what was done in IMPALA-13185.
> We need to make sure cached results are returned immediately when available
> rather than waiting on a build side we don't need; also make sure the build
> side can complete even if its results are not needed by any joins (because
> they all had cache hits). It would be nice to cancel the build side fragment
> if a cache hit is identified on all join fragments, since the build side
> results would no longer be needed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
