[
https://issues.apache.org/jira/browse/IMPALA-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16954703#comment-16954703
]
Tim Armstrong commented on IMPALA-9019:
---------------------------------------
I tracked down a bug - the runtime filters were not showing up in the TPlanNode
for the hash join in some cases. This was reliably reproducible when there was
only a single executor in the cluster.
The issue is that this code mutates the list of runtime filters, removing a
broadcast filter if it's not one of the 3 instances picked to produce it:
https://github.com/apache/impala/blob/master/be/src/runtime/coordinator-backend-state.cc#L173
. The problem is that, if you have 4 instances on a single node, one of them
is guaranteed to not be picked, so all of them get clobbered.
I think this is why I couldn't reproduce it reliably on a 3 node minicluster -
in most cases one of the replicas survived getting clobbered.
> Fix runtime filter bugs with mt_dop
> -----------------------------------
>
> Key: IMPALA-9019
> URL: https://issues.apache.org/jira/browse/IMPALA-9019
> Project: IMPALA
> Issue Type: Sub-task
> Components: Distributed Exec
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Priority: Major
>
> It looks like runtime filters do not get delivered properly. E.g. if I run
> TPC-H Q 9 with mt_dop=4 and runtime_filter_wait_time_ms=999999, the query
> appears to get stuck.
> This causes TPC-H performance to be significantly worse for some queries.
> Strangely, I can only reproduce this on TPC-H scale factor 3, not the smaller
> scale factor.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
