[
https://issues.apache.org/jira/browse/IMPALA-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16956697#comment-16956697
]
ASF subversion and git services commented on IMPALA-9019:
---------------------------------------------------------
Commit 9549d3859ed502b109abb916be818e0909e9e207 in impala's branch
refs/heads/master from Tim Armstrong
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=9549d38 ]
IMPALA-9019: fix runtime filter propagation with mt_dop
The bug was that filter routing table construction removed
filters from the TPlanNode structure for the join when
a finstance was not a producer of that filter. The
TPlanNode is shared between all instances of a fragment
on a backend, so this meant that the filter was removed
for all instances on that backend, often meaning that
no filters would be produced at all.
It was awkward fixing the bug within the framework of
the current data structures, where the routing table
is keyed by filter_id, so I ended up refactoring
the routing table somewhat. This also allowed
fixing a TODO about O(n^2) construction of the
routing table.
Testing:
Add regression test that timed out without fix.
Perf:
Ran a single node TPC-H workload with scale factor
30. No perf change.
Change-Id: I26e3628a982d5d9b8b24eb96b28aff11f8aa6669
Reviewed-on: http://gerrit.cloudera.org:8080/14511
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Fix runtime filter bugs with mt_dop
> -----------------------------------
>
> Key: IMPALA-9019
> URL: https://issues.apache.org/jira/browse/IMPALA-9019
> Project: IMPALA
> Issue Type: Sub-task
> Components: Distributed Exec
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Priority: Major
>
> It looks like runtime filters do not get delivered properly. E.g. if I run
> TPC-H Q 9 with mt_dop=4 and runtime_filter_wait_time_ms=999999, the query
> appears to get stuck.
> This causes TPC-H performance to be significantly worse for some queries.
> Strangely, I can only reproduce this on TPC-H scale factor 3, not the smaller
> scale factor.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
