[
https://issues.apache.org/jira/browse/IMPALA-9154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16980610#comment-16980610
]
ASF subversion and git services commented on IMPALA-9154:
---------------------------------------------------------
Commit e716e76cccf59c2780571429b1b945d6bbc61b8d in impala's branch
refs/heads/master from Fang-Yu Rao
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=e716e76 ]
IMPALA-9154: Revert "IMPALA-7984: Port runtime filter from Thrift RPC to KRPC"
The previous patch porting runtime filter from Thrift RPC to KRPC
introduces a deadlock if there are a very limited number of threads on
the Impala cluster.
Specifically, in that patch a Coordinator used a synchronous KRPC to
propagate an aggregated filter to other hosts. A deadlock would happen
if there is no thread available on the receiving side to answer that
KRPC especially the calling and receiving threads are called from the
same thread pool. One possible way to address this issue is to make
the call of propagating a runtime filter asynchronous to free the
calling thread. Before resolving this issue, we revert this patch for
now.
This reverts commit ec11c18884988e838a8838e1e8ecc37461e1a138.
Change-Id: I32371a515fb607da396914502da8c7fb071406bc
Reviewed-on: http://gerrit.cloudera.org:8080/14780
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> KRPC DataStreamService threads blocked in PublishFilter
> -------------------------------------------------------
>
> Key: IMPALA-9154
> URL: https://issues.apache.org/jira/browse/IMPALA-9154
> Project: IMPALA
> Issue Type: Bug
> Components: Distributed Exec
> Affects Versions: Impala 3.4.0
> Reporter: Tim Armstrong
> Assignee: Fang-Yu Rao
> Priority: Blocker
> Labels: hang
> Attachments: image-2019-11-13-08-30-27-178.png, pstack-exchange.txt
>
>
> I hit this on primitive_many_fragments when doing a single node perf run:
> {noformat}
> ./bin/single_node_perf_run.py --num_impalads=1 --scale=30 --ninja
> --workloads=targeted-perf --iterations=5
> {noformat}tan
> I noticed that the query was hung and the execution threads were hung sending
> row batches. Then looking at the RPCz page, all of the threads were busy:
> !image-2019-11-13-08-30-27-178.png!
> Multiple threads were stuck in UpdateFilter() - see [^pstack-exchange.txt].
> It looks like this is a deadlock bug because a KRPC thread is blocked waiting
> for an RPC that needs to be served by one of the limited threads from that
> same thread pool
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
