Tim Armstrong created IMPALA-6920:
-------------------------------------
Summary: Multithreaded scans are not guaranteed to get a thread
token immediately
Key: IMPALA-6920
URL: https://issues.apache.org/jira/browse/IMPALA-6920
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 2.12.0
Reporter: Tim Armstrong
Assignee: Tim Armstrong
So what happens is that we reserve an optional token for the first scanner
thread but that can be taken by any other operator in the same fragment. What
happens in one fragment in TPC-DS q18a is:
1. The hash join grabs an extra token for the join build. I guess it does this
early so it gets an optional token before other fragments can grab them.
2. The scan node reserves an optional token in Open(). This optional token is
already in use by the hash join.
3. The scan node tries to start the first scanner thread, but there are no
optional tokens available, so it can't start any.
4. Eventually the optional token is given up and the scanner thread can start.
If #4 always happens without the scan making progress, then no deadlock is
possible, but if there's any kind of circular dependency, this can deadlock.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
