Tathagata Das created SPARK-10072:
-------------------------------------
Summary: BlockGenerator can deadlock when the queue block queue of
generate blocks fills up to capacity
Key: SPARK-10072
URL: https://issues.apache.org/jira/browse/SPARK-10072
Project: Spark
Issue Type: Bug
Components: Streaming
Reporter: Tathagata Das
Assignee: Tathagata Das
Generated blocks are inserted into an ArrayBlockingQueue, and another thread
pulls stuff from the ArrayBlockingQueue and pushes it into BlockManager. Now if
that queue fills up to capacity (default is 10 blocks), then the inserting into
queue (done in the function updateCurrentBuffer) get blocked inside a
synchronized block. However, the thread that is pulling blocks from the queue
uses the same lock to check the current (active or stopped) while pulling from
the queue. Since the block generating threads is blocked (as the queue is full)
on the lock, this thread that is supposed to drain the queue gets blocked.
Ergo, deadlock.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
