[ https://issues.apache.org/jira/browse/SPARK-10072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-10072: ------------------------------------ Assignee: Apache Spark (was: Tathagata Das) > BlockGenerator can deadlock when the queue block queue of generate blocks > fills up to capacity > ---------------------------------------------------------------------------------------------- > > Key: SPARK-10072 > URL: https://issues.apache.org/jira/browse/SPARK-10072 > Project: Spark > Issue Type: Bug > Components: Streaming > Reporter: Tathagata Das > Assignee: Apache Spark > Priority: Blocker > > Generated blocks are inserted into an ArrayBlockingQueue, and another thread > pulls stuff from the ArrayBlockingQueue and pushes it into BlockManager. Now > if that queue fills up to capacity (default is 10 blocks), then the inserting > into queue (done in the function updateCurrentBuffer) get blocked inside a > synchronized block. However, the thread that is pulling blocks from the queue > uses the same lock to check the current (active or stopped) while pulling > from the queue. Since the block generating threads is blocked (as the queue > is full) on the lock, this thread that is supposed to drain the queue gets > blocked. Ergo, deadlock. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org