[ https://issues.apache.org/jira/browse/SPARK-9968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-9968: ----------------------------------- Assignee: Tathagata Das (was: Apache Spark) > BlockGenerator lock structure can cause lock starvation of the block updating > thread > ------------------------------------------------------------------------------------ > > Key: SPARK-9968 > URL: https://issues.apache.org/jira/browse/SPARK-9968 > Project: Spark > Issue Type: Sub-task > Components: Streaming > Reporter: Tathagata Das > Assignee: Tathagata Das > > When the rate limiter is actually limiting the rate at which data is inserted > into the buffer, the synchronized block of BlockGenerator.addData stays > blocked for long time. This causes the thread switching the buffer and > generating blocks (synchronized with addData) to starve and not generate > blocks for seconds. The correct solution is to not block on the rate limiter > within the synchronized block for adding data to the buffer. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org