GitHub user tdas opened a pull request:
https://github.com/apache/spark/pull/8204
[SPARK-9968][STREAMING] Reduced time spent within synchronized block to
prevent lock starvation
When the rate limiter is actually limiting the rate at which data is
inserted into the buffer, the synchronized block of BlockGenerator.addData
stays blocked for long time. This causes the thread switching the buffer and
generating blocks (synchronized with addData) to starve and not generate blocks
for seconds. The correct solution is to not block on the rate limiter within
the synchronized block for adding data to the buffer.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/tdas/spark SPARK-9968
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/8204.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #8204
----
commit a73b64567dfa3d6646d90742248914ded3bf6b2f
Author: Tathagata Das <[email protected]>
Date: 2015-08-14T13:21:46Z
Reduced time spent within synchronized block
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
