GitHub user JoshRosen opened a pull request:
https://github.com/apache/spark/pull/9822
[SPARK-11649] Properly set Akka frame size in SparkListenerSuite test
SparkListenerSuite's "onTaskGettingResult() called when result fetched
remotely" test was extremely slow (1 to 4 minutes to run) and recently became
extremely flaky, frequently failing with OutOfMemoryError.
The root cause was the fact that this was using `System.setProperty` to set
the Akka frame size, which was not actually modifying the frame size. As a
result, this test would allocate much more data than necessary. The fix here is
to simply use SparkConf in order to configure the frame size.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/JoshRosen/spark SPARK-11649
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/9822.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #9822
----
commit 1ae1391eb216bae03aebf91607bee8ad9327320d
Author: Josh Rosen <[email protected]>
Date: 2015-11-18T22:44:51Z
Properly set Akka frame size in test.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
