GitHub user zsxwing opened a pull request:
https://github.com/apache/spark/pull/8813
[SPARK-10651][Core][Tests]Increase timeout to fix BroadcastSuite
I noticed only one block manager registered with master in an unsuccessful
build
(https://amplab.cs.berkeley.edu/jenkins/job/Spark-Master-SBT/AMPLAB_JENKINS_BUILD_PROFILE=hadoop2.2,label=spark-test/3534/)
```
15/09/16 13:02:30.981 pool-1-thread-1-ScalaTest-running-BroadcastSuite INFO
SparkContext: Running Spark version 1.6.0-SNAPSHOT
...
15/09/16 13:02:38.133 sparkDriver-akka.actor.default-dispatcher-19 INFO
BlockManagerMasterEndpoint: Registering block manager localhost:48196 with
530.3 MB RAM, BlockManagerId(0, localhost, 48196)
```
In addition, the first block manager needed 7+ seconds to start.
However, there was no exception in this log file. So I checked a successful
build
(https://amplab.cs.berkeley.edu/jenkins/job/Spark-Master-SBT/3536/AMPLAB_JENKINS_BUILD_PROFILE=hadoop2.2,label=spark-test/)
and it needed 4-5 seconds to set up the local cluster:
```
15/09/16 18:11:27.738 sparkWorker1-akka.actor.default-dispatcher-5 INFO
Worker: Running Spark version 1.6.0-SNAPSHOT
...
15/09/16 18:11:30.838 sparkDriver-akka.actor.default-dispatcher-20 INFO
BlockManagerMasterEndpoint: Registering block manager localhost:54202 with
530.3 MB RAM, BlockManagerId(1, localhost, 54202)
15/09/16 18:11:32.112 sparkDriver-akka.actor.default-dispatcher-20 INFO
BlockManagerMasterEndpoint: Registering block manager localhost:32955 with
530.3 MB RAM, BlockManagerId(0, localhost, 32955)
```
In this build, the first block manager needed only 3+ seconds to start.
Comparing these two builds, I guess it's possible that the local cluster in
`BroadcastSuite` cannot be ready in 10 seconds if the Jenkins worker is busy.
So I just increased the timeout to 60 seconds to see if this can fix the issue.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zsxwing/spark fix-BroadcastSuite
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/8813.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #8813
----
commit 4deb98bf6b6ca35ffe4426f706f7e40b66dd7e0f
Author: zsxwing <[email protected]>
Date: 2015-09-18T07:39:47Z
Increase timeout to fix BroadcastSuite
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
