spark git commit: [SPARK-16722][TESTS] Fix a StreamingContext leak in StreamingContextSuite when eventually fails
Repository: spark
Updated Branches:
refs/heads/master db36e1e75 -> e164a04b2
[SPARK-16722][TESTS] Fix a StreamingContext leak in StreamingContextSuite when
eventually fails
## What changes were proposed in this pull request?
This PR moves `ssc.stop()` into `finally` for
`StreamingContextSuite.createValidCheckpoint` to avoid leaking a
StreamingContext since leaking a StreamingContext will fail a lot of tests and
make us hard to find the real failure one.
## How was this patch tested?
Jenkins unit tests
Author: Shixiong Zhu
Closes #14354 from zsxwing/ssc-leak.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/e164a04b
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/e164a04b
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/e164a04b
Branch: refs/heads/master
Commit: e164a04b2ba3503e5c14cd1cd4beb40e0b79925a
Parents: db36e1e
Author: Shixiong Zhu
Authored: Mon Jul 25 18:26:29 2016 -0700
Committer: Tathagata Das
Committed: Mon Jul 25 18:26:29 2016 -0700
--
.../org/apache/spark/streaming/StreamingContextSuite.scala | 9 ++---
1 file changed, 6 insertions(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/spark/blob/e164a04b/streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
--
diff --git
a/streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
b/streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
index 806e181..f1482e5 100644
---
a/streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
+++
b/streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
@@ -819,10 +819,13 @@ class StreamingContextSuite extends SparkFunSuite with
BeforeAndAfter with Timeo
ssc.checkpoint(checkpointDirectory)
ssc.textFileStream(testDirectory).foreachRDD { rdd => rdd.count() }
ssc.start()
-eventually(timeout(1 millis)) {
- assert(Checkpoint.getCheckpointFiles(checkpointDirectory).size > 1)
+try {
+ eventually(timeout(3 millis)) {
+assert(Checkpoint.getCheckpointFiles(checkpointDirectory).size > 1)
+ }
+} finally {
+ ssc.stop()
}
-ssc.stop()
checkpointDirectory
}
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
spark git commit: [SPARK-16722][TESTS] Fix a StreamingContext leak in StreamingContextSuite when eventually fails
Repository: spark
Updated Branches:
refs/heads/branch-2.0 b17fe4e41 -> 9d581dc61
[SPARK-16722][TESTS] Fix a StreamingContext leak in StreamingContextSuite when
eventually fails
## What changes were proposed in this pull request?
This PR moves `ssc.stop()` into `finally` for
`StreamingContextSuite.createValidCheckpoint` to avoid leaking a
StreamingContext since leaking a StreamingContext will fail a lot of tests and
make us hard to find the real failure one.
## How was this patch tested?
Jenkins unit tests
Author: Shixiong Zhu
Closes #14354 from zsxwing/ssc-leak.
(cherry picked from commit e164a04b2ba3503e5c14cd1cd4beb40e0b79925a)
Signed-off-by: Tathagata Das
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/9d581dc6
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/9d581dc6
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/9d581dc6
Branch: refs/heads/branch-2.0
Commit: 9d581dc61951eccf0f06868e0d3f10134f433e82
Parents: b17fe4e
Author: Shixiong Zhu
Authored: Mon Jul 25 18:26:29 2016 -0700
Committer: Tathagata Das
Committed: Mon Jul 25 18:26:37 2016 -0700
--
.../org/apache/spark/streaming/StreamingContextSuite.scala | 9 ++---
1 file changed, 6 insertions(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/spark/blob/9d581dc6/streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
--
diff --git
a/streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
b/streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
index 806e181..f1482e5 100644
---
a/streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
+++
b/streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
@@ -819,10 +819,13 @@ class StreamingContextSuite extends SparkFunSuite with
BeforeAndAfter with Timeo
ssc.checkpoint(checkpointDirectory)
ssc.textFileStream(testDirectory).foreachRDD { rdd => rdd.count() }
ssc.start()
-eventually(timeout(1 millis)) {
- assert(Checkpoint.getCheckpointFiles(checkpointDirectory).size > 1)
+try {
+ eventually(timeout(3 millis)) {
+assert(Checkpoint.getCheckpointFiles(checkpointDirectory).size > 1)
+ }
+} finally {
+ ssc.stop()
}
-ssc.stop()
checkpointDirectory
}
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
