Repository: spark Updated Branches: refs/heads/branch-1.5 f01a96713 -> f945b641c
[SPARK-9869] [STREAMING] Wait for all event notifications before asserting results Author: robbins <[email protected]> Closes #8589 from robbinspg/InputStreamSuite-fix. (cherry picked from commit 754f853b02e9fd221f138c2446445fd56e3f3fb3) Signed-off-by: Andrew Or <[email protected]> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/f945b641 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/f945b641 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/f945b641 Branch: refs/heads/branch-1.5 Commit: f945b641c70790a82c864ec752b673b89bb4310f Parents: f01a967 Author: robbins <[email protected]> Authored: Thu Sep 3 13:48:35 2015 -0700 Committer: Andrew Or <[email protected]> Committed: Thu Sep 3 13:48:46 2015 -0700 ---------------------------------------------------------------------- .../test/scala/org/apache/spark/streaming/InputStreamsSuite.scala | 3 +++ 1 file changed, 3 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/f945b641/streaming/src/test/scala/org/apache/spark/streaming/InputStreamsSuite.scala ---------------------------------------------------------------------- diff --git a/streaming/src/test/scala/org/apache/spark/streaming/InputStreamsSuite.scala b/streaming/src/test/scala/org/apache/spark/streaming/InputStreamsSuite.scala index ec2852d..047e38e 100644 --- a/streaming/src/test/scala/org/apache/spark/streaming/InputStreamsSuite.scala +++ b/streaming/src/test/scala/org/apache/spark/streaming/InputStreamsSuite.scala @@ -76,6 +76,9 @@ class InputStreamsSuite extends TestSuiteBase with BeforeAndAfter { fail("Timeout: cannot finish all batches in 30 seconds") } + // Ensure progress listener has been notified of all events + ssc.scheduler.listenerBus.waitUntilEmpty(500) + // Verify all "InputInfo"s have been reported assert(ssc.progressListener.numTotalReceivedRecords === input.size) assert(ssc.progressListener.numTotalProcessedRecords === input.size) --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
