Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/7385#discussion_r34749845
--- Diff: core/src/test/scala/org/apache/spark/FutureActionSuite.scala ---
@@ -49,4 +50,20 @@ class FutureActionSuite
job.jobIds.size should be (2)
}
+ test("simple async action callbacks should not tie up execution context
threads (SPARK-9026)") {
+ val rdd = sc.parallelize(1 to 10, 2).map(_ => Thread.sleep(1000 *
1000))
+ val pool =
ThreadUtils.newDaemonCachedThreadPool("SimpleFutureActionTest")
+ val executionContext = ExecutionContext.fromExecutorService(pool)
+ val job = rdd.countAsync()
+ try {
+ for (_ <- 1 to 10) {
+ job.onComplete(_ => ())(executionContext)
+ assert(pool.getLargestPoolSize < 10)
--- End diff --
This looks flaky. Even they are non blocking, there is NO guarantee that
one of the 10 scheduled function `_ => ()` will finish by the end of this loop.
So it may happen that in the 10th iteration, the previous 9 scheduled function
are still not finished, the 10th on gets scheduled, and therefore the pool size
= 10.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
