Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/7385#discussion_r34750699
--- Diff: core/src/test/scala/org/apache/spark/FutureActionSuite.scala ---
@@ -49,4 +50,20 @@ class FutureActionSuite
job.jobIds.size should be (2)
}
+ test("simple async action callbacks should not tie up execution context
threads (SPARK-9026)") {
+ val rdd = sc.parallelize(1 to 10, 2).map(_ => Thread.sleep(1000 *
1000))
+ val pool =
ThreadUtils.newDaemonCachedThreadPool("SimpleFutureActionTest")
+ val executionContext = ExecutionContext.fromExecutorService(pool)
+ val job = rdd.countAsync()
+ try {
+ for (_ <- 1 to 10) {
+ job.onComplete(_ => ())(executionContext)
+ assert(pool.getLargestPoolSize < 10)
--- End diff --
I think that my intention when writing this test was to have a test that
demonstrated the eagerly-create-a-thread-per-callback problem with the old
implementation of SimpleFutureAction.
I don't think that this is flaky but I also don't think that this tests
adds much value since we're unlikely to ever switch back to the old inefficient
implementation. I'll just drop this test, since I don't think it's adding any
real value right now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
