Github user viktorklang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22304#discussion_r214555205
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/util/FileBasedWriteAheadLog.scala
---
@@ -65,7 +65,8 @@ private[streaming] class FileBasedWriteAheadLog(
"WriteAheadLogManager" + callerName.map(c => s" for $c").getOrElse("")
}
private val forkJoinPool = ThreadUtils.newForkJoinPool(threadpoolName,
20)
- private val executionContext =
ExecutionContext.fromExecutorService(forkJoinPool)
+ private val executionContext = ExecutionContext
+ .fromExecutorService(forkJoinPool, { e: Throwable => throw e })
--- End diff --
The suggested solution will not work as intended as it will prevent
subsequent callbacks (esp for other ExecutionContexts) to be submitted, esp. in
the case of RejectedExecutionException.
See:
https://github.com/scala/scala/blob/2.12.x/src/library/scala/concurrent/impl/Promise.scala#L68
https://github.com/scala/scala/blob/2.12.x/src/library/scala/concurrent/impl/Promise.scala#L284
Or, in other words, rethrowing is not "reporting" the error.
The new Future encoding included in Scala 2.13.x deals much better with
RejectedExecutionExceptions and InterruptedExceptions, but it is unclear right
now what parts could be backported within the binary compatibility constraints.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
