Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/7385#discussion_r34594767
--- Diff: core/src/main/scala/org/apache/spark/FutureAction.scala ---
@@ -116,37 +120,18 @@ class SimpleFutureAction[T] private[spark](jobWaiter:
JobWaiter[_], resultFunc:
}
override def ready(atMost: Duration)(implicit permit: CanAwait):
SimpleFutureAction.this.type = {
- if (!atMost.isFinite()) {
- awaitResult()
- } else jobWaiter.synchronized {
- val finishTime = System.currentTimeMillis() + atMost.toMillis
- while (!isCompleted) {
- val time = System.currentTimeMillis()
- if (time >= finishTime) {
- throw new TimeoutException
- } else {
- jobWaiter.wait(finishTime - time)
- }
- }
- }
+ jobWaiterFuture.ready(atMost)(permit) // Throws exception if the job
failed.
this
}
@throws(classOf[Exception])
override def result(atMost: Duration)(implicit permit: CanAwait): T = {
- ready(atMost)(permit)
- awaitResult() match {
- case scala.util.Success(res) => res
- case scala.util.Failure(e) => throw e
- }
+ jobWaiterFuture.result(atMost)(permit) // Throws exception if the job
failed.
+ resultFunc // This function is safe to evaluate because the job must
have succeeded.
}
- override def onComplete[U](func: (Try[T]) => U)(implicit executor:
ExecutionContext) {
- executor.execute(new Runnable {
- override def run() {
- func(awaitResult())
- }
- })
+ override def onComplete[U](func: (Try[T]) => U)(implicit executor:
ExecutionContext): Unit = {
+ jobWaiterFuture.map { _ => resultFunc }.onComplete(func)
--- End diff --
I think that @tdas and I considered this and ended up not doing it because
we thought that `resultFunc` would only be computed once, but I suppose it
doesn't hurt to be more explicit here. Even if the current code works, if it's
confusing enough to merit a comment then I think we should just be explicit and
use a lazy val. I'll update this now to do this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
