WeichenXu123 commented on a change in pull request #25235: [SPARK-28483][Core]
Fix canceling a spark job using barrier mode but barrier tasks blocking on
BarrierTaskContext.barrier()
URL: https://github.com/apache/spark/pull/25235#discussion_r315266797
##########
File path: core/src/main/scala/org/apache/spark/rpc/netty/Outbox.scala
##########
@@ -66,14 +66,22 @@ private[netty] case class RpcOutboxMessage(
this.requestId = client.sendRpc(content, this)
}
- def onTimeout(): Unit = {
+ private[netty] def removeRpcRequest(): Unit = {
if (client != null) {
client.removeRpcRequest(requestId)
} else {
- logError("Ask timeout before connecting successfully")
+ logError("Ask terminated before connecting successfully")
}
}
+ def onTimeout(): Unit = {
+ removeRpcRequest()
+ }
+
+ def onAbort(): Unit = {
+ removeRpcRequest()
Review comment:
Not a good idea.
When we want to abort the RPC, we should trigger an RPC aborted exception
(thrown from the `Future` thread itself). And let the `Future` thread to catch
the exception, and do cleanup, and then re-throw the exception.
We can also reference how RPC impl the "timeout" , see
```
val timeoutCancelable = timeoutScheduler.schedule(new Runnable {
override def run(): Unit = {
onFailure(new TimeoutException(s"Cannot receive any reply from
${remoteAddr} " +
s"in ${timeout.duration}"))
}
}, timeout.duration.toNanos, TimeUnit.NANOSECONDS)
```
It only call `onFailure(new TimeoutException(...))` and inside `onFailure`
it call `promise.tryFailure(e)` to raise exception `Future` thread itself.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
