WeichenXu123 commented on a change in pull request #25235: [SPARK-28483][Core]
Fix canceling a spark job using barrier mode but barrier tasks blocking on
BarrierTaskContext.barrier()
URL: https://github.com/apache/spark/pull/25235#discussion_r310346267
##########
File path: core/src/main/scala/org/apache/spark/rpc/RpcEndpointRef.scala
##########
@@ -46,6 +46,17 @@ private[spark] abstract class RpcEndpointRef(conf:
SparkConf)
*/
def send(message: Any): Unit
+ /**
+ * Send a message to the corresponding [[RpcEndpoint.receiveAndReply)]] and
return a [[Future]] to
+ * receive the reply within the specified timeout.
+ * Return a `CancelableFuture` instance which wrap `Future` but with
additional `cancel` method.
+ *
+ * This method only sends the message once and never retries.
+ */
+ def askCancelable[T: ClassTag](message: Any, timeout: RpcTimeout):
CancelableFuture[T] = {
Review comment:
I am ok with the name either `askWithCleanupOnCancel` or `askCancelable`
When at client side we do the rpc cleanup "remove request id from
`outstandingRpcs`", then the rpc callback will be ignored when client receive
its response.
https://github.com/apache/spark/blob/b3394db1930b3c9f55438cb27bb2c584bf041f8e/common/network-common/src/main/java/org/apache/spark/network/client/TransportResponseHandler.java#L189
Not very strict, this behavior, from the caller side, it could be regarded
as "canceled", because once it is cleaned, no responding callback will happen.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
