WeichenXu123 edited a comment on issue #25235: [SPARK-28483][Core] Fix canceling a spark job using barrier mode but barrier tasks blocking on BarrierTaskContext.barrier() URL: https://github.com/apache/spark/pull/25235#issuecomment-514520895 The reason I do not use `thread.interrupt` to cancel the thread is: * In spark framework, when task canceled, the task thread is not interrupted by default, this rely on a configure `spark.job.interruptOnCancel` (default=false). * Even if we can interrupt the task thread, the scala `Future`(RPC requesting in `barrier()`) is running in a backgroud thread, we cannot get the backgroud thread handler so that still cannot interrupt it.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
