[
https://issues.apache.org/jira/browse/SPARK-1582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Patrick Wendell resolved SPARK-1582.
------------------------------------
Resolution: Fixed
Fix Version/s: 1.0.0
> Job cancellation does not interrupt threads
> -------------------------------------------
>
> Key: SPARK-1582
> URL: https://issues.apache.org/jira/browse/SPARK-1582
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 1.0.0, 0.9.1
> Reporter: Aaron Davidson
> Assignee: Aaron Davidson
> Fix For: 1.0.0
>
>
> Cancelling Spark jobs is limited because executors that are blocked are not
> interrupted. In effect, the cancellation will succeed and the job will no
> longer be "running", but executor threads may still be tied up with the
> cancelled job and unable to do further work until complete. This is
> particularly problematic in the case of deadlock or unlimited/long timeouts.
> It would be useful if cancelling a job would call Thread.interrupt() in order
> to interrupt blocking in most situations, such as Object monitors or IO. The
> one caveat is [HDFS-1208|https://issues.apache.org/jira/browse/HDFS-1208],
> where HDFS's DFSClient will not only swallow InterruptedException but may
> reinterpret them as IOException, causing HDFS to mark a node as permanently
> failed. Thus, this feature must be optional and probably off by default.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
