GitHub user aarondav opened a pull request:
https://github.com/apache/spark/pull/498
SPARK-1582 Invoke Thread.interrupt() when cancelling jobs
Sometimes executor threads are blocked waiting for IO or monitors, and the
current implementation of job cancellation may never recover these threads. By
simply invoking Thread.interrupt() during cancellation, we can often safely
unblock the threads and use them for subsequent work.
Note that this feature must remain optional for now because of a bug in
HDFS where Thread.interrupt() may cause nodes to be marked as permanently dead
(as the InterruptedException is reinterpreted as an IOException during
communication with some node).
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/aarondav/spark cancel
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/498.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #498
----
commit 4cb9fd6abb61e841d1014273a3b8fe95e4dfd184
Author: Aaron Davidson <[email protected]>
Date: 2014-04-22T23:57:13Z
SPARK-1582 Invoke Thread.interrupt() when cancelling jobs
Sometimes executor threads are blocked waiting for IO or monitors,
and the current implementation of job killing may never recover
these threads. By simply invoking Thread.interrupt() during
cancellation, we can often safely unblock the threads and use them
for subsequent work.
Note that this feature must remain optional for now because of a
bug in HDFS where Thread.interrupt() may cause nodes to be marked
as permanently dead (as the InterruptedException is reinterpreted
as an IOException during communication with some node).
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
