GitHub user ahirreddy opened a pull request:
https://github.com/apache/spark/pull/541
[SPARK-986]: Job cancelation for PySpark
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ahirreddy/spark python-cancel
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/541.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #541
----
commit db04e16951fe4eb8a965f21c72a7db5422a319a5
Author: Ahir Reddy <[email protected]>
Date: 2014-04-24T18:24:49Z
Added canceling api to PySpark
commit 77223427d579c5321c649fd2f1a273c74f609867
Author: Ahir Reddy <[email protected]>
Date: 2014-04-24T18:31:57Z
Monitor Thread for python workers
commit 8ae96818b6c05c88401c392aacb64099792768af
Author: Ahir Reddy <[email protected]>
Date: 2014-04-24T18:58:11Z
Don't interrupt worker
commit 07b54d956079fe053072294671790cd5de482b5b
Author: Ahir Reddy <[email protected]>
Date: 2014-04-24T19:46:54Z
Fix canceling unit test
commit 5b9cae5fb444e189d796755fcba53d51eb49fd25
Author: Ahir Reddy <[email protected]>
Date: 2014-04-24T19:48:31Z
removed unnecessary vals
commit d9e472f63454caad1873e701d84b591f943cb2ab
Author: Ahir Reddy <[email protected]>
Date: 2014-04-24T20:15:29Z
Revert "removed unnecessary vals"
This reverts commit 5b9cae5fb444e189d796755fcba53d51eb49fd25.
commit adda33755f67d5642526adc9f7e95c77cf008082
Author: Ahir Reddy <[email protected]>
Date: 2014-04-24T20:20:47Z
Busy wait on the ocntext.interrupted flag, and then kill the python worker
commit d422f7b8d13e4179a6335a2ffe3167de74492255
Author: Ahir Reddy <[email protected]>
Date: 2014-04-24T20:21:19Z
Remove unnecesssary vals
commit 27a2f8f41ec53c1ce9cd6fbacdfa3d9716e5f6f8
Author: Ahir Reddy <[email protected]>
Date: 2014-04-24T21:51:13Z
Start the writer thread...
commit adba6ed535a24f80121354b7031961ba974fb591
Author: Ahir Reddy <[email protected]>
Date: 2014-04-24T22:08:35Z
Destroy python workers
commit 4b4100ab62b7137b81428241ca24d2acd212872d
Author: Ahir Reddy <[email protected]>
Date: 2014-04-24T22:16:24Z
Success flag
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
