Merge pull request #74 from rxin/kill
Job cancellation via job group id.
This PR adds a simple API to group together a set of jobs belonging to a thread
and threads spawned from it. It also allows the cancellation of all jobs in
this group.
An example:
sc.setJobDescription("this_is_the_group_id", "some job description")
sc.parallelize(1 to 10000, 2).map { i => Thread.sleep(10); i }.count()
In a separate thread:
sc.cancelJobGroup("this_is_the_group_id")
Project: http://git-wip-us.apache.org/repos/asf/incubator-spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-spark/commit/599dcb0d
Tree: http://git-wip-us.apache.org/repos/asf/incubator-spark/tree/599dcb0d
Diff: http://git-wip-us.apache.org/repos/asf/incubator-spark/diff/599dcb0d
Branch: refs/heads/master
Commit: 599dcb0ddf740e028cc8faac163303be8f9400a6
Parents: 8de9706 806f3a3
Author: Matei Zaharia <ma...@eecs.berkeley.edu>
Authored: Fri Oct 18 22:49:00 2013 -0700
Committer: Matei Zaharia <ma...@eecs.berkeley.edu>
Committed: Fri Oct 18 22:49:00 2013 -0700
----------------------------------------------------------------------
.../scala/org/apache/spark/SparkContext.scala | 22 ++++++++++++--
.../apache/spark/scheduler/DAGScheduler.scala | 26 +++++++++++++---
.../spark/scheduler/DAGSchedulerEvent.scala | 2 ++
.../org/apache/spark/JobCancellationSuite.scala | 32 ++++++++++++++++++++
4 files changed, 75 insertions(+), 7 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-spark/blob/599dcb0d/core/src/main/scala/org/apache/spark/SparkContext.scala
----------------------------------------------------------------------
