GitHub user piaozhexiu opened a pull request:
https://github.com/apache/spark/pull/8216
[SPARK-10001] [CORE] Ctrl-C in spark-shell doesn't kill running job
This patch attaches a signal handler for SIGINT to JobWaiter when a job is
running and detaches it as soon as the job is finished. When ctrl-c is pressed
during this time, spark-shell cancels the running job.
It's worth noting that signal handler is only overridden when job is
running. So if ctrl-c is pressed when spark-shell is idle, it will still
interrupt JVM and exit the shell (same as now).
In addition, if ctrl-c is pressed multiple times when job is running, it
will interrupt JVM and exit the shell. This is to avoid any case that the shell
becomes unresponsive. For eg, consider this scenario. User submits a job in the
shell. But since the job lists a lot of files before running, it might be
hanging in the driver. In this case, user can interrupt JVM and exit the shell
by ctrl-c'ing twice.
Here is summary-
\# | case | before | after
---- | ------------| -------- | ------
1 | shell is idle | ctrl-c will exit the shell | ctrl-c will exit the shell
2 | job is submitted but not yet running (e.g. driver listing input files)
| ctrl-c will exit the shell | ctrl-c will exit the shell [a]
3 | job is submitted and running on cluster | ctrl-c will exit the shell |
ctrl-c will not exit the shell but cancel the job [b]
This patch basically improves ``case 3`` by allowing the user to cancel a
running job on cluster w/o exiting the shell.
[a] 1st ctrl-c will print a log message saying ``Cancelling running job..
This might take some time, so be patient. Press Ctrl-C again to kill JVM.``
2nd ctrl-c will exit the shell.
[b] when job is running, ctrl-c immediately cancels the job, and the prompt
is returned to user.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/piaozhexiu/spark SPARK-10001
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/8216.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #8216
----
commit 7342f1b0be1d80f69a54bd3b4930386abb0a8008
Author: Cheolsoo Park <[email protected]>
Date: 2015-08-14T21:07:31Z
Add signal handler to JobWaiter to handle ctrl-c
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
