GitHub user CodingCat opened a pull request:
https://github.com/apache/spark/pull/5004
[SPARK-4012] exit JVM process when the exception is thrown from an infinite
loop
https://issues.apache.org/jira/browse/SPARK-4012
This patch is a resubmission for https://github.com/apache/spark/pull/2864
What I am proposing in this patch is that ***when the exception is thrown
from an infinite loop, we should stop the process, instead of let JVM throws
exception forever***
So, in the infinite loops where we originally wrapped with a `
logUncaughtExceptions`, I changed to `tryOrExit`, so that the JVM process is
stopped
Early stopped JVM process is helpful for HA scheme design, for example,
The user has a script checking the existence of the pid of the Spark
Streaming driver for monitoring the availability; with the code before this
patch, the JVM process is still available but not functional when the
exceptions are thrown
@andrewor14, @srowen , mind taking further consideration about the change?
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/CodingCat/spark SPARK-4012-1
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/5004.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5004
----
commit 63229590d4645e0da8cb7e4217931a0443f7900d
Author: CodingCat <[email protected]>
Date: 2015-03-12T17:39:39Z
exit JVM process when the exception is thrown from an infinite loop
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
