GitHub user hthuynh2 opened a pull request:
https://github.com/apache/spark/pull/21936
[SPARK-24981][Core] ShutdownHook timeout causes job to fail when succeeded
when SparkContext stop() not called by user program
**Description**
The issue is described in
[SPARK-24981](https://issues.apache.org/jira/browse/SPARK-24981).
**How does this PR fix the issue?**
This PR catch the Exception that is thrown while the Sparkcontext.stop() is
running (when it is called by the ShutdownHookManager).
**How was this patch tested?**
I manually tested it by adding delay (60s) inside the stop(). This make the
shutdownHookManger interrupt the thread that is running stop(). The Interrupted
Exception was catched and the job succeed.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/hthuynh2/spark SPARK_24981
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21936.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21936
----
commit 101bf2b0488ab495dbdaf7b54d9a57d9827cb833
Author: Hieu Huynh <âhieu.huynh@...>
Date: 2018-07-31T18:30:11Z
initial commit
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
