GitHub user BryanCutler opened a pull request:
https://github.com/apache/spark/pull/10714
[SPARK-9844][CORE] File appender race condition during shutdown
When an Executor process is destroyed, the FileAppender that is
asynchronously reading the stderr stream of the process can throw an
IOException during read because the stream is closed. Before the
ExecutorRunner destroys the process, the FileAppender thread is flagged to
stop. This PR wraps the inputStream.read call of the FileAppender in a
try/catch block so that if an IOException is thrown and the thread has been
flagged to stop, it will safely ignore the exception. Additionally, the
FileAppender thread was changed to use Utils.tryWithSafeFinally to better log
any exception that do occur. Added unit tests to verify a IOException is
thrown and logged if FileAppender is not flagged to stop, and that no
IOException when the flag is set.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/BryanCutler/spark
file-appender-read-ioexception-SPARK-9844
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/10714.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #10714
----
commit 6efff7495a6bd630baf20b5238d51644d5abca46
Author: Bryan Cutler <[email protected]>
Date: 2016-01-12T00:22:08Z
[SPARK-9844] Added check in FileAppender to not thrown an IOException if
marked as stopped and changed file operation block to use tryWithSafeFinally
commit f937e183d466d014f1545d823ecdf30f9becdf77
Author: Bryan Cutler <[email protected]>
Date: 2016-01-12T00:22:55Z
[SPARK-9844] Added unit test for closing FileAppender InputStream
asynchronously
commit 1775d98c1d9c9664e0f9ce80082715938c6e0aed
Author: Bryan Cutler <[email protected]>
Date: 2016-01-12T00:27:03Z
fixed import order
commit dc6573715a77107e7fbaaf97b4dbbc18f652b9b3
Author: Bryan Cutler <[email protected]>
Date: 2016-01-12T00:57:09Z
Did not need check for log level ERROR because only logging ERRORS, check
for getThrowableInformation null
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
