GitHub user steveloughran opened a pull request:
https://github.com/apache/spark/pull/17364
[SPARK-20038] [core]: move the currentWriter=null assignments into finally
{} â¦
## What changes were proposed in this pull request?
have the`FileFormatWriter.ExecuteWriteTask.releaseResources()`
implementations set `currentWriter=null` in a finally clause. This guarantees
that if the first call to `currentWriter()` throws an exception, the second
releaseResources() call made during the task cancel process will not trigger a
second attempt to close the stream.
## How was this patch tested?
Tricky. I've been fixing the underlying cause when I saw the problem
(HADOOP-14204), but SPARK-10109 shows I'm not the first to have seen this. I
can't replicate it locally any more, my code no longer being broken.
code review, however, should be straightforward
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/steveloughran/spark stevel/SPARK-20038-close
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/17364.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #17364
----
commit 725740b49a2b37392699092b1b0e08c63a6152ff
Author: Steve Loughran <[email protected]>
Date: 2017-03-20T19:54:51Z
SPARK-20038: move the currentWriter=null assignments into finally {} clauses
Change-Id: I1e07f5b90ba1a2b05978b1d65876d746d07d1f3c
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
