GitHub user stephenh opened a pull request:
https://github.com/apache/spark/pull/5223
[SPARK-6560][CORE] Do not suppress exceptions from writer.write.
If there is a failure in the Hadoop backend while calling
writer.write, we should remember this original exception,
and try to call writer.close(), but if that fails as well,
still report the original exception.
Note that, if writer.write fails, it is likely that writer
was left in an invalid state, and so actually makes it more
likely that writer.close will also fail. Which just increases
the chances for writer.write's exception to be suppressed.
This patch introduces an admittedly potentially too cute
Utils.tryWithSafeFinally method to handle the try/finally
gyrations.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/stephenh/spark
do_not_suppress_writer_exception
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/5223.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5223
----
commit f42e92de46967e0b8dc19c198ea0ea6196c6ebca
Author: Stephen Haberman <[email protected]>
Date: 2015-03-27T04:01:28Z
[SPARK-6560][CORE] Do not suppress exceptions from writer.write.
If there is a failure in the Hadoop backend while calling
writer.write, we should remember this original exception,
and try to call writer.close(), but if that fails as well,
still report the original exception.
Note that, if writer.write fails, it is likely that writer
was left in an invalid state, and so actually makes it more
likely that writer.close will also fail. Which just increases
the chances for writer.write's exception to be suppressed.
This patch introduces an admittedly potentially too cute
Utils.tryWithSafeFinally method to handle the try/finally
gyrations.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
