Stephen Haberman created SPARK-6560:
---------------------------------------
Summary: PairRDDFunctions suppresses exceptions in writeFile
Key: SPARK-6560
URL: https://issues.apache.org/jira/browse/SPARK-6560
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 1.3.0
Reporter: Stephen Haberman
Priority: Minor
In PairRDDFunctions, saveAsHadoopDataset uses a try/finally to manage
SparkHadoopWriter. Briefly:
{code}
try {
... writer.write(...)
} finally {
writer.close()
}
{code}
However, if an exception happens in writer.write, and then writer.close is
called, and an exception in writer.close happens, the original (real) exception
from writer.write is suppressed.
This makes debugging very painful, as the exception that is shown in the logs
(from writer.close) is spurious, and the original, real exception has been lost
and not logged.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
