[
https://issues.apache.org/jira/browse/HADOOP-2926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12574657#action_12574657
]
Raghu Angadi commented on HADOOP-2926:
--------------------------------------
You are right. The example above shows wrong use of closeStream().
> it clear it can only be used after the write operation has failed and is
> being cleaned up.
But this is not true. Many times this used to close sockets where socket needs
to be closed because of unrelated error (say i/o failed on a different stream).
> Ignoring IOExceptions on close
> ------------------------------
>
> Key: HADOOP-2926
> URL: https://issues.apache.org/jira/browse/HADOOP-2926
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.16.0
> Reporter: Owen O'Malley
> Assignee: dhruba borthakur
> Priority: Critical
> Fix For: 0.16.1
>
>
> Currently in HDFS there are a lot of calls to IOUtils.closeStream that are
> from finally blocks. I'm worried that this can lead to data corruption in the
> file system. Take the first instance in DataNode.copyBlock: it writes the
> block and then calls closeStream on the output stream. If there is an error
> at the end of the file that is detected in the close, it will be *completely*
> ignored. Note that logging the error is not enough, the error should be
> thrown so that the client knows the failure happened.
> {code}
> try {
> file1.write(...);
> file2.write(...);
> } finally {
> IOUtils.closeStream(file);
> }
> {code}
> is *bad*. It must be rewritten as:
> {code}
> try {
> file1.write(...);
> file2.write(...);
> file1.close(...);
> file2.close(...);
> } catch (IOException ie) {
> IOUtils.closeStream(file1);
> IOUtils.closeStream(file2);
> throw ie;
> }
> {code}
> I also think that IOUtils.closeStream should be renamed
> IOUtils.cleanupFailedStream or something to make it clear it can only be used
> after the write operation has failed and is being cleaned up.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
