[
https://issues.apache.org/jira/browse/HADOOP-4673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12648483#action_12648483
]
Jothi Padmanabhan commented on HADOOP-4673:
-------------------------------------------
Yes, that needs to be fixed. The reason for that is because of
IFileOutputStream.close just does the checksum calculation, but does not call
close of the underlying output stream.
This possibly should be fixed by treating it the same way as compressed output
stream -- have the functionality of checksum calculation in finish() and have
close call finish and the underlying stream's close.
{code}
public void finish() {
(if finished) {
return;
}
else {
finished = true;
// calcualte checksum and write to
// underlying stream
}
}
public void close() throws IOException {
finish();
out.close();
}
{code}
> IFile.Writer close() uses compressor after returning it to CodecPool.
> ----------------------------------------------------------------------
>
> Key: HADOOP-4673
> URL: https://issues.apache.org/jira/browse/HADOOP-4673
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.18.1, 0.18.2
> Reporter: Hong Tang
> Assignee: Arun C Murthy
>
> The problem is of the same nature as HADOOP-4195.
> The compression codec is returned to the CodecPool, and later is finished in
> "out.close()".
> {code:title=IFile.java|borderStyle=solid}
> public void close() throws IOException {
> // Close the serializers
> keySerializer.close();
> valueSerializer.close();
> // Write EOF_MARKER for key/value length
> WritableUtils.writeVInt(out, EOF_MARKER);
> WritableUtils.writeVInt(out, EOF_MARKER);
> decompressedBytesWritten += 2 * WritableUtils.getVIntSize(EOF_MARKER);
>
> if (compressOutput) {
> // Flush data from buffers into the compressor
> out.flush();
>
> // Flush & return the compressor
> compressedOut.finish();
> compressedOut.resetState();
> CodecPool.returnCompressor(compressor);
> compressor = null;
> }
> // Close the stream
> rawOut.flush();
> compressedBytesWritten = rawOut.getPos() - start;
> // Close the underlying stream iff we own it...
> if (ownOutputStream) {
> out.close();
> }
> out = null;
> }
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
