[
https://issues.apache.org/jira/browse/MAPREDUCE-2258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12980871#action_12980871
]
Todd Lipcon commented on MAPREDUCE-2258:
----------------------------------------
The following is a unit test I wrote on the hadoop-lzo side that does the same
behavior as IFile.Reader.close():
https://github.com/toddlipcon/hadoop-lzo/commit/a5af3b93f52f55828dfc05e7503d38383eec9dc5
It fails reliably since some threads only manage to read part of the data in
the file.
> IFile reader closes stream and compressor in wrong order
> --------------------------------------------------------
>
> Key: MAPREDUCE-2258
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2258
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
> Affects Versions: 0.20.4, 0.22.0
> Reporter: Todd Lipcon
> Fix For: 0.22.0
>
>
> In IFile.Reader.close(), we return the decompressor to the pool and then call
> close() on the input stream. This is backwards and causes a rare race in the
> case of LzopCodec, since LzopInputStream makes a few calls on the
> decompressor object inside close(). If another thread pulls the decompressor
> out of the pool and starts to use it in the meantime, the first thread's
> close() will cause the second thread to potentially miss pieces of data.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
