[
https://issues.apache.org/jira/browse/MAPREDUCE-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13273556#comment-13273556
]
Robert Joseph Evans commented on MAPREDUCE-1347:
------------------------------------------------
I didn't notice that. Yes lets move the test to the same place as the code.
It may need to be on a different JIRA depending on if it uses the mini cluster
or not. If it uses the minicluster we may need to leave part of the test where
it is and put in some real unit tests instead.
> Missing synchronization in MultipleOutputFormat
> -----------------------------------------------
>
> Key: MAPREDUCE-1347
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1347
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 1.0.0
> Reporter: Todd Lipcon
> Assignee: Harsh J
> Attachments: MAPREDUCE-1347.r2.diff, MAPREDUCE-1347.r3.diff,
> MAPREDUCE-1347.r4.diff, MAPREDUCE-1347.r5.diff, MAPREDUCE-1347.r6.diff,
> MAPREDUCE-1347.r7.diff, MAPREDUCE-1347.r8.diff, mapreduce.1347.r1.diff
>
>
> MultipleOutputFormat's RecordWriter implementation doesn't use
> synchronization when accessing the recordWriters member. When using
> multithreaded mappers or reducers, this can result in problems where two
> threads will both try to create the same file, causing
> AlreadyBeingCreatedException. Doing this more fine-grained than just
> synchronizing the whole method is probably a good idea, so that multithreaded
> mappers can actually achieve parallelism writing into separate output streams.
> From what I can tell, the new API's MultipleOutputs seems not to have this
> issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
