> On 2011-06-24 19:39:07, Todd Lipcon wrote: > > I think your issue with the test case is that you're calling > > tMOF.getRecordWriter() inside the thread. So, each thread has its own > > record writer and hence has its own computing map. You should call that > > from outside the thread, and just have the threads do the writing.
/me bangs head on the wall. Thanks for catching it! Tests fail/pass now (w/o and w/ old style sync). Posting patch to JIRA in a bit, unless there are further comments beyond that? - Harsh ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/953/#review910 ----------------------------------------------------------- On 2011-06-24 19:06:46, Harsh J wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/953/ > ----------------------------------------------------------- > > (Updated 2011-06-24 19:06:46) > > > Review request for hadoop-mapreduce and Todd Lipcon. > > > Summary > ------- > > Used the makeComputingMap from Guava's MapMaker to provide a thread safe way > of creating a RecordWriter cache. > > For some reason, the map is not really caching it and is instead trying to > apply() over and over again for the same key-value pairs. > > > This addresses bug MAPREDUCE-1347. > http://issues.apache.org/jira/browse/MAPREDUCE-1347 > > > Diffs > ----- > > mapreduce/ivy.xml 85ee014 > mapreduce/ivy/libraries.properties 9d40aaa > mapreduce/src/java/org/apache/hadoop/mapred/lib/MultipleOutputFormat.java > b8944f1 > > mapreduce/src/test/mapred/org/apache/hadoop/mapred/TestMultipleTextOutputFormat.java > 14c097d > > Diff: https://reviews.apache.org/r/953/diff > > > Testing > ------- > > Added a test case, but it fails with the current behavior of MapMaker's > makeComputingMap() (would pass if its alright) > > > Thanks, > > Harsh > >
