Comment out DistributedLanczosWhatsit. Zing! A unit test takes a bit of code X and checks that code path A goes "tick" and code path B goes "tock" and bogus input C throws an exception. There's no such thing as a "unit test" that runs twelve M/R jobs in a row.
There's MRUnit, which seems trapped in the Hadoop 0.20/0.21/0.22/0.23 morass. This is a squib about how to do unit testing of mappers and reducers with Mockito: http://nubetech.co/testing-hadoop-map-reduce-jobs What the Mahout jobs want is more of a regression test, which would have two purposes: 1) does the whole orchestration still work, and 2) does it still acquire the information it is supposed to acquire? 2a) this requires some amount of real data and a "gold standard" output to match against. On Sat, Aug 6, 2011 at 12:34 PM, Grant Ingersoll <[email protected]> wrote: > Granted, I'm on a slow machine, but our tests take forever to run. On an 2 > core MBP, it takes well over an hour to run all the tests (I did just order a > new MBP, so it will be faster, but it doesn't lend itself to a good OOTB > experience for people) > > One idea would be to add in parallel test execution in Maven. I think this > requires Mvn 3, but I am not sure. Another is to take a look at our tests, > especially the slow ones and see if we can speed them up. > > When I try adding in parallel tests to Maven, I get a bunch of failures in > the tests. > > I was using: > <plugin> > <groupId>org.apache.maven.plugins</groupId> > <artifactId>maven-surefire-plugin</artifactId> > <configuration> > <forkMode>once</forkMode> > <argLine>-Xms256m -Xmx512m</argLine> > <testFailureIgnore>false</testFailureIgnore> > <redirectTestOutputToFile>true</redirectTestOutputToFile> > <parallel>classes</parallel> > <threadCount>5</threadCount> > </configuration> > </plugin> > > Anyone played around with this stuff? I suspect the failures are due to > tests stomping on each other, but I am still digging in. > > -Grant -- Lance Norskog [email protected]
