This test is indeed by far the culprit. I already reduced its test input size to hurry it up, but it's gone slow again.
Lance, indeed, these are not all unit tests -- nobody said they were. The test is useful. I do suggest, however, we comment it out. Jake suggested it coudl be made faster but I don't think he followed up. Sean On Sun, Aug 7, 2011 at 12:13 AM, Lance Norskog <[email protected]> wrote: > Comment out DistributedLanczosWhatsit. Zing! > > A unit test takes a bit of code X and checks that code path A goes > "tick" and code path B goes "tock" and bogus input C throws an > exception. There's no such thing as a "unit test" that runs twelve M/R > jobs in a row. > > There's MRUnit, which seems trapped in the Hadoop 0.20/0.21/0.22/0.23 > morass. This is a squib about how to do unit testing of mappers and > reducers with Mockito: > > http://nubetech.co/testing-hadoop-map-reduce-jobs > > What the Mahout jobs want is more of a regression test, which would > have two purposes: > 1) does the whole orchestration still work, and > 2) does it still acquire the information it is supposed to acquire? > 2a) this requires some amount of real data and a "gold standard" > output to match against. > > On Sat, Aug 6, 2011 at 12:34 PM, Grant Ingersoll <[email protected]> > wrote: > > Granted, I'm on a slow machine, but our tests take forever to run. On an > 2 core MBP, it takes well over an hour to run all the tests (I did just > order a new MBP, so it will be faster, but it doesn't lend itself to a good > OOTB experience for people) > > > > One idea would be to add in parallel test execution in Maven. I think > this requires Mvn 3, but I am not sure. Another is to take a look at our > tests, especially the slow ones and see if we can speed them up. > > > > When I try adding in parallel tests to Maven, I get a bunch of failures > in the tests. > > > > I was using: > > <plugin> > > <groupId>org.apache.maven.plugins</groupId> > > <artifactId>maven-surefire-plugin</artifactId> > > <configuration> > > <forkMode>once</forkMode> > > <argLine>-Xms256m -Xmx512m</argLine> > > <testFailureIgnore>false</testFailureIgnore> > > <redirectTestOutputToFile>true</redirectTestOutputToFile> > > <parallel>classes</parallel> > > <threadCount>5</threadCount> > > </configuration> > > </plugin> > > > > Anyone played around with this stuff? I suspect the failures are due to > tests stomping on each other, but I am still digging in. > > > > -Grant > > > > -- > Lance Norskog > [email protected] >
