Am Freitag, 30. August 2013 schrieb Stevo Slavić : > Thanks Andreas, > > Removing parallel=classes did the trick for timely console reporting. > Having parallel=classes didn't bother m-surefire-p in 2.15 and earlier.
In surefire 2.16, the whole Junit parallel stuff was rewritten, so some changed behavior is to be expected. > With forkCount > 1 and reuseForks = false, parallel=classes should just be > ignored. But, as long as there is solution, I'm fine. > > IMO specifying test parameter shouldn't disable the ordering (runOrder), > since specifying test parameter (by docs) is equivalent to specifying > includes patterns in pom, and includes should be used only to filter > scanned test classes, ordering should be applied after scanning, and before > executing test classes. > > I understand that there are no guarantees on order when only two test > classes are run in parallel with forkCount > 1. > Still if runOrder is applied before forking VMs and forked VMs as you > describe poll for next test class to execute, and I run a dozen of tests > (specified with test parameter) each of which takes different number of > more than few seconds to finish, and I have forkCount set to 2, I'd expect > runOrder to be somewhat respected - for the first two tests which get run > it would be random, but for the rest it should be pretty much in specified > runOrder, until two finish at roughly same time and there is (again as on > beginning) concurrency which test class will run next. Do you agree? Totally. Would you please create a bug report for that? Thanks, Andreas > I tried that (test specifying dozen of test classes, forkCount at 2, > reuseForks at false, and different runOrder settings), and still I don't > see runOrder being respected. > > Btw my original suspicion, that the two of Apache Mahout tests do not run > well together, was wrong - failing test had a bug, it makes use of random > number generator, but didn't set test seed to make execution deterministic. > > Kind regards, > Stevo Slavic. > > > On Fri, Aug 30, 2013 at 8:06 AM, Andreas Gudian > <[email protected]<javascript:;> > >wrote: > > > I'll have to check the code, but I'm pretty sure that the order of the > test > > classes to be handed over to the forked VMs is supposed to be what was is > > specified with runOrder. > > > > However, using -Dtest= might disable the ordering - but again, I'd have > to > > check the code. > > > > Bit there's more: when having two classes to test and two forked VMs, > there > > is no real way to tell which test comes first. Both VMs are started right > > away and only when the are up and running and ready to execute a test, > they > > ask the main process for one test class. Which VM will be the first to > get > > a class is quite random. > > > > For the delayed reporting to console, try removing parallel=classes from > > the config. IThe classes-setting has no effect when forking to more than > > one fork anyway. If the output is still delayed then, create a > bug-report, > > please. > > > > > > Am Donnerstag, 29. August 2013 schrieb Stevo Slavić : > > > > > Hello Apache Maven community, > > > > > > To reproduce random failing test issue, I'm trying to run two tests in > > > parallel, but start running them in specific order. Problem is that > > > runOrder (added via SUREFIRE-614) doesn't seem to be respected, at > least > > > not when forkCount and/or test is specified as well. > > > > > > When trying to run two tests from Apache Mahout core module (trunk), I > > > cannot get surefire to run them in reverse alphabetical order. > > > > > > Regardless if I run: > > > mvn -Dtest=*KMeansTest -DrunOrder=reversealphabetical test > > > > > > or > > > mvn -Dtest=*KMeansTest -DrunOrder=alphabetical test > > > > > > the two tests which match the pattern always get run in alphabetical > > order: > > > > > > ------------------------------------------------------- > > > T E S T S > > > ------------------------------------------------------- > > > Running org.apache.mahout.clustering.streaming.cluster.BallKMeansTest > > > Running > > org.apache.mahout.clustering.streaming.cluster.StreamingKMeansTest > > > Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 66.706 > > sec > > > - in org.apache.mahout.clustering.streaming.cluster.BallKMeansTest > > > Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 82.15 > > sec - > > > in org.apache.mahout.clustering.streaming.cluster.StreamingKMeansTest > > > > > > Results : > > > > > > Tests run: 11, Failures: 0, Errors: 0, Skipped: 0 > > > > > > > > > > > > > > > Surefire 2.15 is used, and configuration is: > > > > > > > > > <plugin> > > > <groupId>org.apache.maven.plugins</groupId> > > > <artifactId>maven-surefire-plugin</artifactId> > > > <configuration> > > > <forkCount>1.5C</forkCount> > > > <reuseForks>false</reuseForks> > > > <threadCount>1</threadCount> > > > <perCoreThreadCount>false</perCoreThreadCount> > > > <parallel>classes</parallel> > > > <argLine>-Xmx512m</argLine> > > > <testFailureIgnore>false</testFailureIgnore> > > > <redirectTestOutputToFile>true</redirectTestOutputToFile> > > > <systemPropertyVariables> > > > > > > > <mahout.test.directory>${project.build.directory}</mahout.test.directory> > > > </systemPropertyVariables> > > > </configuration> > > > </plugin> > > > > > > > > > When I use latest surefire 2.16, outcome is even more odd - output that > > any > > > of the tests is running takes longer to render, it seems to be rendered > > > together with info that test has completed, so by looking at build log > > > output it may appear as if tests were not running in parallel even > though >
