I still wasn't able to reproduce your issue. Looks like TestNG is running in "mixed mode" [1][2]. The last missing information is the version of JUnit and TestNG you are using. Can you provide this piece of configuration?
[1]: http://testng.org/doc/migrating.html [2]: http://testng.org/doc/documentation-main.html#junit 2013/7/24 Andreas Dolk <[email protected]> > Hi Francesco, > > I'm using 2.15 > > And here's the result from a test run, that's what happens. The tests are > *only* jUnit tests. I've only replaced path and package names. BTW, the > jUnit times are net execution times (unfair!!), testNG reports the total > times (fair) ;) The other annoying part is that TestNG picks up far more > classes then jUnit... > > The tests are auto-compiled from jnario specs (jnario.org) which shouldn't > make a difference - at the end it's classes compiled from java source > files. > > Regards, > Andreas > > > > > mvn -Dtest=JnSpec* test > > ... > > [INFO] --- maven-surefire-plugin:2.15:test (default-test) @ a42-order-be > --- > [INFO] Surefire report directory: /<path>/target/surefire-reports > [INFO] Using configured provider > org.apache.maven.surefire.junitcore.JUnitCoreProvider > [INFO] Using configured provider > org.apache.maven.surefire.testng.TestNGProvider > [INFO] parallel='none', perCoreThreadCount=true, threadCount=2, > useUnlimitedThreads=false > > ------------------------------------------------------- > T E S T S > ------------------------------------------------------- > > ------------------------------------------------------- > T E S T S > ------------------------------------------------------- > Running <package>.JnSpecBPMNProcessSpec > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.199 sec - > in <package>.JnSpecBPMNProcessSpec > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec > Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.485 sec - > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec > Running <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.471 sec - > in <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec > Running > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec > Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.975 sec - > in > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec > Running > > <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnAlreadyDeliveredOrderitemSpec > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.262 sec - > in > > <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnAlreadyDeliveredOrderitemSpec > Running > > <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnUndeliveredOrderitemSpec > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.215 sec - > in > > <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnUndeliveredOrderitemSpec > Running > > <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnAlreadyDeliveredOrderitemSpec > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.225 sec - > in > > <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnAlreadyDeliveredOrderitemSpec > Running > > <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnUndeliveredOrderitemSpec > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.028 sec - > in > > <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnUndeliveredOrderitemSpec > > Results : > > Tests run: 14, Failures: 0, Errors: 0, Skipped: 0 > > > ------------------------------------------------------- > T E S T S > ------------------------------------------------------- > > ------------------------------------------------------- > T E S T S > ------------------------------------------------------- > Running <package>.JnSpecBPMNProcessSpec$1 > Configuring TestNG with: TestNG652Configurator > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.595 sec - > in <package>.JnSpecBPMNProcessSpec$1 > Running <package>.JnSpecBPMNProcessSpec > Configuring TestNG with: TestNG652Configurator > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.693 sec - > in <package>.JnSpecBPMNProcessSpec > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$1 > Configuring TestNG with: TestNG652Configurator > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.658 sec - > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$1 > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$2$1 > Configuring TestNG with: TestNG652Configurator > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.683 sec - > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$2$1 > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$2 > Configuring TestNG with: TestNG652Configurator > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.621 sec - > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$2 > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$3 > Configuring TestNG with: TestNG652Configurator > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.534 sec - > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$3 > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$4$1$1 > Configuring TestNG with: TestNG652Configurator > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.572 sec - > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$4$1$1 > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$4$1 > Configuring TestNG with: TestNG652Configurator > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.142 sec - > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$4$1 > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$4 > Configuring TestNG with: TestNG652Configurator > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.678 sec - > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$4 > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$5$1 > Configuring TestNG with: TestNG652Configurator > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.989 sec - > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$5$1 > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$5 > Configuring TestNG with: TestNG652Configurator > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.461 sec - > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$5 > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec > Configuring TestNG with: TestNG652Configurator > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.8 sec - > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec > Running > <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$1 > Configuring TestNG with: TestNG652Configurator > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.138 sec - > in <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$1 > Running > <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$2 > Configuring TestNG with: TestNG652Configurator > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.404 sec - > in <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$2 > Running > <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$3 > Configuring TestNG with: TestNG652Configurator > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.746 sec - > in <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$3 > Running > <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$4 > Configuring TestNG with: TestNG652Configurator > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.242 sec - > in <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$4 > Running <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec > Configuring TestNG with: TestNG652Configurator > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.571 sec - > in <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec > Running > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$1 > Configuring TestNG with: TestNG652Configurator > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.28 sec - > in > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$1 > Running > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$2 > Configuring TestNG with: TestNG652Configurator > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.334 sec - > in > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$2 > Running > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$3 > Configuring TestNG with: TestNG652Configurator > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.363 sec - > in > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$3 > Running > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$4 > Configuring TestNG with: TestNG652Configurator > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.148 sec - > in > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$4 > Running > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$5 > Configuring TestNG with: TestNG652Configurator > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.506 sec - > in > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$5 > Running > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec > Configuring TestNG with: TestNG652Configurator > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.767 sec - > in > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec > Running > > <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnAlreadyDeliveredOrderitemSpec > Configuring TestNG with: TestNG652Configurator > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.883 sec - > in > > <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnAlreadyDeliveredOrderitemSpec > Running > > <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnUndeliveredOrderitemSpec > Configuring TestNG with: TestNG652Configurator > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.606 sec - > in > > <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnUndeliveredOrderitemSpec > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$1 > Configuring TestNG with: TestNG652Configurator > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.275 sec - > in <package>.JnSpecVerifyingTheCancellationMessageSpec$1 > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$2 > Configuring TestNG with: TestNG652Configurator > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.147 sec - > in <package>.JnSpecVerifyingTheCancellationMessageSpec$2 > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$3 > Configuring TestNG with: TestNG652Configurator > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.246 sec - > in <package>.JnSpecVerifyingTheCancellationMessageSpec$3 > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1$1$1$1 > Configuring TestNG with: TestNG652Configurator > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.514 sec - > in <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1$1$1$1 > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1$1$1 > Configuring TestNG with: TestNG652Configurator > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.406 sec - > in <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1$1$1 > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1$1 > Configuring TestNG with: TestNG652Configurator > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.614 sec - > in <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1$1 > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1 > Configuring TestNG with: TestNG652Configurator > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.569 sec - > in <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1 > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$4 > Configuring TestNG with: TestNG652Configurator > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.538 sec - > in <package>.JnSpecVerifyingTheCancellationMessageSpec$4 > Running <package>.JnSpecVerifyingTheCancellationMessageSpec > Configuring TestNG with: TestNG652Configurator > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.459 sec - > in <package>.JnSpecVerifyingTheCancellationMessageSpec > > Results : > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0 > > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD SUCCESS > [INFO] > ------------------------------------------------------------------------ > > > > 2013/7/24 Francesco Mari <[email protected]> > > > I tried to reproduce the problem with the version 2.14.1 of the plugin, > but > > everything works correctly. Which version of the plugin are you using? > > > > > > 2013/7/24 Andreas Dolk <[email protected]> > > > > > Sure, here we go: > > > > > > <plugin> > > > <groupId>org.apache.maven.plugins</groupId> > > > <artifactId>maven-surefire-plugin</artifactId> > > > <version>${version.maven-surefire-plugin}</version> > > > <dependencies> > > > <dependency> > > > > <groupId>org.apache.maven.surefire</groupId> > > > <artifactId>surefire-junit47</artifactId> > > > > > > <version>${version.maven-surefire-plugin}</version> > > > </dependency> > > > <dependency> > > > > <groupId>org.apache.maven.surefire</groupId> > > > <artifactId>surefire-testng</artifactId> > > > > > > <version>${version.maven-surefire-plugin}</version> > > > </dependency> > > > </dependencies> > > > <configuration> > > > <includes> > > > <include>**/*Spec*.java</include> > > > <include>**/*Feature.java</include> > > > <include>**/*Test.java</include> > > > <include>**/*TestCase.java</include> > > > </includes> > > > <forkCount>1</forkCount> > > > <reuseForks>false</reuseForks> > > > </configuration> > > > </plugin> > > > > > > > > > > > > Andreas > > > > > > > > > 2013/7/24 Francesco Mari <[email protected]> > > > > > > > Can you provide your configuration for the m-surefire-p? > > > > > > > > > > > > 2013/7/24 Andreas Dolk <[email protected]> > > > > > > > > > Hello all, > > > > > > > > > > we have maven builds for projects that use both jUnit and testNG > > tests. > > > > So > > > > > I configured the maven surefire plugin to use multiple providers ( > > > > > > > > > > > > > > > > > > > > http://maven.apache.org/surefire/maven-surefire-plugin/examples/providers.html > > > > > ) > > > > > and it works pretty well - apart from the problem, that now TestNG > > > tries > > > > to > > > > > run every single jUnit test in the project. It takes about 2 > seconds > > > for > > > > > each test until it finds out that it has nothing to to (only jUnit > > > > > annotations on the test class) > > > > > > > > > > The annoying part is that now some tests that failed with jUnit are > > > > marked > > > > > ok after TestNG didn't find a problem. > > > > > > > > > > Is there anyway to configure surefire or the providers so that they > > > only > > > > > run "their own" tests and ignore the others? A solution where > rename > > > the > > > > > test to filter by names would be fine for me. > > > > > > > > > > Best regards, > > > > > > > > > > > > > > > Andreas > > > > > > > > > > > > > > > > > > > > > -- > > > Andreas Dolk > > > > > > Zurmainerstraße 33 > > > D-54292 Trier > > > Phone「+49 651 4362884」 > > > Mobile「+49 177 4970815」 > > > EMail「[email protected]」 > > > > > > > > > -- > Andreas Dolk > > Zurmainerstraße 33 > D-54292 Trier > Phone「+49 651 4362884」 > Mobile「+49 177 4970815」 > EMail「[email protected]」 >
