Well, I guess you found a bug. TestNG should be able to handle JUnit4 tests, and thus the surefire TestNG provider should be able to hand over the tests in a way that TestNG understands what we want it to do.
Would you be so kind and file a bug at https://jira.codehaus.org/browse/SUREFIRE? No need to provide a test case, I already hacked one together myself. ;) Thanks, Andreas 2013/7/25 Andreas Dolk <[email protected]> > I'm still struggling - I now think that testng really is in mixed mode and > tries but fails to execute the jUnit test - maybe, because they use a > special runner. > > Is there any way to disable the mixed mode for a maven build (entirely)? > How can I pass that property "testng.mixed=false" to testng without having > to write a testng.xml file (unless it is real trivial...) > > Best regards, > Andreas > > > 2013/7/24 Francesco Mari <[email protected]> > > > I still wasn't able to reproduce your issue. > > > > Looks like TestNG is running in "mixed mode" [1][2]. The last missing > > information is the version of JUnit and TestNG you are using. Can you > > provide this piece of configuration? > > > > [1]: http://testng.org/doc/migrating.html > > [2]: http://testng.org/doc/documentation-main.html#junit > > > > > > 2013/7/24 Andreas Dolk <[email protected]> > > > > > Hi Francesco, > > > > > > I'm using 2.15 > > > > > > And here's the result from a test run, that's what happens. The tests > are > > > *only* jUnit tests. I've only replaced path and package names. BTW, the > > > jUnit times are net execution times (unfair!!), testNG reports the > total > > > times (fair) ;) The other annoying part is that TestNG picks up far > more > > > classes then jUnit... > > > > > > The tests are auto-compiled from jnario specs (jnario.org) which > > shouldn't > > > make a difference - at the end it's classes compiled from java source > > > files. > > > > > > Regards, > > > Andreas > > > > > > > > > > > > > mvn -Dtest=JnSpec* test > > > > > > ... > > > > > > [INFO] --- maven-surefire-plugin:2.15:test (default-test) @ > a42-order-be > > > --- > > > [INFO] Surefire report directory: /<path>/target/surefire-reports > > > [INFO] Using configured provider > > > org.apache.maven.surefire.junitcore.JUnitCoreProvider > > > [INFO] Using configured provider > > > org.apache.maven.surefire.testng.TestNGProvider > > > [INFO] parallel='none', perCoreThreadCount=true, threadCount=2, > > > useUnlimitedThreads=false > > > > > > ------------------------------------------------------- > > > T E S T S > > > ------------------------------------------------------- > > > > > > ------------------------------------------------------- > > > T E S T S > > > ------------------------------------------------------- > > > Running <package>.JnSpecBPMNProcessSpec > > > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.199 > > sec - > > > in <package>.JnSpecBPMNProcessSpec > > > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec > > > Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.485 > > sec - > > > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec > > > Running > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec > > > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.471 > > sec - > > > in <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec > > > Running > > > > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec > > > Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.975 > > sec - > > > in > > > > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec > > > Running > > > > > > > > > <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnAlreadyDeliveredOrderitemSpec > > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.262 > > sec - > > > in > > > > > > > > > <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnAlreadyDeliveredOrderitemSpec > > > Running > > > > > > > > > <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnUndeliveredOrderitemSpec > > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.215 > > sec - > > > in > > > > > > > > > <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnUndeliveredOrderitemSpec > > > Running > > > > > > > > > <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnAlreadyDeliveredOrderitemSpec > > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.225 > > sec - > > > in > > > > > > > > > <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnAlreadyDeliveredOrderitemSpec > > > Running > > > > > > > > > <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnUndeliveredOrderitemSpec > > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.028 > > sec - > > > in > > > > > > > > > <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnUndeliveredOrderitemSpec > > > > > > Results : > > > > > > Tests run: 14, Failures: 0, Errors: 0, Skipped: 0 > > > > > > > > > ------------------------------------------------------- > > > T E S T S > > > ------------------------------------------------------- > > > > > > ------------------------------------------------------- > > > T E S T S > > > ------------------------------------------------------- > > > Running <package>.JnSpecBPMNProcessSpec$1 > > > Configuring TestNG with: TestNG652Configurator > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.595 > > sec - > > > in <package>.JnSpecBPMNProcessSpec$1 > > > Running <package>.JnSpecBPMNProcessSpec > > > Configuring TestNG with: TestNG652Configurator > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.693 > > sec - > > > in <package>.JnSpecBPMNProcessSpec > > > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$1 > > > Configuring TestNG with: TestNG652Configurator > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.658 > > sec - > > > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$1 > > > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$2$1 > > > Configuring TestNG with: TestNG652Configurator > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.683 > > sec - > > > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$2$1 > > > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$2 > > > Configuring TestNG with: TestNG652Configurator > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.621 > > sec - > > > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$2 > > > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$3 > > > Configuring TestNG with: TestNG652Configurator > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.534 > > sec - > > > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$3 > > > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$4$1$1 > > > Configuring TestNG with: TestNG652Configurator > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.572 > > sec - > > > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$4$1$1 > > > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$4$1 > > > Configuring TestNG with: TestNG652Configurator > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.142 > > sec - > > > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$4$1 > > > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$4 > > > Configuring TestNG with: TestNG652Configurator > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.678 > > sec - > > > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$4 > > > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$5$1 > > > Configuring TestNG with: TestNG652Configurator > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.989 > > sec - > > > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$5$1 > > > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$5 > > > Configuring TestNG with: TestNG652Configurator > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.461 > > sec - > > > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$5 > > > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec > > > Configuring TestNG with: TestNG652Configurator > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.8 > sec - > > > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec > > > Running > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$1 > > > Configuring TestNG with: TestNG652Configurator > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.138 > > sec - > > > in <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$1 > > > Running > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$2 > > > Configuring TestNG with: TestNG652Configurator > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.404 > > sec - > > > in <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$2 > > > Running > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$3 > > > Configuring TestNG with: TestNG652Configurator > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.746 > > sec - > > > in <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$3 > > > Running > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$4 > > > Configuring TestNG with: TestNG652Configurator > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.242 > > sec - > > > in <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$4 > > > Running > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec > > > Configuring TestNG with: TestNG652Configurator > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.571 > > sec - > > > in <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec > > > Running > > > > > > > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$1 > > > Configuring TestNG with: TestNG652Configurator > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.28 > sec > > - > > > in > > > > > > > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$1 > > > Running > > > > > > > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$2 > > > Configuring TestNG with: TestNG652Configurator > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.334 > > sec - > > > in > > > > > > > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$2 > > > Running > > > > > > > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$3 > > > Configuring TestNG with: TestNG652Configurator > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.363 > > sec - > > > in > > > > > > > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$3 > > > Running > > > > > > > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$4 > > > Configuring TestNG with: TestNG652Configurator > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.148 > > sec - > > > in > > > > > > > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$4 > > > Running > > > > > > > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$5 > > > Configuring TestNG with: TestNG652Configurator > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.506 > > sec - > > > in > > > > > > > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$5 > > > Running > > > > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec > > > Configuring TestNG with: TestNG652Configurator > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.767 > > sec - > > > in > > > > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec > > > Running > > > > > > > > > <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnAlreadyDeliveredOrderitemSpec > > > Configuring TestNG with: TestNG652Configurator > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.883 > > sec - > > > in > > > > > > > > > <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnAlreadyDeliveredOrderitemSpec > > > Running > > > > > > > > > <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnUndeliveredOrderitemSpec > > > Configuring TestNG with: TestNG652Configurator > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.606 > > sec - > > > in > > > > > > > > > <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnUndeliveredOrderitemSpec > > > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$1 > > > Configuring TestNG with: TestNG652Configurator > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.275 > > sec - > > > in <package>.JnSpecVerifyingTheCancellationMessageSpec$1 > > > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$2 > > > Configuring TestNG with: TestNG652Configurator > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.147 > > sec - > > > in <package>.JnSpecVerifyingTheCancellationMessageSpec$2 > > > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$3 > > > Configuring TestNG with: TestNG652Configurator > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.246 > > sec - > > > in <package>.JnSpecVerifyingTheCancellationMessageSpec$3 > > > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1$1$1$1 > > > Configuring TestNG with: TestNG652Configurator > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.514 > > sec - > > > in <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1$1$1$1 > > > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1$1$1 > > > Configuring TestNG with: TestNG652Configurator > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.406 > > sec - > > > in <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1$1$1 > > > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1$1 > > > Configuring TestNG with: TestNG652Configurator > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.614 > > sec - > > > in <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1$1 > > > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1 > > > Configuring TestNG with: TestNG652Configurator > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.569 > > sec - > > > in <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1 > > > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$4 > > > Configuring TestNG with: TestNG652Configurator > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.538 > > sec - > > > in <package>.JnSpecVerifyingTheCancellationMessageSpec$4 > > > Running <package>.JnSpecVerifyingTheCancellationMessageSpec > > > Configuring TestNG with: TestNG652Configurator > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.459 > > sec - > > > in <package>.JnSpecVerifyingTheCancellationMessageSpec > > > > > > Results : > > > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0 > > > > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] BUILD SUCCESS > > > [INFO] > > > > ------------------------------------------------------------------------ > > > > > > > > > > > > 2013/7/24 Francesco Mari <[email protected]> > > > > > > > I tried to reproduce the problem with the version 2.14.1 of the > plugin, > > > but > > > > everything works correctly. Which version of the plugin are you > using? > > > > > > > > > > > > 2013/7/24 Andreas Dolk <[email protected]> > > > > > > > > > Sure, here we go: > > > > > > > > > > <plugin> > > > > > <groupId>org.apache.maven.plugins</groupId> > > > > > <artifactId>maven-surefire-plugin</artifactId> > > > > > > > <version>${version.maven-surefire-plugin}</version> > > > > > <dependencies> > > > > > <dependency> > > > > > > > > <groupId>org.apache.maven.surefire</groupId> > > > > > > <artifactId>surefire-junit47</artifactId> > > > > > > > > > > <version>${version.maven-surefire-plugin}</version> > > > > > </dependency> > > > > > <dependency> > > > > > > > > <groupId>org.apache.maven.surefire</groupId> > > > > > > <artifactId>surefire-testng</artifactId> > > > > > > > > > > <version>${version.maven-surefire-plugin}</version> > > > > > </dependency> > > > > > </dependencies> > > > > > <configuration> > > > > > <includes> > > > > > <include>**/*Spec*.java</include> > > > > > <include>**/*Feature.java</include> > > > > > <include>**/*Test.java</include> > > > > > <include>**/*TestCase.java</include> > > > > > </includes> > > > > > <forkCount>1</forkCount> > > > > > <reuseForks>false</reuseForks> > > > > > </configuration> > > > > > </plugin> > > > > > > > > > > > > > > > > > > > > Andreas > > > > > > > > > > > > > > > 2013/7/24 Francesco Mari <[email protected]> > > > > > > > > > > > Can you provide your configuration for the m-surefire-p? > > > > > > > > > > > > > > > > > > 2013/7/24 Andreas Dolk <[email protected]> > > > > > > > > > > > > > Hello all, > > > > > > > > > > > > > > we have maven builds for projects that use both jUnit and > testNG > > > > tests. > > > > > > So > > > > > > > I configured the maven surefire plugin to use multiple > providers > > ( > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://maven.apache.org/surefire/maven-surefire-plugin/examples/providers.html > > > > > > > ) > > > > > > > and it works pretty well - apart from the problem, that now > > TestNG > > > > > tries > > > > > > to > > > > > > > run every single jUnit test in the project. It takes about 2 > > > seconds > > > > > for > > > > > > > each test until it finds out that it has nothing to to (only > > jUnit > > > > > > > annotations on the test class) > > > > > > > > > > > > > > The annoying part is that now some tests that failed with jUnit > > are > > > > > > marked > > > > > > > ok after TestNG didn't find a problem. > > > > > > > > > > > > > > Is there anyway to configure surefire or the providers so that > > they > > > > > only > > > > > > > run "their own" tests and ignore the others? A solution where > > > rename > > > > > the > > > > > > > test to filter by names would be fine for me. > > > > > > > > > > > > > > Best regards, > > > > > > > > > > > > > > > > > > > > > Andreas > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Andreas Dolk > > > > > > > > > > Zurmainerstraße 33 > > > > > D-54292 Trier > > > > > Phone「+49 651 4362884」 > > > > > Mobile「+49 177 4970815」 > > > > > EMail「[email protected]」 > > > > > > > > > > > > > > > > > > > > > -- > > > Andreas Dolk > > > > > > Zurmainerstraße 33 > > > D-54292 Trier > > > Phone「+49 651 4362884」 > > > Mobile「+49 177 4970815」 > > > EMail「[email protected]」 > > > > > > > > > -- > Andreas Dolk > > Zurmainerstraße 33 > D-54292 Trier > Phone「+49 651 4362884」 > Mobile「+49 177 4970815」 > EMail「[email protected]」 >
