But on second thought - TestNG looked at the test but actually didn't execute them. It reported "Tests run: 0, Failures: 0, Errors: 0, Skipped: 0". That should be different in "mixed mode", shouldn't it? If I could activate mixed mode correctly, then I could drop the jUnit provider anyway and have TestNG execute all test classes. That would even be a far better solution to my real problem.
I need the additional jUnit tests only because I have some test written in xtend (where the integration with TestNG is not that good in eclipse right now) and because I use jnario, which creates jUnit tests only. So my real problem is running all tests in a maven build. If it is possible with TestNG only, even better! Best regards, Andreas 2013/7/24 Andreas Dolk <[email protected]> > Ah sure, sorry: > > jUnit: 4.11 > TestNG: 6.8.5 > > And thanks for the "mixed mode" hint, I study the reference, maybe I find > something in the build files! > > Best regards, > Andreas > > > 2013/7/24 Francesco Mari <[email protected]> > >> I still wasn't able to reproduce your issue. >> >> Looks like TestNG is running in "mixed mode" [1][2]. The last missing >> information is the version of JUnit and TestNG you are using. Can you >> provide this piece of configuration? >> >> [1]: http://testng.org/doc/migrating.html >> [2]: http://testng.org/doc/documentation-main.html#junit >> >> >> 2013/7/24 Andreas Dolk <[email protected]> >> >> > Hi Francesco, >> > >> > I'm using 2.15 >> > >> > And here's the result from a test run, that's what happens. The tests >> are >> > *only* jUnit tests. I've only replaced path and package names. BTW, the >> > jUnit times are net execution times (unfair!!), testNG reports the total >> > times (fair) ;) The other annoying part is that TestNG picks up far more >> > classes then jUnit... >> > >> > The tests are auto-compiled from jnario specs (jnario.org) which >> shouldn't >> > make a difference - at the end it's classes compiled from java source >> > files. >> > >> > Regards, >> > Andreas >> > >> > >> > >> > > mvn -Dtest=JnSpec* test >> > >> > ... >> > >> > [INFO] --- maven-surefire-plugin:2.15:test (default-test) @ a42-order-be >> > --- >> > [INFO] Surefire report directory: /<path>/target/surefire-reports >> > [INFO] Using configured provider >> > org.apache.maven.surefire.junitcore.JUnitCoreProvider >> > [INFO] Using configured provider >> > org.apache.maven.surefire.testng.TestNGProvider >> > [INFO] parallel='none', perCoreThreadCount=true, threadCount=2, >> > useUnlimitedThreads=false >> > >> > ------------------------------------------------------- >> > T E S T S >> > ------------------------------------------------------- >> > >> > ------------------------------------------------------- >> > T E S T S >> > ------------------------------------------------------- >> > Running <package>.JnSpecBPMNProcessSpec >> > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.199 >> sec - >> > in <package>.JnSpecBPMNProcessSpec >> > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec >> > Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.485 >> sec - >> > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec >> > Running >> <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec >> > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.471 >> sec - >> > in <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec >> > Running >> > >> <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec >> > Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.975 >> sec - >> > in >> > >> <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec >> > Running >> > >> > >> <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnAlreadyDeliveredOrderitemSpec >> > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.262 >> sec - >> > in >> > >> > >> <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnAlreadyDeliveredOrderitemSpec >> > Running >> > >> > >> <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnUndeliveredOrderitemSpec >> > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.215 >> sec - >> > in >> > >> > >> <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnUndeliveredOrderitemSpec >> > Running >> > >> > >> <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnAlreadyDeliveredOrderitemSpec >> > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.225 >> sec - >> > in >> > >> > >> <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnAlreadyDeliveredOrderitemSpec >> > Running >> > >> > >> <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnUndeliveredOrderitemSpec >> > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.028 >> sec - >> > in >> > >> > >> <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnUndeliveredOrderitemSpec >> > >> > Results : >> > >> > Tests run: 14, Failures: 0, Errors: 0, Skipped: 0 >> > >> > >> > ------------------------------------------------------- >> > T E S T S >> > ------------------------------------------------------- >> > >> > ------------------------------------------------------- >> > T E S T S >> > ------------------------------------------------------- >> > Running <package>.JnSpecBPMNProcessSpec$1 >> > Configuring TestNG with: TestNG652Configurator >> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.595 >> sec - >> > in <package>.JnSpecBPMNProcessSpec$1 >> > Running <package>.JnSpecBPMNProcessSpec >> > Configuring TestNG with: TestNG652Configurator >> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.693 >> sec - >> > in <package>.JnSpecBPMNProcessSpec >> > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$1 >> > Configuring TestNG with: TestNG652Configurator >> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.658 >> sec - >> > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$1 >> > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$2$1 >> > Configuring TestNG with: TestNG652Configurator >> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.683 >> sec - >> > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$2$1 >> > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$2 >> > Configuring TestNG with: TestNG652Configurator >> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.621 >> sec - >> > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$2 >> > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$3 >> > Configuring TestNG with: TestNG652Configurator >> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.534 >> sec - >> > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$3 >> > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$4$1$1 >> > Configuring TestNG with: TestNG652Configurator >> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.572 >> sec - >> > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$4$1$1 >> > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$4$1 >> > Configuring TestNG with: TestNG652Configurator >> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.142 >> sec - >> > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$4$1 >> > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$4 >> > Configuring TestNG with: TestNG652Configurator >> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.678 >> sec - >> > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$4 >> > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$5$1 >> > Configuring TestNG with: TestNG652Configurator >> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.989 >> sec - >> > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$5$1 >> > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$5 >> > Configuring TestNG with: TestNG652Configurator >> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.461 >> sec - >> > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$5 >> > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec >> > Configuring TestNG with: TestNG652Configurator >> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.8 sec >> - >> > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec >> > Running >> > <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$1 >> > Configuring TestNG with: TestNG652Configurator >> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.138 >> sec - >> > in <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$1 >> > Running >> > <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$2 >> > Configuring TestNG with: TestNG652Configurator >> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.404 >> sec - >> > in <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$2 >> > Running >> > <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$3 >> > Configuring TestNG with: TestNG652Configurator >> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.746 >> sec - >> > in <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$3 >> > Running >> > <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$4 >> > Configuring TestNG with: TestNG652Configurator >> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.242 >> sec - >> > in <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$4 >> > Running >> <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec >> > Configuring TestNG with: TestNG652Configurator >> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.571 >> sec - >> > in <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec >> > Running >> > >> > >> <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$1 >> > Configuring TestNG with: TestNG652Configurator >> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.28 >> sec - >> > in >> > >> > >> <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$1 >> > Running >> > >> > >> <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$2 >> > Configuring TestNG with: TestNG652Configurator >> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.334 >> sec - >> > in >> > >> > >> <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$2 >> > Running >> > >> > >> <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$3 >> > Configuring TestNG with: TestNG652Configurator >> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.363 >> sec - >> > in >> > >> > >> <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$3 >> > Running >> > >> > >> <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$4 >> > Configuring TestNG with: TestNG652Configurator >> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.148 >> sec - >> > in >> > >> > >> <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$4 >> > Running >> > >> > >> <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$5 >> > Configuring TestNG with: TestNG652Configurator >> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.506 >> sec - >> > in >> > >> > >> <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$5 >> > Running >> > >> <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec >> > Configuring TestNG with: TestNG652Configurator >> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.767 >> sec - >> > in >> > >> <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec >> > Running >> > >> > >> <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnAlreadyDeliveredOrderitemSpec >> > Configuring TestNG with: TestNG652Configurator >> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.883 >> sec - >> > in >> > >> > >> <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnAlreadyDeliveredOrderitemSpec >> > Running >> > >> > >> <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnUndeliveredOrderitemSpec >> > Configuring TestNG with: TestNG652Configurator >> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.606 >> sec - >> > in >> > >> > >> <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnUndeliveredOrderitemSpec >> > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$1 >> > Configuring TestNG with: TestNG652Configurator >> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.275 >> sec - >> > in <package>.JnSpecVerifyingTheCancellationMessageSpec$1 >> > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$2 >> > Configuring TestNG with: TestNG652Configurator >> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.147 >> sec - >> > in <package>.JnSpecVerifyingTheCancellationMessageSpec$2 >> > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$3 >> > Configuring TestNG with: TestNG652Configurator >> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.246 >> sec - >> > in <package>.JnSpecVerifyingTheCancellationMessageSpec$3 >> > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1$1$1$1 >> > Configuring TestNG with: TestNG652Configurator >> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.514 >> sec - >> > in <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1$1$1$1 >> > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1$1$1 >> > Configuring TestNG with: TestNG652Configurator >> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.406 >> sec - >> > in <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1$1$1 >> > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1$1 >> > Configuring TestNG with: TestNG652Configurator >> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.614 >> sec - >> > in <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1$1 >> > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1 >> > Configuring TestNG with: TestNG652Configurator >> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.569 >> sec - >> > in <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1 >> > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$4 >> > Configuring TestNG with: TestNG652Configurator >> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.538 >> sec - >> > in <package>.JnSpecVerifyingTheCancellationMessageSpec$4 >> > Running <package>.JnSpecVerifyingTheCancellationMessageSpec >> > Configuring TestNG with: TestNG652Configurator >> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.459 >> sec - >> > in <package>.JnSpecVerifyingTheCancellationMessageSpec >> > >> > Results : >> > >> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0 >> > >> > [INFO] >> > ------------------------------------------------------------------------ >> > [INFO] BUILD SUCCESS >> > [INFO] >> > ------------------------------------------------------------------------ >> > >> > >> > >> > 2013/7/24 Francesco Mari <[email protected]> >> > >> > > I tried to reproduce the problem with the version 2.14.1 of the >> plugin, >> > but >> > > everything works correctly. Which version of the plugin are you using? >> > > >> > > >> > > 2013/7/24 Andreas Dolk <[email protected]> >> > > >> > > > Sure, here we go: >> > > > >> > > > <plugin> >> > > > <groupId>org.apache.maven.plugins</groupId> >> > > > <artifactId>maven-surefire-plugin</artifactId> >> > > > >> <version>${version.maven-surefire-plugin}</version> >> > > > <dependencies> >> > > > <dependency> >> > > > >> > <groupId>org.apache.maven.surefire</groupId> >> > > > >> <artifactId>surefire-junit47</artifactId> >> > > > >> > > > <version>${version.maven-surefire-plugin}</version> >> > > > </dependency> >> > > > <dependency> >> > > > >> > <groupId>org.apache.maven.surefire</groupId> >> > > > <artifactId>surefire-testng</artifactId> >> > > > >> > > > <version>${version.maven-surefire-plugin}</version> >> > > > </dependency> >> > > > </dependencies> >> > > > <configuration> >> > > > <includes> >> > > > <include>**/*Spec*.java</include> >> > > > <include>**/*Feature.java</include> >> > > > <include>**/*Test.java</include> >> > > > <include>**/*TestCase.java</include> >> > > > </includes> >> > > > <forkCount>1</forkCount> >> > > > <reuseForks>false</reuseForks> >> > > > </configuration> >> > > > </plugin> >> > > > >> > > > >> > > > >> > > > Andreas >> > > > >> > > > >> > > > 2013/7/24 Francesco Mari <[email protected]> >> > > > >> > > > > Can you provide your configuration for the m-surefire-p? >> > > > > >> > > > > >> > > > > 2013/7/24 Andreas Dolk <[email protected]> >> > > > > >> > > > > > Hello all, >> > > > > > >> > > > > > we have maven builds for projects that use both jUnit and testNG >> > > tests. >> > > > > So >> > > > > > I configured the maven surefire plugin to use multiple >> providers ( >> > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> http://maven.apache.org/surefire/maven-surefire-plugin/examples/providers.html >> > > > > > ) >> > > > > > and it works pretty well - apart from the problem, that now >> TestNG >> > > > tries >> > > > > to >> > > > > > run every single jUnit test in the project. It takes about 2 >> > seconds >> > > > for >> > > > > > each test until it finds out that it has nothing to to (only >> jUnit >> > > > > > annotations on the test class) >> > > > > > >> > > > > > The annoying part is that now some tests that failed with jUnit >> are >> > > > > marked >> > > > > > ok after TestNG didn't find a problem. >> > > > > > >> > > > > > Is there anyway to configure surefire or the providers so that >> they >> > > > only >> > > > > > run "their own" tests and ignore the others? A solution where >> > rename >> > > > the >> > > > > > test to filter by names would be fine for me. >> > > > > > >> > > > > > Best regards, >> > > > > > >> > > > > > >> > > > > > Andreas >> > > > > > >> > > > > >> > > > >> > > > >> > > > >> > > > -- >> > > > Andreas Dolk >> > > > >> > > > Zurmainerstraße 33 >> > > > D-54292 Trier >> > > > Phone「+49 651 4362884」 >> > > > Mobile「+49 177 4970815」 >> > > > EMail「[email protected]」 >> > > > >> > > >> > >> > >> > >> > -- >> > Andreas Dolk >> > >> > Zurmainerstraße 33 >> > D-54292 Trier >> > Phone「+49 651 4362884」 >> > Mobile「+49 177 4970815」 >> > EMail「[email protected]」 >> > >> > > > > -- > Andreas Dolk > > Zurmainerstraße 33 > D-54292 Trier > Phone「+49 651 4362884」 > Mobile「+49 177 4970815」 > EMail「[email protected]」 > -- Andreas Dolk Zurmainerstraße 33 D-54292 Trier Phone「+49 651 4362884」 Mobile「+49 177 4970815」 EMail「[email protected]」
