I hope it's a bug and that it's easy to fix :) Would be great if I could keep TestNG and use jnario in the same project!
Here is the ticket reference: https://jira.codehaus.org/browse/SUREFIRE-1019 Thanks! Andreas 2013/7/25 Andreas Gudian <[email protected]> > Well, I guess you found a bug. TestNG should be able to handle JUnit4 > tests, and thus the surefire TestNG provider should be able to hand over > the tests in a way that TestNG understands what we want it to do. > > Would you be so kind and file a bug at > https://jira.codehaus.org/browse/SUREFIRE? No need to provide a test case, > I already hacked one together myself. ;) > > Thanks, > Andreas > > > > 2013/7/25 Andreas Dolk <[email protected]> > > > I'm still struggling - I now think that testng really is in mixed mode > and > > tries but fails to execute the jUnit test - maybe, because they use a > > special runner. > > > > Is there any way to disable the mixed mode for a maven build (entirely)? > > How can I pass that property "testng.mixed=false" to testng without > having > > to write a testng.xml file (unless it is real trivial...) > > > > Best regards, > > Andreas > > > > > > 2013/7/24 Francesco Mari <[email protected]> > > > > > I still wasn't able to reproduce your issue. > > > > > > Looks like TestNG is running in "mixed mode" [1][2]. The last missing > > > information is the version of JUnit and TestNG you are using. Can you > > > provide this piece of configuration? > > > > > > [1]: http://testng.org/doc/migrating.html > > > [2]: http://testng.org/doc/documentation-main.html#junit > > > > > > > > > 2013/7/24 Andreas Dolk <[email protected]> > > > > > > > Hi Francesco, > > > > > > > > I'm using 2.15 > > > > > > > > And here's the result from a test run, that's what happens. The tests > > are > > > > *only* jUnit tests. I've only replaced path and package names. BTW, > the > > > > jUnit times are net execution times (unfair!!), testNG reports the > > total > > > > times (fair) ;) The other annoying part is that TestNG picks up far > > more > > > > classes then jUnit... > > > > > > > > The tests are auto-compiled from jnario specs (jnario.org) which > > > shouldn't > > > > make a difference - at the end it's classes compiled from java source > > > > files. > > > > > > > > Regards, > > > > Andreas > > > > > > > > > > > > > > > > > mvn -Dtest=JnSpec* test > > > > > > > > ... > > > > > > > > [INFO] --- maven-surefire-plugin:2.15:test (default-test) @ > > a42-order-be > > > > --- > > > > [INFO] Surefire report directory: /<path>/target/surefire-reports > > > > [INFO] Using configured provider > > > > org.apache.maven.surefire.junitcore.JUnitCoreProvider > > > > [INFO] Using configured provider > > > > org.apache.maven.surefire.testng.TestNGProvider > > > > [INFO] parallel='none', perCoreThreadCount=true, threadCount=2, > > > > useUnlimitedThreads=false > > > > > > > > ------------------------------------------------------- > > > > T E S T S > > > > ------------------------------------------------------- > > > > > > > > ------------------------------------------------------- > > > > T E S T S > > > > ------------------------------------------------------- > > > > Running <package>.JnSpecBPMNProcessSpec > > > > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.199 > > > sec - > > > > in <package>.JnSpecBPMNProcessSpec > > > > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec > > > > Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.485 > > > sec - > > > > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec > > > > Running > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec > > > > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.471 > > > sec - > > > > in <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec > > > > Running > > > > > > > > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec > > > > Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.975 > > > sec - > > > > in > > > > > > > > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec > > > > Running > > > > > > > > > > > > > > <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnAlreadyDeliveredOrderitemSpec > > > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.262 > > > sec - > > > > in > > > > > > > > > > > > > > <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnAlreadyDeliveredOrderitemSpec > > > > Running > > > > > > > > > > > > > > <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnUndeliveredOrderitemSpec > > > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.215 > > > sec - > > > > in > > > > > > > > > > > > > > <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnUndeliveredOrderitemSpec > > > > Running > > > > > > > > > > > > > > <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnAlreadyDeliveredOrderitemSpec > > > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.225 > > > sec - > > > > in > > > > > > > > > > > > > > <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnAlreadyDeliveredOrderitemSpec > > > > Running > > > > > > > > > > > > > > <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnUndeliveredOrderitemSpec > > > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.028 > > > sec - > > > > in > > > > > > > > > > > > > > <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnUndeliveredOrderitemSpec > > > > > > > > Results : > > > > > > > > Tests run: 14, Failures: 0, Errors: 0, Skipped: 0 > > > > > > > > > > > > ------------------------------------------------------- > > > > T E S T S > > > > ------------------------------------------------------- > > > > > > > > ------------------------------------------------------- > > > > T E S T S > > > > ------------------------------------------------------- > > > > Running <package>.JnSpecBPMNProcessSpec$1 > > > > Configuring TestNG with: TestNG652Configurator > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.595 > > > sec - > > > > in <package>.JnSpecBPMNProcessSpec$1 > > > > Running <package>.JnSpecBPMNProcessSpec > > > > Configuring TestNG with: TestNG652Configurator > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.693 > > > sec - > > > > in <package>.JnSpecBPMNProcessSpec > > > > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$1 > > > > Configuring TestNG with: TestNG652Configurator > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.658 > > > sec - > > > > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$1 > > > > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$2$1 > > > > Configuring TestNG with: TestNG652Configurator > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.683 > > > sec - > > > > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$2$1 > > > > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$2 > > > > Configuring TestNG with: TestNG652Configurator > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.621 > > > sec - > > > > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$2 > > > > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$3 > > > > Configuring TestNG with: TestNG652Configurator > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.534 > > > sec - > > > > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$3 > > > > Running > <package>.JnSpecCreateTheResultMessageOfACancellationSpec$4$1$1 > > > > Configuring TestNG with: TestNG652Configurator > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.572 > > > sec - > > > > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$4$1$1 > > > > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$4$1 > > > > Configuring TestNG with: TestNG652Configurator > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.142 > > > sec - > > > > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$4$1 > > > > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$4 > > > > Configuring TestNG with: TestNG652Configurator > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.678 > > > sec - > > > > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$4 > > > > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$5$1 > > > > Configuring TestNG with: TestNG652Configurator > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.989 > > > sec - > > > > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$5$1 > > > > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$5 > > > > Configuring TestNG with: TestNG652Configurator > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.461 > > > sec - > > > > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$5 > > > > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec > > > > Configuring TestNG with: TestNG652Configurator > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.8 > > sec - > > > > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec > > > > Running > > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$1 > > > > Configuring TestNG with: TestNG652Configurator > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.138 > > > sec - > > > > in > <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$1 > > > > Running > > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$2 > > > > Configuring TestNG with: TestNG652Configurator > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.404 > > > sec - > > > > in > <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$2 > > > > Running > > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$3 > > > > Configuring TestNG with: TestNG652Configurator > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.746 > > > sec - > > > > in > <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$3 > > > > Running > > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$4 > > > > Configuring TestNG with: TestNG652Configurator > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.242 > > > sec - > > > > in > <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$4 > > > > Running > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec > > > > Configuring TestNG with: TestNG652Configurator > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.571 > > > sec - > > > > in <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec > > > > Running > > > > > > > > > > > > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$1 > > > > Configuring TestNG with: TestNG652Configurator > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.28 > > sec > > > - > > > > in > > > > > > > > > > > > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$1 > > > > Running > > > > > > > > > > > > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$2 > > > > Configuring TestNG with: TestNG652Configurator > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.334 > > > sec - > > > > in > > > > > > > > > > > > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$2 > > > > Running > > > > > > > > > > > > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$3 > > > > Configuring TestNG with: TestNG652Configurator > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.363 > > > sec - > > > > in > > > > > > > > > > > > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$3 > > > > Running > > > > > > > > > > > > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$4 > > > > Configuring TestNG with: TestNG652Configurator > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.148 > > > sec - > > > > in > > > > > > > > > > > > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$4 > > > > Running > > > > > > > > > > > > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$5 > > > > Configuring TestNG with: TestNG652Configurator > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.506 > > > sec - > > > > in > > > > > > > > > > > > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$5 > > > > Running > > > > > > > > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec > > > > Configuring TestNG with: TestNG652Configurator > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.767 > > > sec - > > > > in > > > > > > > > > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec > > > > Running > > > > > > > > > > > > > > <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnAlreadyDeliveredOrderitemSpec > > > > Configuring TestNG with: TestNG652Configurator > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.883 > > > sec - > > > > in > > > > > > > > > > > > > > <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnAlreadyDeliveredOrderitemSpec > > > > Running > > > > > > > > > > > > > > <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnUndeliveredOrderitemSpec > > > > Configuring TestNG with: TestNG652Configurator > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.606 > > > sec - > > > > in > > > > > > > > > > > > > > <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnUndeliveredOrderitemSpec > > > > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$1 > > > > Configuring TestNG with: TestNG652Configurator > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.275 > > > sec - > > > > in <package>.JnSpecVerifyingTheCancellationMessageSpec$1 > > > > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$2 > > > > Configuring TestNG with: TestNG652Configurator > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.147 > > > sec - > > > > in <package>.JnSpecVerifyingTheCancellationMessageSpec$2 > > > > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$3 > > > > Configuring TestNG with: TestNG652Configurator > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.246 > > > sec - > > > > in <package>.JnSpecVerifyingTheCancellationMessageSpec$3 > > > > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1$1$1$1 > > > > Configuring TestNG with: TestNG652Configurator > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.514 > > > sec - > > > > in <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1$1$1$1 > > > > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1$1$1 > > > > Configuring TestNG with: TestNG652Configurator > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.406 > > > sec - > > > > in <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1$1$1 > > > > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1$1 > > > > Configuring TestNG with: TestNG652Configurator > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.614 > > > sec - > > > > in <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1$1 > > > > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1 > > > > Configuring TestNG with: TestNG652Configurator > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.569 > > > sec - > > > > in <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1 > > > > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$4 > > > > Configuring TestNG with: TestNG652Configurator > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.538 > > > sec - > > > > in <package>.JnSpecVerifyingTheCancellationMessageSpec$4 > > > > Running <package>.JnSpecVerifyingTheCancellationMessageSpec > > > > Configuring TestNG with: TestNG652Configurator > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.459 > > > sec - > > > > in <package>.JnSpecVerifyingTheCancellationMessageSpec > > > > > > > > Results : > > > > > > > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0 > > > > > > > > [INFO] > > > > > > ------------------------------------------------------------------------ > > > > [INFO] BUILD SUCCESS > > > > [INFO] > > > > > > ------------------------------------------------------------------------ > > > > > > > > > > > > > > > > 2013/7/24 Francesco Mari <[email protected]> > > > > > > > > > I tried to reproduce the problem with the version 2.14.1 of the > > plugin, > > > > but > > > > > everything works correctly. Which version of the plugin are you > > using? > > > > > > > > > > > > > > > 2013/7/24 Andreas Dolk <[email protected]> > > > > > > > > > > > Sure, here we go: > > > > > > > > > > > > <plugin> > > > > > > <groupId>org.apache.maven.plugins</groupId> > > > > > > > <artifactId>maven-surefire-plugin</artifactId> > > > > > > > > > <version>${version.maven-surefire-plugin}</version> > > > > > > <dependencies> > > > > > > <dependency> > > > > > > > > > > <groupId>org.apache.maven.surefire</groupId> > > > > > > > > <artifactId>surefire-junit47</artifactId> > > > > > > > > > > > > <version>${version.maven-surefire-plugin}</version> > > > > > > </dependency> > > > > > > <dependency> > > > > > > > > > > <groupId>org.apache.maven.surefire</groupId> > > > > > > > > <artifactId>surefire-testng</artifactId> > > > > > > > > > > > > <version>${version.maven-surefire-plugin}</version> > > > > > > </dependency> > > > > > > </dependencies> > > > > > > <configuration> > > > > > > <includes> > > > > > > <include>**/*Spec*.java</include> > > > > > > <include>**/*Feature.java</include> > > > > > > <include>**/*Test.java</include> > > > > > > <include>**/*TestCase.java</include> > > > > > > </includes> > > > > > > <forkCount>1</forkCount> > > > > > > <reuseForks>false</reuseForks> > > > > > > </configuration> > > > > > > </plugin> > > > > > > > > > > > > > > > > > > > > > > > > Andreas > > > > > > > > > > > > > > > > > > 2013/7/24 Francesco Mari <[email protected]> > > > > > > > > > > > > > Can you provide your configuration for the m-surefire-p? > > > > > > > > > > > > > > > > > > > > > 2013/7/24 Andreas Dolk <[email protected]> > > > > > > > > > > > > > > > Hello all, > > > > > > > > > > > > > > > > we have maven builds for projects that use both jUnit and > > testNG > > > > > tests. > > > > > > > So > > > > > > > > I configured the maven surefire plugin to use multiple > > providers > > > ( > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://maven.apache.org/surefire/maven-surefire-plugin/examples/providers.html > > > > > > > > ) > > > > > > > > and it works pretty well - apart from the problem, that now > > > TestNG > > > > > > tries > > > > > > > to > > > > > > > > run every single jUnit test in the project. It takes about 2 > > > > seconds > > > > > > for > > > > > > > > each test until it finds out that it has nothing to to (only > > > jUnit > > > > > > > > annotations on the test class) > > > > > > > > > > > > > > > > The annoying part is that now some tests that failed with > jUnit > > > are > > > > > > > marked > > > > > > > > ok after TestNG didn't find a problem. > > > > > > > > > > > > > > > > Is there anyway to configure surefire or the providers so > that > > > they > > > > > > only > > > > > > > > run "their own" tests and ignore the others? A solution where > > > > rename > > > > > > the > > > > > > > > test to filter by names would be fine for me. > > > > > > > > > > > > > > > > Best regards, > > > > > > > > > > > > > > > > > > > > > > > > Andreas > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Andreas Dolk > > > > > > > > > > > > Zurmainerstraße 33 > > > > > > D-54292 Trier > > > > > > Phone「+49 651 4362884」 > > > > > > Mobile「+49 177 4970815」 > > > > > > EMail「[email protected]」 > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Andreas Dolk > > > > > > > > Zurmainerstraße 33 > > > > D-54292 Trier > > > > Phone「+49 651 4362884」 > > > > Mobile「+49 177 4970815」 > > > > EMail「[email protected]」 > > > > > > > > > > > > > > > -- > > Andreas Dolk > > > > Zurmainerstraße 33 > > D-54292 Trier > > Phone「+49 651 4362884」 > > Mobile「+49 177 4970815」 > > EMail「[email protected]」 > > > -- Andreas Dolk Zurmainerstraße 33 D-54292 Trier Phone「+49 651 4362884」 Mobile「+49 177 4970815」 EMail「[email protected]」
