Re: Maven 2/3 and Cobetura plugin with both TestNG and JUnit tests (with Surefire plugin configuration)
The ability to run multiple providers within a single execution is a
fairly new feature, and
I'm not surprised that works as expected.
There are hundreds of blog posts out there that explain how to run
testng/junit using multiple executions of surefire,
which was the best available method at the time these were written.
Unfortunately they outdated as the new
method you are using is much better.
I'll see if I can include a sentence or two about this in the docs
somewhere, maybe
http://maven.apache.org/plugins/maven-surefire-plugin/examples/providers.html
(Actually I'm thinking the docs need some restructuring...)
Kristian
On Tue, Aug 2, 2011 at 8:18 PM, Larry Shatzer, Jr. lar...@gmail.com wrote:
One last update. If I remove the execution sections that tell
Surefire to run both TestNG and Junit, and put dependencies on the
Surefire Plugin like so:
dependencies
dependency
groupIdorg.apache.maven.surefire/groupId
artifactIdsurefire-junit4/artifactId
version${surefire.version}/version
/dependency
dependency
groupIdorg.apache.maven.surefire/groupId
artifactIdsurefire-testng/artifactId
version${surefire.version}/version
/dependency
/dependencies
I get the results I expect.
---
T E S T S
---
Running com.company.JUnitTest
JUnit
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.037 sec
Flushing results...
Flushing results done
Cobertura: Loaded information on 1 classes.
Cobertura: Saved information on 1 classes.
Results :
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
---
T E S T S
---
Running TestSuite
TestNG
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.311 sec
Flushing results...
Flushing results done
Cobertura: Loaded information on 1 classes.
Cobertura: Saved information on 1 classes.
Results :
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org
-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org
Re: Maven 2/3 and Cobetura plugin with both TestNG and JUnit tests (with Surefire plugin configuration)
On Mon, Aug 1, 2011 at 9:56 AM, Larry Shatzer, Jr. lar...@gmail.com wrote: I want to run with both JUnit and TestNG. By removing JUnit (and the tests that JUnit run), it ran just fine, as do the inverse, just JUnit and no TestNG. It is the combination of both of them. To help, instead of having gists, I created a github project: https://github.com/larrys/CoverageBug which you can fetch and run locally and see exactly what I'm talking about and be able to possibly tweak or debug any issues that I lack the knowledge to do. I removed cobertura out of the picture and ran emma instead, and produced similar results. --- T E S T S --- Running com.company.JUnitTest Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec Running com.company.TestNGTest Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec Results : Tests run: 0, Failures: 0, Errors: 0, Skipped: 0 If I just run the test lifecycle, it will execute the tests. Somehow a plugin that calls the test lifecycle in Maven 3 when both TestNG and JUnit are configured to run have issues. I also tried the latest surefire plugin snapshot build (built locally), and it has no effect. So I started going through various versions of Surefire. 2.7.1, 2.7, 2.6, 2.5, 2.4.3, 2.4.2, and 2.4.1 had similar output as below (ran test phase 3 times, but still did not execute tests!): --- T E S T S --- Running com.company.JUnitTest Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec Running com.company.TestNGTest Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec There are no tests to run. Results : Tests run: 0, Failures: 0, Errors: 0, Skipped: 0 [INFO] [INFO] --- maven-surefire-plugin:2.7.1:test (test-junit) @ coveragebug --- [INFO] Surefire report directory: C:\development\coverage-bug\coveragebug\target\surefire-reports --- T E S T S --- Running com.company.JUnitTest Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec Running com.company.TestNGTest Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec There are no tests to run. Results : Tests run: 0, Failures: 0, Errors: 0, Skipped: 0 [INFO] [INFO] --- maven-surefire-plugin:2.7.1:test (test-testng) @ coveragebug --- [INFO] Surefire report directory: C:\development\coverage-bug\coveragebug\target\surefire-reports --- T E S T S --- Running com.company.JUnitTest Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec Running com.company.TestNGTest Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec There are no tests to run. Results : Tests run: 0, Failures: 0, Errors: 0, Skipped: 0 When I get to 2.3.1: --- T E S T S --- There are no tests to run. Results : Tests run: 0, Failures: 0, Errors: 0, Skipped: 0 [INFO] [INFO] --- maven-surefire-plugin:2.3.1:test (test-junit) @ coveragebug --- [INFO] Surefire report directory: C:\development\coverage-bug\coveragebug\target\surefire-reports --- T E S T S --- There are no tests to run. Results : Tests run: 0, Failures: 0, Errors: 0, Skipped: 0 [INFO] [INFO] --- maven-surefire-plugin:2.3.1:test (test-testng) @ coveragebug --- [INFO] Surefire report directory: C:\development\coverage-bug\coveragebug\target\surefire-reports --- T E S T S --- There are no tests to run. Results : Tests run: 0, Failures: 0, Errors: 0, Skipped: 0 It does it three times, but does not actually show the test classes it is supposed to try to be running. 2.3 has issues with TestNG: --- T E S T S --- org.apache.maven.surefire.booter.SurefireExecutionException: org.testng.xml.XmlSuite.setParallel(Z)V; nested exception is java .lang.NoSuchMethodError: org.testng.xml.XmlSuite.setParallel(Z)V java.lang.NoSuchMethodError: org.testng.xml.XmlSuite.setParallel(Z)V at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:123) at org.apache.maven.surefire.Surefire.run(Surefire.java:132) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
Re: Maven 2/3 and Cobetura plugin with both TestNG and JUnit tests (with Surefire plugin configuration)
One last update. If I remove the execution sections that tell
Surefire to run both TestNG and Junit, and put dependencies on the
Surefire Plugin like so:
dependencies
dependency
groupIdorg.apache.maven.surefire/groupId
artifactIdsurefire-junit4/artifactId
version${surefire.version}/version
/dependency
dependency
groupIdorg.apache.maven.surefire/groupId
artifactIdsurefire-testng/artifactId
version${surefire.version}/version
/dependency
/dependencies
I get the results I expect.
---
T E S T S
---
Running com.company.JUnitTest
JUnit
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.037 sec
Flushing results...
Flushing results done
Cobertura: Loaded information on 1 classes.
Cobertura: Saved information on 1 classes.
Results :
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
---
T E S T S
---
Running TestSuite
TestNG
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.311 sec
Flushing results...
Flushing results done
Cobertura: Loaded information on 1 classes.
Cobertura: Saved information on 1 classes.
Results :
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org
RE: Maven 2/3 and Cobetura plugin with both TestNG and JUnit tests (with Surefire plugin configuration)
org.testng.xml.XmlSuite.setParallel(Z)Vparameter is string and returns void
go here
http://testng.org/doc/download.html
and get the 6.1.1 testng
/* Note this method in XMLSuite */
/**
* Sets the parallel mode
* @param parallel the parallel mode
*/
public void setParallel(String parallel) {
m_parallel = parallel;
}
/* usable pom.xml */
?xml version=1.0 encoding=UTF-8?
!--
The MIT License
Copyright (c) 2009 codehaus.org.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the Software), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED AS IS, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
--
project xmlns=http://maven.apache.org/POM/4.0.0;
xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance;
xsi:schemaLocation=http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd;
modelVersion4.0.0/modelVersion
groupIdorg.testng/groupId
artifactIdtestng/artifactId
version6.1.1/version
packagingjar/packaging
dependencies
dependency
groupIdcom.google.inject/groupId
artifactIdguice/artifactId
version2.0/version
/dependency
dependency
groupIdbsh/groupId
artifactIdbsh/artifactId
version1.3.0/version
/dependency
dependency
groupIdorg.apache.tools/groupId
artifactIdant/artifactId
version1.8.0/version
/dependency
dependency
groupIdcom.beust/groupId
artifactIdjcommander/artifactId
version1.5/version
/dependency
dependency
groupIdorg.yaml/groupId
artifactIdsnakeyaml/artifactId
version1.4/version
/dependency
dependency
groupIdjunit.framework/groupId
artifactIdjunit/artifactId
version4.7/version
/dependency
/dependencies
/project
make this mod to Suite3
public class Suite3 //{
extends junit.framework.TestCase { /*MCG */
include MIT License so we dont upset benson
Martin --
__
Put longwinded disclaimer here
Date: Tue, 2 Aug 2011 10:57:16 -0600
Subject: Re: Maven 2/3 and Cobetura plugin with both TestNG and JUnit tests
(with Surefire plugin configuration)
From: lar...@gmail.com
To: dev@maven.apache.org
On Mon, Aug 1, 2011 at 9:56 AM, Larry Shatzer, Jr. lar...@gmail.com wrote:
I want to run with both JUnit and TestNG. By removing JUnit (and the
tests that JUnit run), it ran just fine, as do the inverse, just JUnit
and no TestNG. It is the combination of both of them.
To help, instead of having gists, I created a github project:
https://github.com/larrys/CoverageBug which you can fetch and run
locally and see exactly what I'm talking about and be able to possibly
tweak or debug any issues that I lack the knowledge to do.
I removed cobertura out of the picture and ran emma instead, and
produced similar results.
---
T E S T S
---
Running com.company.JUnitTest
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec
Running com.company.TestNGTest
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec
Results :
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
If I just run the test lifecycle, it will execute the tests. Somehow a
plugin that calls the test lifecycle in Maven 3 when both TestNG and
JUnit are configured to run have issues. I also tried the latest
surefire plugin snapshot build (built locally), and it has no effect.
So I started going through various versions of Surefire.
2.7.1, 2.7, 2.6, 2.5, 2.4.3, 2.4.2, and 2.4.1 had similar output as
below (ran test phase 3 times, but still did not execute tests!):
---
T E S T S
---
Running com.company.JUnitTest
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec
Running com.company.TestNGTest
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec
There are no tests to run.
Results :
Tests run: 0,
Re: Maven 2/3 and Cobetura plugin with both TestNG and JUnit tests (with Surefire plugin configuration)
On Thu, Jul 28, 2011 at 6:04 AM, Martin Gainty mgai...@hotmail.com wrote:
Larry-
try loading both dependencies in your local repository and running offline
with mvn -o (at least your dependencies will be found)
did you check cobertura for errors?
mvn -e -X cobertura:cobertura -Dquiet=true
I did not see any obvious errors.
take junit out of the mix and run testng as solo Test dependency then check
output folders
${project.reporting.outputDirectory}/cobertura.
if these tests fail you may have found a bug in which cause you need to file
a JIRA at
http://jira.codehaus.org/browse/MCOBERTURA
I want to run with both JUnit and TestNG. By removing JUnit (and the
tests that JUnit run), it ran just fine, as do the inverse, just JUnit
and no TestNG. It is the combination of both of them.
To help, instead of having gists, I created a github project:
https://github.com/larrys/CoverageBug which you can fetch and run
locally and see exactly what I'm talking about and be able to possibly
tweak or debug any issues that I lack the knowledge to do.
Thanks
-- Larry
-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org
RE: Maven 2/3 and Cobetura plugin with both TestNG and JUnit tests (with Surefire plugin configuration)
Larry-
try loading both dependencies in your local repository and running offline with
mvn -o (at least your dependencies will be found)
did you check cobertura for errors?
mvn -e -X cobertura:cobertura -Dquiet=true
take junit out of the mix and run testng as solo Test dependency then check
output folders
${project.reporting.outputDirectory}/cobertura.
if these tests fail you may have found a bug in which cause you need to file a
JIRA at
http://jira.codehaus.org/browse/MCOBERTURA
hth
Martin-
__
Jogi és Bizalmassági kinyilatkoztatás/Verzicht und
Vertraulichkeitanmerkung/Note de déni et de confidentialité
Ez az
üzenet bizalmas. Ha nem ön az akinek szánva volt, akkor kérjük, hogy
jelentse azt nekünk vissza. Semmiféle továbbítása vagy másolatának
készítése nem megengedett. Ez az üzenet csak ismeret cserét szolgál és
semmiféle jogi alkalmazhatósága sincs. Mivel az electronikus üzenetek
könnyen megváltoztathatóak, ezért minket semmi felelöség nem terhelhet
ezen üzenet tartalma miatt.
Diese Nachricht ist vertraulich. Sollten Sie nicht der vorgesehene Empfaenger
sein, so bitten wir hoeflich um eine Mitteilung. Jede unbefugte Weiterleitung
oder Fertigung einer Kopie ist unzulaessig. Diese Nachricht dient lediglich dem
Austausch von Informationen und entfaltet keine rechtliche Bindungswirkung.
Aufgrund der leichten Manipulierbarkeit von E-Mails koennen wir keine Haftung
fuer den Inhalt uebernehmen.
Ce message est confidentiel et peut être privilégié. Si vous n'êtes pas le
destinataire prévu, nous te demandons avec bonté que pour satisfaire informez
l'expéditeur. N'importe quelle diffusion non autorisée ou la copie de ceci est
interdite. Ce message sert à l'information seulement et n'aura pas n'importe
quel effet légalement obligatoire. Étant donné que les email peuvent facilement
être sujets à la manipulation, nous ne pouvons accepter aucune responsabilité
pour le contenu fourni.
Date: Wed, 27 Jul 2011 19:32:43 -0600
Subject: Maven 2/3 and Cobetura plugin with both TestNG and JUnit tests (with
Surefire plugin configuration)
From: lar...@gmail.com
To: dev@maven.apache.org
I did send this to the users list, but got no response in over a week.
I know dev lists are not a magical escalation path, but this might be
a better venue for this email.
I have an example project at https://gist.github.com/1090223
This project has both a JUnit test and a TestNG test. Following some
instructions online to get surefire be able to run both I added lines
19-45 in the pom.
This works fine when running mvn test, except it runs the TestNG
tests twice. I can fix that by commenting out lines 34-43.
Now when I run mvn cobertura:cobertura, this is where things get weird.
In Maven 2.2.1, it runs both TestNG and JUnit just fine, and produces
the correct coverage.
Now in Maven 3.0.3 it just runs the JUnit tests. (With lines 34-43
still commented out). When I bring those back, this is the output:
---
T E S T S
---
Running com.company.JUnitTest
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec
Running com.company.TestNGTest
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.002 sec
There are no tests to run.
Results :
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
So it appears to try to run both tests, but fails to actually run any
tests, and gets 0% coverage.
It appears there is something odd between Maven 2.2.1 and Maven 3.0.3
when the cobertura plugin runs, and its reading of the Surefire plugin
configuration.
Ideally I would only use JUnit or TestNG, and this would not be a
problem. However, we are using TestNG exclusively, and wanted to
introduce a tool that currently only is executed as a JUnit test
(Spock). I wanted to avoid changing all our tests from TestNG to JUnit
unless absolutely necessary.
It appears to be a regression from Maven 2.2.1 to Maven 3.0.3, unless
it was a bug in Maven 2 that was fixed.
-- Larry
-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org
Maven 2/3 and Cobetura plugin with both TestNG and JUnit tests (with Surefire plugin configuration)
I did send this to the users list, but got no response in over a week. I know dev lists are not a magical escalation path, but this might be a better venue for this email. I have an example project at https://gist.github.com/1090223 This project has both a JUnit test and a TestNG test. Following some instructions online to get surefire be able to run both I added lines 19-45 in the pom. This works fine when running mvn test, except it runs the TestNG tests twice. I can fix that by commenting out lines 34-43. Now when I run mvn cobertura:cobertura, this is where things get weird. In Maven 2.2.1, it runs both TestNG and JUnit just fine, and produces the correct coverage. Now in Maven 3.0.3 it just runs the JUnit tests. (With lines 34-43 still commented out). When I bring those back, this is the output: --- T E S T S --- Running com.company.JUnitTest Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec Running com.company.TestNGTest Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.002 sec There are no tests to run. Results : Tests run: 0, Failures: 0, Errors: 0, Skipped: 0 So it appears to try to run both tests, but fails to actually run any tests, and gets 0% coverage. It appears there is something odd between Maven 2.2.1 and Maven 3.0.3 when the cobertura plugin runs, and its reading of the Surefire plugin configuration. Ideally I would only use JUnit or TestNG, and this would not be a problem. However, we are using TestNG exclusively, and wanted to introduce a tool that currently only is executed as a JUnit test (Spock). I wanted to avoid changing all our tests from TestNG to JUnit unless absolutely necessary. It appears to be a regression from Maven 2.2.1 to Maven 3.0.3, unless it was a bug in Maven 2 that was fixed. -- Larry - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
