Re: Can cobertura and surefire-report plugins be run without running the tests twice?
Thank you for pointing out to that discussion. Even though I don't agree completely - if running the coverage tool Cobertura changes the status of your tests, I'd say that in most likelihood the problem should be cobertura and not your code or your test - I might live with running the cobertura goal separately and even move it to the CI system: it delays quick feedback to the developer locally anyway. Miguel Almeida On Mon, Sep 26, 2011 at 4:08 PM, Stephen Connolly stephen.alan.conno...@gmail.com wrote: http://maven.40175.n5.nabble.com/Cobertura-and-Surefire-td3338334.html On 26 September 2011 15:46, Miguel Almeida migueldealme...@gmail.com wrote: I am using the cobertura plugin to create test coverage reports, and surefire-report for test reports. See [1] for my maven 3 configuration on the top-tier of a multi-module project. The goals I am running are: Surefire: surefire-report:report Cobertura: cobertura:cobertura Is it possible to run the both reports in the same run *withou*t *running the tests twice*? I have tried two approaches: 1) mvn both goals 2) Site: site:site However, because site:site ran for too long (more than 2 minutes, vs a 30 second surefire-report), I inspected its output, and found out it was running each test twice (eg, I found two string matches of Running org.iwrs.web.IndexActionTest). To reduce total goal time, can *tests run once and produce surefire and cobertura reports*? Thank you, Miguel Almeida [1] plugins plugin groupIdorg.apache.maven.plugins/groupId artifactIdmaven-site-plugin/artifactId version3.0-beta-3/version configuration reportPlugins plugin groupIdorg.apache.maven.plugins/groupId artifactIdmaven-javadoc-plugin/artifactId version2.7/version /plugin plugin groupIdorg.apache.maven.plugins/groupId artifactIdmaven-surefire-report-plugin/artifactId version2.6/version /plugin plugin groupIdorg.codehaus.mojo/groupId artifactIdcobertura-maven-plugin/artifactId configuration instrumentation excludes excludecom/work/**/*Fake*.class/exclude /excludes /instrumentation /configuration /plugin /reportPlugins /configuration /plugin - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Can cobertura and surefire-report plugins be run without running the tests twice?
In my experience you have: * a 40% chance of the failure being a bad test * a 40% chance of the failure being a bug in your production code * a 20% chance of the failure being a side-effect of instrumentation which invalidates the test If you and all the developers in your team cannot explain what will be output by the example code on page 33 of Java Concurrency In Practice ( http://jcip.net/listings/NoVisibility.java ) and why then please please Run the Damn Tests Twice If you can explain, then you will run the damn tests twice anyway. The above example is just one way where the JVM spec can confuse you... There are others which have nothing to do with concurrency, but I don't have handy links to them. Just Run The Damn Tests Twice(TM) -Stephen On 27 September 2011 12:14, Miguel Almeida migueldealme...@gmail.com wrote: Thank you for pointing out to that discussion. Even though I don't agree completely - if running the coverage tool Cobertura changes the status of your tests, I'd say that in most likelihood the problem should be cobertura and not your code or your test - I might live with running the cobertura goal separately and even move it to the CI system: it delays quick feedback to the developer locally anyway. Miguel Almeida On Mon, Sep 26, 2011 at 4:08 PM, Stephen Connolly stephen.alan.conno...@gmail.com wrote: http://maven.40175.n5.nabble.com/Cobertura-and-Surefire-td3338334.html On 26 September 2011 15:46, Miguel Almeida migueldealme...@gmail.com wrote: I am using the cobertura plugin to create test coverage reports, and surefire-report for test reports. See [1] for my maven 3 configuration on the top-tier of a multi-module project. The goals I am running are: Surefire: surefire-report:report Cobertura: cobertura:cobertura Is it possible to run the both reports in the same run *withou*t *running the tests twice*? I have tried two approaches: 1) mvn both goals 2) Site: site:site However, because site:site ran for too long (more than 2 minutes, vs a 30 second surefire-report), I inspected its output, and found out it was running each test twice (eg, I found two string matches of Running org.iwrs.web.IndexActionTest). To reduce total goal time, can *tests run once and produce surefire and cobertura reports*? Thank you, Miguel Almeida [1] plugins plugin groupIdorg.apache.maven.plugins/groupId artifactIdmaven-site-plugin/artifactId version3.0-beta-3/version configuration reportPlugins plugin groupIdorg.apache.maven.plugins/groupId artifactIdmaven-javadoc-plugin/artifactId version2.7/version /plugin plugin groupIdorg.apache.maven.plugins/groupId artifactIdmaven-surefire-report-plugin/artifactId version2.6/version /plugin plugin groupIdorg.codehaus.mojo/groupId artifactIdcobertura-maven-plugin/artifactId configuration instrumentation excludes excludecom/work/**/*Fake*.class/exclude /excludes /instrumentation /configuration /plugin /reportPlugins /configuration /plugin - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Can cobertura and surefire-report plugins be run without running the tests twice?
I am using the cobertura plugin to create test coverage reports, and surefire-report for test reports. See [1] for my maven 3 configuration on the top-tier of a multi-module project. The goals I am running are: Surefire: surefire-report:report Cobertura: cobertura:cobertura Is it possible to run the both reports in the same run *withou*t *running the tests twice*? I have tried two approaches: 1) mvn both goals 2) Site: site:site However, because site:site ran for too long (more than 2 minutes, vs a 30 second surefire-report), I inspected its output, and found out it was running each test twice (eg, I found two string matches of Running org.iwrs.web.IndexActionTest). To reduce total goal time, can *tests run once and produce surefire and cobertura reports*? Thank you, Miguel Almeida [1] plugins plugin groupIdorg.apache.maven.plugins/groupId artifactIdmaven-site-plugin/artifactId version3.0-beta-3/version configuration reportPlugins plugin groupIdorg.apache.maven.plugins/groupId artifactIdmaven-javadoc-plugin/artifactId version2.7/version /plugin plugin groupIdorg.apache.maven.plugins/groupId artifactIdmaven-surefire-report-plugin/artifactId version2.6/version /plugin plugin groupIdorg.codehaus.mojo/groupId artifactIdcobertura-maven-plugin/artifactId configuration instrumentation excludes excludecom/work/**/*Fake*.class/exclude /excludes /instrumentation /configuration /plugin /reportPlugins /configuration /plugin
Re: Can cobertura and surefire-report plugins be run without running the tests twice?
http://maven.40175.n5.nabble.com/Cobertura-and-Surefire-td3338334.html On 26 September 2011 15:46, Miguel Almeida migueldealme...@gmail.com wrote: I am using the cobertura plugin to create test coverage reports, and surefire-report for test reports. See [1] for my maven 3 configuration on the top-tier of a multi-module project. The goals I am running are: Surefire: surefire-report:report Cobertura: cobertura:cobertura Is it possible to run the both reports in the same run *withou*t *running the tests twice*? I have tried two approaches: 1) mvn both goals 2) Site: site:site However, because site:site ran for too long (more than 2 minutes, vs a 30 second surefire-report), I inspected its output, and found out it was running each test twice (eg, I found two string matches of Running org.iwrs.web.IndexActionTest). To reduce total goal time, can *tests run once and produce surefire and cobertura reports*? Thank you, Miguel Almeida [1] plugins plugin groupIdorg.apache.maven.plugins/groupId artifactIdmaven-site-plugin/artifactId version3.0-beta-3/version configuration reportPlugins plugin groupIdorg.apache.maven.plugins/groupId artifactIdmaven-javadoc-plugin/artifactId version2.7/version /plugin plugin groupIdorg.apache.maven.plugins/groupId artifactIdmaven-surefire-report-plugin/artifactId version2.6/version /plugin plugin groupIdorg.codehaus.mojo/groupId artifactIdcobertura-maven-plugin/artifactId configuration instrumentation excludes excludecom/work/**/*Fake*.class/exclude /excludes /instrumentation /configuration /plugin /reportPlugins /configuration /plugin - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Cobertura and Surefire
Jeff Jensen wrote: Sounds nice, but this doesn't meet my requirement, that the tests with coverage-checks (and only one time, not twice) should run, when the developers do mvn test. There is no acceptable solution to this requirement. Many of us would like to not run them twice, and in actuality running them twice is not a problem for things like site gen. Thank you for this information, that it's currently simply not possible. I think that's a shortcoming of cobertura-maven-plugin. It should be possible to overwrite the default surefire-configuration within the cobertura-configuration. But for now I have to look for another way (thank you and the others for the suggestions) we unfortunately don't use a regular CI-server but a bunch of shellscripts and cron-jobs. So it's not as trivial as with Hudson (Jenkins?) to use this kind of build-pipeline. Looks like it is time to invest a couple of days and set it up! Your work life will be easier and maintenance cheaper. I completely aggree with you and I'd love to replace the scripts by a neat Hudson-installation. But the only way to do this (or the only way I can imagine) is to create a buildjob which simply calls our current master-buildscript. But I don't think I improved anything by doing this. We can't use the most of hudson out of the box. Even the updating of the working-copy is not trivial: not only that we use CM Synergy (little support for it in Hudson and no public Java-API), but we need a combination of CM Synergy and Change Synergy to calculate the tasks to update. I would really appreciate to migrate the build of the last 100 modules to Maven without the need of specialized builds for each target-stage, using a mainstream SCM-tool and so on... Unfortunately I think the client wouldn't pay for this work (not to forget the QA of the whole application). But I think, I go off on a tangent. -- NEU: FreePhone - kostenlos mobil telefonieren und surfen! Jetzt informieren: http://www.gmx.net/de/go/freephone - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Cobertura and Surefire
Stephen Connolly wrote: Because people who have not read and understood concurrency in practice often do not understand how the synchronization points affect jvm sequencing, people often wrongly suspect that result of instrumented and non-instrumented code is the same. I'm not sure, why caring about concurrency is important in this context. I'm not using Maven 3 (so no concurrent builds) and I'm not using concurrent tests with JUnit. But it's quite plain to me it is possible, that instrumented code behaves different to uninstrumented code, so testing the instrumented code can pass the tests while uninstrumented fail. (i.e. because of attributes added by instrumentation, which are unexpected when working with reflection on these attributes) But I think it doesn't happen on a regular basis, that the instrumented code pass or fails the tests, where the uninstrumented doesn't. But sadly it's more regular in our team, that the Cobertura-step fails (not because failing tests but because of too low coverage). Wayne Fay wrote: Perhaps bind something to your scm commit hook so the code simply can't make it into your SCM tool without meeting certain standards for coverage? That would mean, that during the commit we should run all the tests? I'm not sure, I want the commit to last about two minutes. Beside this, we have to use Synergy/CM as SCM tool, which is not capable of pre-commit-hooks (as far as I know). Hilco Wijbenga wrote: A normal unit test should run once, more is just a waste of time. The fact that Maven will run unit tests multiple times is a bug that we apparently have to live with. It is not a feature. If this bug were fixed and you still wanted to run your unit tests twice then you could configure that quite easily in your POM. Everybody happy I don't really like that Maven needs to run tests twice (or more often), too. But I see it is neccessary regarding Cobertura, because -- as stated to Stephen -- it can't be guaranteed, that the instrumented code behaves the same as the uninstrumented. Because of this it is not sufficient to run the tests only on the instrumented code. So this is not a bug in Maven but a problem with Cobertura. -- GMX DSL Doppel-Flat ab 19,99 Euro/mtl.! Jetzt mit gratis Handy-Flat! http://portal.gmx.net/de/go/dsl - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Cobertura and Surefire
On 13 January 2011 09:50, Stefan Schulze algr...@gmx.de wrote: Stephen Connolly wrote: Because people who have not read and understood concurrency in practice often do not understand how the synchronization points affect jvm sequencing, people often wrongly suspect that result of instrumented and non-instrumented code is the same. I'm not sure, why caring about concurrency is important in this context. I'm not using Maven 3 (so no concurrent builds) and I'm not using concurrent tests with JUnit. Between synchronization points, the JVM is allowed to re-order operations as long as the end result remains the same. Cobertura puts a synchronization point in between every line and branch point. Thus the re-ordering of lines cannot take place any more. Start up a JVM and have a look at how many threads are running Still think you are in a single threaded world? Also some simple libraries use threads behind your back... and even the Java Runtime Libraries do that too... Still think you don't have to know/worry about concurrency? I shall repeat... We had some appears to be simple single threaded code that we were testing with a simple test case. The tests were only being run instrumented... the tests were passing the bug was there... seen in production system by big customer... in the end of the day, this was identified as the original author not understanding about the JVM's contract w.r.t. re-ordering operations in-between synchronization points and the fact that using cobertura introduced a synchronization point at every line so that the code as written could only work when instrumented. Run the damn tests at least twice. (I'd like to say run them on multiple JVMs too but Oracle seems intent on merging all the JVMs into openjdk) -Stephen But it's quite plain to me it is possible, that instrumented code behaves different to uninstrumented code, so testing the instrumented code can pass the tests while uninstrumented fail. (i.e. because of attributes added by instrumentation, which are unexpected when working with reflection on these attributes) But I think it doesn't happen on a regular basis, that the instrumented code pass or fails the tests, where the uninstrumented doesn't. But sadly it's more regular in our team, that the Cobertura-step fails (not because failing tests but because of too low coverage). It happens when you least expect it. Wayne Fay wrote: Perhaps bind something to your scm commit hook so the code simply can't make it into your SCM tool without meeting certain standards for coverage? That would mean, that during the commit we should run all the tests? I'm not sure, I want the commit to last about two minutes. Beside this, we have to use Synergy/CM as SCM tool, which is not capable of pre-commit-hooks (as far as I know). Hilco Wijbenga wrote: A normal unit test should run once, more is just a waste of time. The fact that Maven will run unit tests multiple times is a bug that we apparently have to live with. It is not a feature. If this bug were fixed and you still wanted to run your unit tests twice then you could configure that quite easily in your POM. Everybody happy I don't really like that Maven needs to run tests twice (or more often), too. But I see it is neccessary regarding Cobertura, because -- as stated to Stephen -- it can't be guaranteed, that the instrumented code behaves the same as the uninstrumented. Because of this it is not sufficient to run the tests only on the instrumented code. So this is not a bug in Maven but a problem with Cobertura. -- GMX DSL Doppel-Flat ab 19,99 Euro/mtl.! Jetzt mit gratis Handy-Flat! http://portal.gmx.net/de/go/dsl - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Cobertura and Surefire
Stephen Connolly wrote: [...] Run the damn tests at least twice. Ok, I see your point. But I never tried to run the tests only instrumented. I just want to execute the more-likely-failing tests earlier in the lifecycle and the not-so-likely-failing tests later. So of course I want to execute both uninstrumented _and_ instrumented tests. -- Neu: GMX De-Mail - Einfach wie E-Mail, sicher wie ein Brief! Jetzt De-Mail-Adresse reservieren: http://portal.gmx.net/de/go/demail - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Cobertura and Surefire
IMO solution is simple - discipline your developers to run verify before commiting. CI should help you determine whom to blame when build is broken and then you can apply disciplinary meassures. Regards, Stevo. On Thu, Jan 13, 2011 at 11:24 AM, Stefan Schulze algr...@gmx.de wrote: Stephen Connolly wrote: [...] Run the damn tests at least twice. Ok, I see your point. But I never tried to run the tests only instrumented. I just want to execute the more-likely-failing tests earlier in the lifecycle and the not-so-likely-failing tests later. So of course I want to execute both uninstrumented _and_ instrumented tests. -- Neu: GMX De-Mail - Einfach wie E-Mail, sicher wie ein Brief! Jetzt De-Mail-Adresse reservieren: http://portal.gmx.net/de/go/demail - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Cobertura and Surefire
Perhaps a different CI job structuring? This has worked well for me at many customers: Have a CI job for only compile and unit tests - this maintains the most important very fast turnaround. Have a second CI job for (longer running) IT tests that only runs with success of the first. Have a third job for coverage, standalone or as part of a full system site gen or using Sonar or other... On Thu, Jan 13, 2011 at 5:14 AM, Stevo Slavić ssla...@gmail.com wrote: IMO solution is simple - discipline your developers to run verify before commiting. CI should help you determine whom to blame when build is broken and then you can apply disciplinary meassures. Regards, Stevo. On Thu, Jan 13, 2011 at 11:24 AM, Stefan Schulze algr...@gmx.de wrote: Stephen Connolly wrote: [...] Run the damn tests at least twice. Ok, I see your point. But I never tried to run the tests only instrumented. I just want to execute the more-likely-failing tests earlier in the lifecycle and the not-so-likely-failing tests later. So of course I want to execute both uninstrumented _and_ instrumented tests. -- Neu: GMX De-Mail - Einfach wie E-Mail, sicher wie ein Brief! Jetzt De-Mail-Adresse reservieren: http://portal.gmx.net/de/go/demail - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Cobertura and Surefire
Jeff Jensen wrote: Perhaps a different CI job structuring? This has worked well for me at many customers: Have a CI job for only compile and unit tests - this maintains the most important very fast turnaround. Have a second CI job for (longer running) IT tests that only runs with success of the first. Have a third job for coverage, standalone or as part of a full system site gen or using Sonar or other... Sounds nice, but this doesn't meet my requirement, that the tests with coverage-checks (and only one time, not twice) should run, when the developers do mvn test. We have a quite small difference between coverage-threshold and actual coverage, so I think it's important, that the developers check their coverage during development, so they can react early. Another problem to your solution is, that we unfortunately don't use a regular CI-server but a bunch of shellscripts and cron-jobs. So it's not as trivial as with Hudson (Jenkins?) to use this kind of build-pipeline. -- GMX DSL Doppel-Flat ab 19,99 Euro/mtl.! Jetzt mit gratis Handy-Flat! http://portal.gmx.net/de/go/dsl - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
AW: Cobertura and Surefire
Jeff Jensen wrote: Perhaps a different CI job structuring? This has worked well for me at many customers: Have a CI job for only compile and unit tests - this maintains the most important very fast turnaround. Have a second CI job for (longer running) IT tests that only runs with success of the first. Have a third job for coverage, standalone or as part of a full system site gen or using Sonar or other... Sounds nice, but this doesn't meet my requirement, that the tests with coverage-checks (and only one time, not twice) should run, when the developers do mvn test. We have a quite small difference between coverage-threshold and actual coverage, so I think it's important, that the developers check their coverage during development, so they can react early. Another problem to your solution is, that we unfortunately don't use a regular CI-server but a bunch of shellscripts and cron-jobs. So it's not as trivial as with Hudson (Jenkins?) to use this kind of build-pipeline. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Cobertura and Surefire
Sounds nice, but this doesn't meet my requirement, that the tests with coverage-checks (and only one time, not twice) should run, when the developers do mvn test. It sounds like Maven cannot, for whatever reason, meet your requirement. We have a quite small difference between coverage-threshold and actual coverage, so I think it's important, that the developers check their coverage during development, so they can react early. It seems like you are only running into this problem because your coverage is running arbitrarily close to the 80% cutoff that you defined for test coverage. Perhaps you should instead focus some energies on bumping your test coverage a lot so you are not so close to the cutoff (so people aren't casually bumping into it so regularly), or knock the cutoff down to 75% or so to give some breathing room? Another problem to your solution is, that we unfortunately don't use a regular CI-server but a bunch of shellscripts and cron-jobs. So it's not as trivial as with Hudson (Jenkins?) to use this kind of build-pipeline. Why can't you set up a more formal CI server like Hudson? This is solution proposed by Jeff is a reasonable one and I know this same strategy is used by a lot of groups doing CI. Wayne - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Cobertura and Surefire
On Thu, Jan 13, 2011 at 9:01 AM, Schulze, Stefan (EXTERN: CKC) extern.stefan.schul...@volkswagen.de wrote: Jeff Jensen wrote: Perhaps a different CI job structuring? This has worked well for me at many customers: Have a CI job for only compile and unit tests - this maintains the most important very fast turnaround. Have a second CI job for (longer running) IT tests that only runs with success of the first. Have a third job for coverage, standalone or as part of a full system site gen or using Sonar or other... Sounds nice, but this doesn't meet my requirement, that the tests with coverage-checks (and only one time, not twice) should run, when the developers do mvn test. There is no acceptable solution to this requirement. Many of us would like to not run them twice, and in actuality running them twice is not a problem for things like site gen. However, my thinking is to separate the two runs anyway, as coverage is not needed with each local build. Run mvn test until the tests all pass and the developer thinks there is reasonable coverage, then run mvn cobertura:cobertura and view the results. I think running coverage all the time is not necessary until ready to inspect it, particularly when I know I have a lots of work to do on the story/feature; coverage is irrelevant at the start with full TDD (tests exist, no code exists, compile errors exist) and until I am ready to see the needed additional cases. That's for me; YMMV! :-) We have a quite small difference between coverage-threshold and actual coverage, so I think it's important, that the developers check their coverage during development, so they can react early. What IDE is in use? If Eclipse, consider using eCobertura, a Cobertura Eclipse plugin, to run the tests in Eclipse and visually see the coverage as needed. Another problem to your solution is, that Actually, it is a problem with your environment/practices, not with the solution! ;-) we unfortunately don't use a regular CI-server but a bunch of shellscripts and cron-jobs. So it's not as trivial as with Hudson (Jenkins?) to use this kind of build-pipeline. Looks like it is time to invest a couple of days and set it up! Your work life will be easier and maintenance cheaper.
Cobertura and Surefire
Hi, In our project, the tests take some time - more than the developers are willing to execute twice (Surefire and Cobertura) over and over again. First I bound Cobertura to verify-phase, but because the developers only run test to run the tests only once, they never go through the coverage-checks and the build on the buildserver breaks regularly because of too less coverage. Because usually the result of instrumented and non-instrumented code is the same, I would like to bind Cobertura on the test-phase and Surefire on the verify-phase. But I can't get it work. Maven always executes either both plugins in the test-phase or successfully skips Surefire but effectivly skips Cobertura, too, because of skipped Surefire. Currently (both are skipped) I tried to use the following plugin-configuration: plugin groupIdorg.codehaus.mojo/groupId artifactIdcobertura-maven-plugin/artifactId version2.4/version configuration instrumentation excludes exclude**/*Exception.class/exclude /excludes /instrumentation check totalLineRate80/totalLineRate haltOnFailuretrue/haltOnFailure /check /configuration executions execution phasetest/phase goals goalclean/goal goalcheck/goal /goals /execution /executions /plugin plugin groupIdorg.apache.maven.plugins/groupId artifactIdmaven-surefire-plugin/artifactId configuration skiptrue/skip /configuration executions execution phaseverify/phase goals goaltest/goal /goals configuration skipfalse/skip /configuration /execution /executions /plugin Does anybody know how to configure this? -- GMX DSL Doppel-Flat ab 19,99 Euro/mtl.! Jetzt mit gratis Handy-Flat! http://portal.gmx.net/de/go/dsl - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Cobertura and Surefire
Because people who have not read and understood concurrency in practice often do not understand how the synchronization points affect jvm sequencing, people often wrongly suspect that result of instrumented and non-instrumented code is the same. I have had bugs which were not caught by unit tests because: 1. The test passed when the code was not instrumented and failed when the code was instrumented 2. The test failed when the code was not instrumented and passed when the code was instrumented As a result of my experiences I will not help you -Stephen On 12 January 2011 15:29, Stefan Schulze algr...@gmx.de wrote: Hi, In our project, the tests take some time - more than the developers are willing to execute twice (Surefire and Cobertura) over and over again. First I bound Cobertura to verify-phase, but because the developers only run test to run the tests only once, they never go through the coverage-checks and the build on the buildserver breaks regularly because of too less coverage. Because usually the result of instrumented and non-instrumented code is the same, I would like to bind Cobertura on the test-phase and Surefire on the verify-phase. But I can't get it work. Maven always executes either both plugins in the test-phase or successfully skips Surefire but effectivly skips Cobertura, too, because of skipped Surefire. Currently (both are skipped) I tried to use the following plugin-configuration: plugin groupIdorg.codehaus.mojo/groupId artifactIdcobertura-maven-plugin/artifactId version2.4/version configuration instrumentation excludes exclude**/*Exception.class/exclude /excludes /instrumentation check totalLineRate80/totalLineRate haltOnFailuretrue/haltOnFailure /check /configuration executions execution phasetest/phase goals goalclean/goal goalcheck/goal /goals /execution /executions /plugin plugin groupIdorg.apache.maven.plugins/groupId artifactIdmaven-surefire-plugin/artifactId configuration skiptrue/skip /configuration executions execution phaseverify/phase goals goaltest/goal /goals configuration skipfalse/skip /configuration /execution /executions /plugin Does anybody know how to configure this? -- GMX DSL Doppel-Flat ab 19,99 Euro/mtl.! Jetzt mit gratis Handy-Flat! http://portal.gmx.net/de/go/dsl - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Cobertura and Surefire
First I bound Cobertura to verify-phase, but because the developers only run test to run the tests only once, they never go through the coverage-checks and the build on the buildserver breaks regularly because of too less coverage. Perhaps bind something to your scm commit hook so the code simply can't make it into your SCM tool without meeting certain standards for coverage? Wayne - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Cobertura and Surefire
On 12 January 2011 08:12, Stephen Connolly stephen.alan.conno...@gmail.com wrote: Because people who have not read and understood concurrency in practice often do not understand how the synchronization points affect jvm sequencing, people often wrongly suspect that result of instrumented and non-instrumented code is the same. I have had bugs which were not caught by unit tests because: 1. The test passed when the code was not instrumented and failed when the code was instrumented 2. The test failed when the code was not instrumented and passed when the code was instrumented This is just silly. If you want to test your concurrent code then write specific tests for it. What you are doing is changing the environment (a tiny bit) and hoping that you will find something. If that is your strategy then make a big change to your environment and run your tests in really different environments: IDE vs CLI, Windows vs Linux, desktop vs build server, etcetera. A normal unit test should run once, more is just a waste of time. The fact that Maven will run unit tests multiple times is a bug that we apparently have to live with. It is not a feature. If this bug were fixed and you still wanted to run your unit tests twice then you could configure that quite easily in your POM. Everybody happy. As a result of my experiences I will not help you Then why reply? Are you suggesting we now all start replying to everything so that everyone knows we can not/will not help and why? :-P - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Cobertura and Surefire
actually these were tests of code that the developers thought not to include threading or concurrency I've been badly bitten by running the tests only once. they are unit tests, they should be fast enough to run them twice. and you should run them more than once and in a random order each time. now if it's integration tests... whole different storey. re your other point. I am consyantly amased at the java developers who think they don't have to know about threading... they all think that running the tests once with cobertura and having a successful run means that everything is fine... that is not best practice. maven is about being opinionated software. maven's opinion is that only by running the tests un instrumented can you prove the tests pass. if you don't like mavens opinion, you can have a hard time fighting it. stay on the naven way and run the damn tests twice - Stephen --- Sent from my Android phone, so random spelling mistakes, random nonsense words and other nonsense are a direct result of using swype to type on the screen On 12 Jan 2011 17:39, Hilco Wijbenga hilco.wijbe...@gmail.com wrote:
cobertura and surefire 2.4.1 - is it working?
Hi, I have being going through most documentation and bugs for cobertura/surefire. Tried all possible versions and configuration, but nothing works. Can someone share working sufrefire 2.4.1 and cobertura? Thanks, Erez.
Re: cobertura and surefire 2.4.1 - is it working?
Just found that if changing forkMode from never to once, it works. Erez. On Feb 9, 2008 10:56 PM, Erez Nahir [EMAIL PROTECTED] wrote: Hi, I have being going through most documentation and bugs for cobertura/surefire. Tried all possible versions and configuration, but nothing works. Can someone share working sufrefire 2.4.1 and cobertura? Thanks, Erez.
