2010/1/4 Stephen Connolly <stephen.alan.conno...@gmail.com>: > 2010/1/4 Dennis Lundberg <denn...@apache.org>: >> Just so I understand, when you say the ITs "won't work for release", do >> you mean >> >> 1. They will fail the release process > > mvn release:prepare will always fail because th tests do not stage the > artifacts into the local repo >
my refactoring of the tests is to switch running them to the integration-test phase and use invoker:install to put the surefire artifacts into a staged local repo, and have a never fail test that copies settings.xml to a staging location adding the current local repo as a remote repo so that you can pass the tests from behind a corporate proxy, and so that the tests run efficiently... as such I suspect I will need both surefire and failsafe to achieve the required testing (though I could use two executions of surefire and hack with includes/excludes) So I will use failsafe-maven-plugin 2.4.3 and maven-surefire-plugin 2.4.3 to test the release of surefire 2.5.... and to save my effort porting the changes from 2.4.3 to 2.5 for failsafe, I'll move that into the release process at the same time and then 2.6 can use maven-surefire-plugin 2.5 and maven-failsafe-plugin 2.5 for its release >> >> or >> >> 2. The current IT coverage is bad >> >> Stephen Connolly wrote: >>> sorry that may have come off a tad rude. >>> >>> technically I could rewrite the tests without adding the failsafe stuff, >>> but that would look hacky and maven should show best practice not hacky >>> >>> if you want to release 2.5 before Friday, fine... otherwise I'll do it >>> my way >>> >>> Sent from my [rhymes with tryPod] ;-) >>> >>> On 4 Jan 2010, at 22:21, Stephen Connolly >>> <stephen.alan.conno...@gmail.com> wrote: >>> >>>> no can do. I need to fix the integration tests... they won't work for >>>> release as written... to rewrite them I need failsafe >>>> >>>> Sent from my [rhymes with tryPod] ;-) >>>> >>>> On 4 Jan 2010, at 22:13, Dennis Lundberg <denn...@apache.org> wrote: >>>> >>>>> I'm not familiar enough to with the code bases to judge your proposal. >>>>> >>>>> What I'd like though is a new step, before the ones you listed, and that >>>>> is to release Surefire 2.5 with whatever is in trunk *first*. >>>>> >>>>> Stephen Connolly wrote: >>>>>> OK, >>>>>> >>>>>> maven-surefire-plugin is well known to everyone >>>>>> >>>>>> it's lesser-known sister is failsafe-maven-plugin >>>>>> >>>>>> failsafe was written (by me) to solve some of the issues of running >>>>>> integration tests with the maven lifecycle. >>>>>> >>>>>> the lifecycle has a number of phases >>>>>> >>>>>> * some crappy phases >>>>>> * test >>>>>> * some more crappy phases >>>>>> * pre-integration-test >>>>>> * integration-test >>>>>> * post-integration-test >>>>>> * verify >>>>>> * yet more crappy phases >>>>>> >>>>>> surefire binds to the test phase, and by default will fail your build >>>>>> at the test phase if any of the tests it runs fail. >>>>>> >>>>>> That is all well and good when running unit tests... >>>>>> >>>>>> when we want to run integration tests, we usually want to set up some >>>>>> environment which we are integrating with, e.g. start a jetty server >>>>>> and deploy the current application to that server, or start a database >>>>>> server (e.g. derby) and configure the data source, etc >>>>>> >>>>>> so we use the pre-integration-test phase to do this set-up, and we >>>>>> tidy-up in the post-integration-test phase... >>>>>> >>>>>> when our pre-integration-test stuff is all running in the maven JVM, >>>>>> everything is hunky-dory >>>>>> (http://www.taytohunkydorys.ie/brands/hunky_dorys.asp). we bind a >>>>>> second execution of surefire:test to the integration-test phase and >>>>>> when/if the tests fail, the JVM gets killed and our integration test >>>>>> environment gets destroyed... >>>>>> >>>>>> however, if our pre-integration-test stuff affects external processes, >>>>>> we are stuck because we have no way to tidy-up in the >>>>>> post-integration-test phase (BTW, IMHO nobody should ever run "mvn >>>>>> integration-test". 1. it's too long to type, and 2, you reallly want >>>>>> to run "mvn verify" as that will give the post-integration-test phase >>>>>> a chance to run) >>>>>> >>>>>> failsafe solves the issue by decoupling failing the build from running >>>>>> the tests, failsafe:integration-test never fails the build, that is >>>>>> left up to failsafe:verify, and thus (as long as you never invoke a >>>>>> phase in the range [pre-integration-test,post-integration-test]) your >>>>>> tidy-up plugin configuration will always run. >>>>>> >>>>>> So, as part of rolling the 2.5 release of surefire, I was looking into >>>>>> merging the two plugins... >>>>>> >>>>>> My initial idea was to just move the mojo's into m-surefire-p as is, >>>>>> thus m-s-p would have three mojos: test, integration-test and verify >>>>>> >>>>>> However, there are advantages to keeping them as separate plugins: >>>>>> >>>>>> * Use case 1: Ignore Unit Test Failures and run the integration tests >>>>>> (I know Bob has broken the unit tests for that new feature he is >>>>>> writing, but I want to check I have not created a regression in the >>>>>> stuff I'm working on) - the verify mojo parses the xml result files to >>>>>> see if any tests failed. If we use the same plugin, we should really >>>>>> use the same directory (e.g. surefire-reports) It does not make sense >>>>>> for surefire to have one goal report to surefire-reports and the other >>>>>> report to failsafe-reports... and it we change the directory for the >>>>>> unit tests to e.g. test-reports that could have a regression imact. >>>>>> >>>>>> * Use case 2: Separate Reporting (as separate plugins, they generate >>>>>> reports in separate directories as before) >>>>>> >>>>>> * Use case 3: Separate default binding... consider the case where >>>>>> pluginManagement specifies executions, with separate plugins I can >>>>>> safely define /project/build/plugins/plugin/m-surefire-p and not drag >>>>>> in the default execution of failsafe:integration-test and >>>>>> failsafe-verify >>>>>> >>>>>> So my proposal is as follows: >>>>>> >>>>>> 1. Move failsafe-maven-plugin to >>>>>> https://svn.apache.org/repos/asf/maven/surefire/trunk/maven-failsafe-plugin >>>>>> >>>>>> >>>>>> 2. Refactor maven-surefire-plugin taking the code that is common with >>>>>> failsafe into common module >>>>>> >>>>>> 3. Refactor maven-failsafe-plugin to use the common module. >>>>>> >>>>>> In pseudo code SurefirePlugin would be reduced to >>>>>> >>>>>> execute() { >>>>>> common.runTests(); >>>>>> } >>>>>> >>>>>> Failsafe's integration-test mojo would become >>>>>> >>>>>> execute() { >>>>>> try { >>>>>> common.runTests(); >>>>>> } catch (Throwable t) { >>>>>> // ignore >>>>>> } >>>>>> } >>>>>> >>>>>> 4. Refactor maven-surefire-reporting-plugin to move the xml results >>>>>> parser into the common module >>>>>> >>>>>> 5. Refactor Failsafe's verify mojo to use the xml results parse from >>>>>> the common module. >>>>>> >>>>>> OK, so that's a tad long, but what do people think? >>>>>> >>>>>> Any objections? >>>>>> >>>>>> -Stephen >>>>>> >>>>>> Lazy consensus, 72hrs, i.e. you have until 18:00GMT on 7th Jan 2010 to >>>>>> -1 this (after which you'll have to -1 the commit if you feel strongly >>>>>> against the direction I'm suggesting) >>>>>> >>>>>> --------------------------------------------------------------------- >>>>>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >>>>>> For additional commands, e-mail: dev-h...@maven.apache.org >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Dennis Lundberg >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >>>>> For additional commands, e-mail: dev-h...@maven.apache.org >>>>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >>> For additional commands, e-mail: dev-h...@maven.apache.org >>> >>> >> >> >> -- >> Dennis Lundberg >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >> For additional commands, e-mail: dev-h...@maven.apache.org >> >> > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
