> -----Original Message-----
> From: Brett Porter [mailto:[EMAIL PROTECTED]
> Sent: jeudi 12 janvier 2006 23:56
> To: Maven Developers List
> Subject: Re: [codehaus-confluence] Maven: best practices - testing
> strategies (comment added)
>
> Vincent Massol wrote:
> > There's still a need to define the different types of tests. What we're
> > trying to do in m2 is find generic phases that work for all types of
> build
> > actions. The mapping between the phases and the tools is done
> separately. So
> > we still need to define the unit, integration and functional tests
> phases,
> > independently of what tools are used to execute them. This is required
> > because we need to know when the tests should be triggered in the
> lifecycle.
>
> +1. I recall doing some work on this, getting some feedback in the
> original JIRA, and linking some discussion. Has anyone dug that up?
>
> Maybe the names need to be more specific. While everyone recognises
> "integration testing", everyone also thinks it is something different :)
I'd be happy to drop the integration test phase altogether. I think it can
be replaced by the functional test phase. An integration test is a
functional test of some sort in my vocabulary. If someone else's vocabulary
consider it a unit test then it can be put in the unit test phase.
> > Now that said we could have specific plugins for each framework (cactus,
> > httpunit, etc) but I think we'd loose some flexibility if we had to code
> > those. Instead what I'm suggesting here is that there's no need for
> those
> > plugins (note: we could still have them but they would only be
> convenience
> > plugins, used to fill the test classpath automatically for example).
>
> I agree.
There might still be use cases where users want to split their unit tests
(or it or functional tests) into several directories so that they are
executed separately (and possibly under different profiles).
So we still need to ensure that it's possible to have several executions of
the surefire plugin and to ensure that the testSourceDirectory of surefire
could be overridden. For example:
src/test
|_ unit (reserved name)
|_ unit2 (custom name)
|_ integration (reserved name)
|_ integration2 (custom name)
<profiles>
[...]
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<executions>
<execution>
<id>unit2</id>
<phase>test</phase>
<configuration>
<testSourceDirectory>
${project.build.testDirectory}/unit
</testSourceDirectory>
</configuration>
</execution>
[...]
<execution>
<id>integration2</id>
<phase>integration-test/functional-test</phase>
<configuration>
<testSourceDirectory>
${project.build.testDirectory}/unit
</testSourceDirectory>
</configuration>
</execution>
By default surefire would run the tests in unit/ and integration/
directories but it could be made to also run those in unit2/ and
integration2/.
> > tests are junit tests and thus would be run by the generic surefire
> plugin.
> > What you would need though is the ability to do things before and after
> the
> > tests (like deploy J2EE modules to a container, start the container,
> stop
> > it, etc). This is why there are before-integration-test and
> > after-integration-test phases defined.
>
> +1. The phases of testing need to be identified and applied consistently
> too (compiling, resources, etc). Integration testing probably needs a
> packaging step.
Yes you're right this is required (compiling, resources, packaging, etc).
For packaging I can provide an example. Imagine you're using Cactus. There's
a need to "cactify" the runtime WAR/EAR by adding Cactus jars inside WAR/EAR
and modifying deployment descriptors to add redirectors. That cactification
step would fit nicely in this packaging phase. Note that it could also be
done in the pre-integration-test/pre-functional-test phase but a packaging
phase would be cleaner.
> > For jmeter it's different as it's not made of junit tests so you'd need
> a
> > jmeter test runner in the form of a plugin.
>
> Yep, we need to add this ability to the surefire plugin. I think it is
> already possible - but would need to check and it could certainly use
> some refactoring. Basically the api/provider model: surefire is the api,
> surefire booter is surefire-junit I think, then there is
> surefire-testng, surefire-cactus (if any different to straight junit),
> surefire-jmeter, etc.
Sounds cool provided it's easy to add a new surefire-xxx from outside. I
mean imagine I'm working on the Cactus project and I want to create a
surefire-cactus. Could I provide it as a jar in the Cactus repository and
could it then be wired somehow so that the surefire plugin would use it?
If this is not possible then I fear people will simply duplicate the
surefire plugin's sources into their own plugin.
> To take a tangent...
>
> That might expose a limitation in the output of the plugin though - or
> do we let it do reporting in its own way? So most will spit out "junit
> style xml report", "junit style txt report" for pass/fail. Then maybe
> jmeter spits out a different xml report for transforming into the site.
>
> I think in contrast, the reporting plugins should not be a provider
> style - we should make the current "surefire report" the "test results
> report", and have different reporting plugins for jmeter, etc.
I'm still unsure of the need to have a single surefire plugin for tests with
different providers. The alternative would be different test plugins all
using a common surefire jar for the basic execution (delegation vs
inheritance).
If we had this then the report mojos could be put in the different test
plugins.
> Some of the handling may go in surefire itself so that various reports
> can use it - eg I'm sure cactus is still "different but similar" to the
> junit - so would use some of the code but add some of its own.
Actually for Cactus I don't think any change would be needed for junit test
execution as the execution is broken into 4 parts:
- cactify the module to deploy (WAR or EAR) (cactus plugin)
- start the container and deploy the cactified module to it (cargo plugin)
- execute the tests as pure junit tests (surefire plugin)
- stop the container (cargo plugin)
So the only requirement would be the ability to configure the different
steps in the right order.
However we'd need to take into account that users may want to run the cactus
tests on several different containers in a row too. And they would want to
have a report that aggregates the results for all containers...
Hmm... I'm starting to wonder if defining the workflow in pom.xml wouldn't
be too complex after all and maybe it would be better defined in a cactus
plugin.
> > All that said, I think we should support different executions for the
> > surefire plugin so that it should be possible to define arbitrary test
> > directories should the need arise.
>
> This is tied in to the ability to do likewise with compile/resources.
Right
> We
> should push this to the 2.1 discussion on changing the execution model.
> I think we actually need a single build to be able to re-execute the
> lifecycle in a different way.
I agree.
Thanks
-Vincent
___________________________________________________________________________
Nouveau : téléphonez moins cher avec Yahoo! Messenger ! Découvez les tarifs
exceptionnels pour appeler la France et l'international.
Téléchargez sur http://fr.messenger.yahoo.com
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
