I would like to add some "non-binding" comments here regarding 3).
Actually I really like the concept Jason described and which is currently used in Seam3. Having worked with it a bit I can confirm that it runs fine. And somehow it's the best of both worlds (in regard to 3a vs 3b). That the full integration test suite with all containers cannot run in a single Maven invocation is IMHO not a real problem. We could write a simple script for this. The only thing that could be optimized in this setup is the separation of container dependent tests via package name. But perhaps this could be solved by Arquillian? What I would really like to see is the ability to configure a test so it is only executed in a specific container (something similar to @TargetsContainer). This way tests that require to be run in a full container could be executed only in JBoss and Glassfish but would be skipped in embedded Weld. Christian 2011/12/16 Jason Porter <[email protected]>: > Comments inline. > > On Fri, Dec 16, 2011 at 01:38, Mark Struberg <[email protected]> wrote: > >> Hi! >> >> Another round of discussion - this time about how to setup our unit >> testing and integration testing. >> >> I think it's pretty much clear that we will gonna use Arquillian for all >> our tests. That was also fundamental part of the incubator proposal. >> And instead of inventing yet another abstraction layer, I'd favour to just >> use Arquillian and contribute all things we need to this project. >> So here a formal vote >> >> 1.) >> >> for using Arquillian as test integration framework. >> +1 from me >> > > +1 from me > > >> >> 2.) >> >> What do we like to use from the unit tests itself? TestNG or JUnit? >> JUnit has better Arquillian integration and TestNG is better for 'real >> world' projects, because it allows to define test dependencies. >> So which one to take? >> But we should definitely only use 1 of the two exclusively! >> > > +1 for using one exclusively > > As for which, I'm also +0 I like both for different reasons, but JUnit is > certainly the easier of the two to use with Arquillian. > > Also, are we open to having other languages for tests or using other > testing structures such as spock (it works with Arquillian, BTW), RSpec, > scala, etc.? > > >> 3.) >> 'Integrated' tests vs 'Integration Tests' >> We have 2 options to test our projects >> >> 3.a.) create a full unit test in each module (e.g. deltaspike/core/impl) >> and add a profile for each and every server (maybe we can trim this down >> with pom imports?). >> Then run the build with one after each other: >> >> $> mvn clean test -Powb >> $> mvn clean test -Pweld >> $> mvn clean test -Powb-tc >> $> mvn clean test -Pweld-tc >> $> mvn clean test -Pgf31 >> $> mvn clean test -Pas7 >> $> mvn clean test -Ptomee >> >> This might pretty much blow up our poms... >> >> >> 3.b) create a full unit test in each module (e.g. deltaspike/core/impl) >> and add only 2 profiles directly ('weld' and 'owb' (probably resin later)) >> Then add a deltaspike/test/base/core with the very basic parent stuff and >> 1 module for each integrated container, e.g. >> deltaspike/test/weld-tc >> deltaspike/test/owb-tc >> deltaspike/test/weld-jetty >> deltaspike/test/owb-jetty >> deltaspike/test/tomee >> deltaspike/test/gf31 >> deltaspike/test/geronimo >> deltaspike/test/websphere >> deltaspike/test/as7 >> deltaspike/test/as6 >> etc. >> >> The initial setup costs are higher, but it would be pretty easy to add new >> containers that way. >> >> Which route shall we take? >> > > In Seam 3 we have an approach which is a little bit of a mixture of both. > We have regular unit tests (with mockito, stubs, etc) under > impl/src/test/... and then we have a separate module called testsuite which > contained all the integration tests. We used pom imports to get all the > containers setup correctly and then had different profiles which pulled in > the poms and also configured surefire for that server. Inside the code > structure we separated the servers by package: common, jbossas, glassfish, > etc. That way all of the tests were in one location, they were easy to > execute, the tests stayed DRY, and the IDE wasn't much of a problem. The > two downsides we had were 1) the extra config in the pom and 2) you can't > run the full integration suite in one maven execution. We spent quite a bit > of time examining options and trying some of them out. This was the best we > came up with, and also the one Aslak thought would work best as well. > > >> LieGrue, >> strub >> >> > > > -- > Jason Porter > http://lightguard-jp.blogspot.com > http://twitter.com/lightguardjp > > Software Engineer > Open Source Advocate > Author of Seam Catch - Next Generation Java Exception Handling > > PGP key id: 926CCFF5 > PGP key available at: keyserver.net, pgp.mit.edu -- Christian Kaltepoth Blog: http://chkal.blogspot.com/ Twitter: http://twitter.com/chkal
