@Mark: Actually @TargetsContainer is an existing feature of Arquillian. It allows to build container specific deployments.
https://docs.jboss.org/author/display/ARQ/Multiple+Containers Unfortunately it won't fit our needs because the test completely fails if there is no matching deployment for the container the test is executed against. So I guess we have to build something new for this. I really like your suggestion regarding the four environments. It absolutely make sense to declare environment constraints (like SE, Servlet, EE, etc.) instead of listing concrete containers. Christian 2011/12/17 Mark Struberg <[email protected]>: > Hi Christian! > > > @TargetContainer sounds good, but I'd rather make it a > > > @TargetEnvironment() > > with values: SE, Servlet, EE-light, EE-full > > The excludes should be done in the single test integration modules, and the > goal should be to get down to zero excluded tests for all containers. > > > @Jason: that mixture for the tests looks fine to me as well. Can you please > hack a sample structure in the DeltaSpikePlayground git repo? > > > LieGrue, > strub > > > > >>________________________________ >> From: Christian Kaltepoth <[email protected]> >>To: [email protected] >>Sent: Saturday, December 17, 2011 7:55 AM >>Subject: Re: basic decisions: test setup >> >>I would like to add some "non-binding" comments here regarding 3). >> >>Actually I really like the concept Jason described and which is >>currently used in Seam3. Having worked with it a bit I can confirm >>that it runs fine. And somehow it's the best of both worlds (in regard >>to 3a vs 3b). >> >>That the full integration test suite with all containers cannot run in >>a single Maven invocation is IMHO not a real problem. We could write a >>simple script for this. >> >>The only thing that could be optimized in this setup is the separation >>of container dependent tests via package name. But perhaps this could >>be solved by Arquillian? What I would really like to see is the >>ability to configure a test so it is only executed in a specific >>container (something similar to @TargetsContainer). This way tests >>that require to be run in a full container could be executed only in >>JBoss and Glassfish but would be skipped in embedded Weld. >> >>Christian >> >> >>2011/12/16 Jason Porter <[email protected]>: >>> Comments inline. >>> >>> On Fri, Dec 16, 2011 at 01:38, Mark Struberg <[email protected]> wrote: >>> >>>> Hi! >>>> >>>> Another round of discussion - this time about how to setup our unit >>>> testing and integration testing. >>>> >>>> I think it's pretty much clear that we will gonna use Arquillian for all >>>> our tests. That was also fundamental part of the incubator proposal. >>>> And instead of inventing yet another abstraction layer, I'd favour to just >>>> use Arquillian and contribute all things we need to this project. >>>> So here a formal vote >>>> >>>> 1.) >>>> >>>> for using Arquillian as test integration framework. >>>> +1 from me >>>> >>> >>> +1 from me >>> >>> >>>> >>>> 2.) >>>> >>>> What do we like to use from the unit tests itself? TestNG or JUnit? >>>> JUnit has better Arquillian integration and TestNG is better for 'real >>>> world' projects, because it allows to define test dependencies. >>>> So which one to take? >>>> But we should definitely only use 1 of the two exclusively! >>>> >>> >>> +1 for using one exclusively >>> >>> As for which, I'm also +0 I like both for different reasons, but JUnit is >>> certainly the easier of the two to use with Arquillian. >>> >>> Also, are we open to having other languages for tests or using other >>> testing structures such as spock (it works with Arquillian, BTW), RSpec, >>> scala, etc.? >>> >>> >>>> 3.) >>>> 'Integrated' tests vs 'Integration Tests' >>>> We have 2 options to test our projects >>>> >>>> 3.a.) create a full unit test in each module (e.g. deltaspike/core/impl) >>>> and add a profile for each and every server (maybe we can trim this down >>>> with pom imports?). >>>> Then run the build with one after each other: >>>> >>>> $> mvn clean test -Powb >>>> $> mvn clean test -Pweld >>>> $> mvn clean test -Powb-tc >>>> $> mvn clean test -Pweld-tc >>>> $> mvn clean test -Pgf31 >>>> $> mvn clean test -Pas7 >>>> $> mvn clean test -Ptomee >>>> >>>> This might pretty much blow up our poms... >>>> >>>> >>>> 3.b) create a full unit test in each module (e.g. deltaspike/core/impl) >>>> and add only 2 profiles directly ('weld' and 'owb' (probably resin later)) >>>> Then add a deltaspike/test/base/core with the very basic parent stuff and >>>> 1 module for each integrated container, e.g. >>>> deltaspike/test/weld-tc >>>> deltaspike/test/owb-tc >>>> deltaspike/test/weld-jetty >>>> deltaspike/test/owb-jetty >>>> deltaspike/test/tomee >>>> deltaspike/test/gf31 >>>> deltaspike/test/geronimo >>>> deltaspike/test/websphere >>>> deltaspike/test/as7 >>>> deltaspike/test/as6 >>>> etc. >>>> >>>> The initial setup costs are higher, but it would be pretty easy to add new >>>> containers that way. >>>> >>>> Which route shall we take? >>>> >>> >>> In Seam 3 we have an approach which is a little bit of a mixture of both. >>> We have regular unit tests (with mockito, stubs, etc) under >>> impl/src/test/... and then we have a separate module called testsuite which >>> contained all the integration tests. We used pom imports to get all the >>> containers setup correctly and then had different profiles which pulled in >>> the poms and also configured surefire for that server. Inside the code >>> structure we separated the servers by package: common, jbossas, glassfish, >>> etc. That way all of the tests were in one location, they were easy to >>> execute, the tests stayed DRY, and the IDE wasn't much of a problem. The >>> two downsides we had were 1) the extra config in the pom and 2) you can't >>> run the full integration suite in one maven execution. We spent quite a bit >>> of time examining options and trying some of them out. This was the best we >>> came up with, and also the one Aslak thought would work best as well. >>> >>> >>>> LieGrue, >>>> strub >>>> >>>> >>> >>> >>> -- >>> Jason Porter >>> http://lightguard-jp.blogspot.com >>> http://twitter.com/lightguardjp >>> >>> Software Engineer >>> Open Source Advocate >>> Author of Seam Catch - Next Generation Java Exception Handling >>> >>> PGP key id: 926CCFF5 >>> PGP key available at: keyserver.net, pgp.mit.edu >> >> >> >>-- >>Christian Kaltepoth >>Blog: http://chkal.blogspot.com/ >>Twitter: http://twitter.com/chkal >> >> >> -- Christian Kaltepoth Blog: http://chkal.blogspot.com/ Twitter: http://twitter.com/chkal
