We might solve this simply with a package naming rule. And then we configure surefire to just run the tests with the respective package?
Not the most advanced solution, but will work. LieGrue, strub ----- Original Message ----- > From: Christian Kaltepoth <[email protected]> > To: [email protected] > Cc: > Sent: Saturday, December 17, 2011 4:43 PM > Subject: Re: basic decisions: test setup > > @Mark: > > Actually @TargetsContainer is an existing feature of Arquillian. It > allows to build container specific deployments. > > https://docs.jboss.org/author/display/ARQ/Multiple+Containers > > Unfortunately it won't fit our needs because the test completely fails > if there is no matching deployment for the container the test is > executed against. So I guess we have to build something new for this. > > I really like your suggestion regarding the four environments. It > absolutely make sense to declare environment constraints (like SE, > Servlet, EE, etc.) instead of listing concrete containers. > > Christian > > > > 2011/12/17 Mark Struberg <[email protected]>: >> Hi Christian! >> >> >> @TargetContainer sounds good, but I'd rather make it a >> >> >> @TargetEnvironment() >> >> with values: SE, Servlet, EE-light, EE-full >> >> The excludes should be done in the single test integration modules, and the > goal should be to get down to zero excluded tests for all containers. >> >> >> @Jason: that mixture for the tests looks fine to me as well. Can you please > hack a sample structure in the DeltaSpikePlayground git repo? >> >> >> LieGrue, >> strub >> >> >> >> >>> ________________________________ >>> From: Christian Kaltepoth <[email protected]> >>> To: [email protected] >>> Sent: Saturday, December 17, 2011 7:55 AM >>> Subject: Re: basic decisions: test setup >>> >>> I would like to add some "non-binding" comments here regarding > 3). >>> >>> Actually I really like the concept Jason described and which is >>> currently used in Seam3. Having worked with it a bit I can confirm >>> that it runs fine. And somehow it's the best of both worlds (in > regard >>> to 3a vs 3b). >>> >>> That the full integration test suite with all containers cannot run in >>> a single Maven invocation is IMHO not a real problem. We could write a >>> simple script for this. >>> >>> The only thing that could be optimized in this setup is the separation >>> of container dependent tests via package name. But perhaps this could >>> be solved by Arquillian? What I would really like to see is the >>> ability to configure a test so it is only executed in a specific >>> container (something similar to @TargetsContainer). This way tests >>> that require to be run in a full container could be executed only in >>> JBoss and Glassfish but would be skipped in embedded Weld. >>> >>> Christian >>> >>> >>> 2011/12/16 Jason Porter <[email protected]>: >>>> Comments inline. >>>> >>>> On Fri, Dec 16, 2011 at 01:38, Mark Struberg > <[email protected]> wrote: >>>> >>>>> Hi! >>>>> >>>>> Another round of discussion - this time about how to setup our > unit >>>>> testing and integration testing. >>>>> >>>>> I think it's pretty much clear that we will gonna use > Arquillian for all >>>>> our tests. That was also fundamental part of the incubator > proposal. >>>>> And instead of inventing yet another abstraction layer, I'd > favour to just >>>>> use Arquillian and contribute all things we need to this > project. >>>>> So here a formal vote >>>>> >>>>> 1.) >>>>> >>>>> for using Arquillian as test integration framework. >>>>> +1 from me >>>>> >>>> >>>> +1 from me >>>> >>>> >>>>> >>>>> 2.) >>>>> >>>>> What do we like to use from the unit tests itself? TestNG or > JUnit? >>>>> JUnit has better Arquillian integration and TestNG is better > for 'real >>>>> world' projects, because it allows to define test > dependencies. >>>>> So which one to take? >>>>> But we should definitely only use 1 of the two exclusively! >>>>> >>>> >>>> +1 for using one exclusively >>>> >>>> As for which, I'm also +0 I like both for different reasons, > but JUnit is >>>> certainly the easier of the two to use with Arquillian. >>>> >>>> Also, are we open to having other languages for tests or using > other >>>> testing structures such as spock (it works with Arquillian, BTW), > RSpec, >>>> scala, etc.? >>>> >>>> >>>>> 3.) >>>>> 'Integrated' tests vs 'Integration Tests' >>>>> We have 2 options to test our projects >>>>> >>>>> 3.a.) create a full unit test in each module (e.g. > deltaspike/core/impl) >>>>> and add a profile for each and every server (maybe we can trim > this down >>>>> with pom imports?). >>>>> Then run the build with one after each other: >>>>> >>>>> $> mvn clean test -Powb >>>>> $> mvn clean test -Pweld >>>>> $> mvn clean test -Powb-tc >>>>> $> mvn clean test -Pweld-tc >>>>> $> mvn clean test -Pgf31 >>>>> $> mvn clean test -Pas7 >>>>> $> mvn clean test -Ptomee >>>>> >>>>> This might pretty much blow up our poms... >>>>> >>>>> >>>>> 3.b) create a full unit test in each module (e.g. > deltaspike/core/impl) >>>>> and add only 2 profiles directly ('weld' and > 'owb' (probably resin later)) >>>>> Then add a deltaspike/test/base/core with the very basic parent > stuff and >>>>> 1 module for each integrated container, e.g. >>>>> deltaspike/test/weld-tc >>>>> deltaspike/test/owb-tc >>>>> deltaspike/test/weld-jetty >>>>> deltaspike/test/owb-jetty >>>>> deltaspike/test/tomee >>>>> deltaspike/test/gf31 >>>>> deltaspike/test/geronimo >>>>> deltaspike/test/websphere >>>>> deltaspike/test/as7 >>>>> deltaspike/test/as6 >>>>> etc. >>>>> >>>>> The initial setup costs are higher, but it would be pretty easy > to add new >>>>> containers that way. >>>>> >>>>> Which route shall we take? >>>>> >>>> >>>> In Seam 3 we have an approach which is a little bit of a mixture of > both. >>>> We have regular unit tests (with mockito, stubs, etc) under >>>> impl/src/test/... and then we have a separate module called > testsuite which >>>> contained all the integration tests. We used pom imports to get all > the >>>> containers setup correctly and then had different profiles which > pulled in >>>> the poms and also configured surefire for that server. Inside the > code >>>> structure we separated the servers by package: common, jbossas, > glassfish, >>>> etc. That way all of the tests were in one location, they were easy > to >>>> execute, the tests stayed DRY, and the IDE wasn't much of a > problem. The >>>> two downsides we had were 1) the extra config in the pom and 2) you > can't >>>> run the full integration suite in one maven execution. We spent > quite a bit >>>> of time examining options and trying some of them out. This was the > best we >>>> came up with, and also the one Aslak thought would work best as > well. >>>> >>>> >>>>> LieGrue, >>>>> strub >>>>> >>>>> >>>> >>>> >>>> -- >>>> Jason Porter >>>> http://lightguard-jp.blogspot.com >>>> http://twitter.com/lightguardjp >>>> >>>> Software Engineer >>>> Open Source Advocate >>>> Author of Seam Catch - Next Generation Java Exception Handling >>>> >>>> PGP key id: 926CCFF5 >>>> PGP key available at: keyserver.net, pgp.mit.edu >>> >>> >>> >>> -- >>> Christian Kaltepoth >>> Blog: http://chkal.blogspot.com/ >>> Twitter: http://twitter.com/chkal >>> >>> >>> > > > > -- > Christian Kaltepoth > Blog: http://chkal.blogspot.com/ > Twitter: http://twitter.com/chkal >
