Great! When can we get it in Arquillian? Sent from my iPhone
On Dec 22, 2011, at 18:10, Dan Allen <[email protected]> wrote: > You've hit a topic very near and dear to my heart, runtime environment > compatibility filters. > > Awhile ago I laid out a proposal for this feature *very* similar to what > Mark suggested (e.g., @RunInEnvironments). You can review the proposal here: > > http://community.jboss.org/thread/156500 > > After many discussions with Aslak, what we realized that the both > environment qualifiers (e.g., JavaEE6Web) and the binding to a container > implementation (e.g., JBoss AS 7) really need to be defined by the user (as > opposed to defined in Arquillian). What Arquillian needs to provide is the > mechanism for filtering the test cases that aren't suitable for the current > container being used. > > Our latest idea, concocted during Devoxx, is as follows: > > You define a qualifier annotation that represents a type of environment > (e.g., typically some sort of specification) > > @Retention(RetentionPolicy.RUNTIME) > @Target(ElementType.TYPE) > @ComplianceStereotype > public @interface JavaEE6Web {} > > (I suppose the annotation could instead be an enum as well...not sure the > right choice). > > Then you annotate your test class to indicate the requirement. > > @RunWith(Arquillian.class) > @RunsInEnvironment(JavaEE6Web.class) > public class JavaEE6WebSpecificTestCase { > ... > } > > Now, so far, nothing special would happen. Arquillian would have no way to > determine whether the current container is compatible with the JavaEE6Web > stereotype. For that, we have to tell Arquillian about the relationship via > arquillian.xml: > > <arquillian> > <container qualifier="jbossas-remote-7"> > <compliance qualifier="org.example.compliance.JavaEE6Web"/> > ... > </container> > </arquillian> > > > Naturally, since Arquillian can be configured using Java, you could > establish these bindings in a Java class, and thus via a JAR if you wanted. > > The point is, Arquillian would know whether a certain container is > compliant with the stereotype specified on the test class. If it isn't, the > test is skipped (or otherwise filtered out). > > I would definitely push to get this feature into Arquillian because I've > long believed it's a very important feature. While you can implement your > own test filtering based on package names (or TestNG groups), that's > exactly the type of work that Arquillian tries to eliminate from your > plate. If a test couldn't possibly work on a given container, then > Arquillian shouldn't even attempt to run it. > > -Dan > > p.s. We could also support groups/inheritence, so that JavaEE 6 Web > compliance would infer CDI compliance. In that case, this container would > also match the org.example.compliance.CDI environment stereotype. > > >> I like the idea, however, that would be a very core change to both core >> arquillian and the containers. I don't think we'll get that anytime soon. >> Maybe a simple package convention by environment will work for now? >> >> Sent from my iPhone >> >> On Dec 17, 2011, at 9:19, Mark Struberg <[email protected]> wrote: >> >>> true, we would need something like >>> >>> @RunInEnvironments({Environment.EeLight, Environment.EeFull, >> Environment.Servlet}) >>> >>> @RunInEnvironments(Environment.JavaSE) >>> @RunInEnvironments(Environments.ALL) >>> >>> LieGrue, >>> strub >>> >>> >>> >>> ----- Original Message ----- >>>> From: Christian Kaltepoth <[email protected]> >>>> To: [email protected] >>>> Cc: >>>> Sent: Saturday, December 17, 2011 5:04 PM >>>> Subject: Re: basic decisions: test setup >>>> >>>> @Mark: >>>> >>>> But what about tests that should run in EE-light, EE-full and Servlet >>>> but not in SE. Or tests that should run in EE-light/full but not in >>>> Servlet and SE. I think this could get very complex. >>>> >>>> Christian >>>> >>>> >>>> 2011/12/17 Mark Struberg <[email protected]>: >>>>> We might solve this simply with a package naming rule. And then we >>>> configure surefire to just run the tests with the respective package? >>>>> >>>>> Not the most advanced solution, but will work. >>>>> >>>>> >>>>> LieGrue, >>>>> strub >>>>> >>>>> >>>>> >>>>> ----- Original Message ----- >>>>>> From: Christian Kaltepoth <[email protected]> >>>>>> To: [email protected] >>>>>> Cc: >>>>>> Sent: Saturday, December 17, 2011 4:43 PM >>>>>> Subject: Re: basic decisions: test setup >>>>>> >>>>>> @Mark: >>>>>> >>>>>> Actually @TargetsContainer is an existing feature of Arquillian. It >>>>>> allows to build container specific deployments. >>>>>> >>>>>> https://docs.jboss.org/author/display/ARQ/Multiple+Containers >>>>>> >>>>>> Unfortunately it won't fit our needs because the test completely >>>> fails >>>>>> if there is no matching deployment for the container the test is >>>>>> executed against. So I guess we have to build something new for this. >>>>>> >>>>>> I really like your suggestion regarding the four environments. It >>>>>> absolutely make sense to declare environment constraints (like SE, >>>>>> Servlet, EE, etc.) instead of listing concrete containers. >>>>>> >>>>>> Christian >>>>>> >>>>>> >>>>>> >>>>>> 2011/12/17 Mark Struberg <[email protected]>: >>>>>>> Hi Christian! >>>>>>> >>>>>>> >>>>>>> @TargetContainer sounds good, but I'd rather make it a >>>>>>> >>>>>>> >>>>>>> @TargetEnvironment() >>>>>>> >>>>>>> with values: SE, Servlet, EE-light, EE-full >>>>>>> >>>>>>> The excludes should be done in the single test integration >>>> modules, and the >>>>>> goal should be to get down to zero excluded tests for all containers. >>>>>>> >>>>>>> >>>>>>> @Jason: that mixture for the tests looks fine to me as well. Can >>>> you please >>>>>> hack a sample structure in the DeltaSpikePlayground git repo? >>>>>>> >>>>>>> >>>>>>> LieGrue, >>>>>>> strub >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>> ________________________________ >>>>>>>> From: Christian Kaltepoth <[email protected]> >>>>>>>> To: [email protected] >>>>>>>> Sent: Saturday, December 17, 2011 7:55 AM >>>>>>>> Subject: Re: basic decisions: test setup >>>>>>>> >>>>>>>> I would like to add some "non-binding" comments here >>>> regarding >>>>>> 3). >>>>>>>> >>>>>>>> Actually I really like the concept Jason described and which is >>>>>>>> currently used in Seam3. Having worked with it a bit I can >>>> confirm >>>>>>>> that it runs fine. And somehow it's the best of both worlds >>>> (in >>>>>> regard >>>>>>>> to 3a vs 3b). >>>>>>>> >>>>>>>> That the full integration test suite with all containers cannot >>>> run in >>>>>>>> a single Maven invocation is IMHO not a real problem. We could >>>> write a >>>>>>>> simple script for this. >>>>>>>> >>>>>>>> The only thing that could be optimized in this setup is the >>>> separation >>>>>>>> of container dependent tests via package name. But perhaps this >>>> could >>>>>>>> be solved by Arquillian? What I would really like to see is the >>>>>>>> ability to configure a test so it is only executed in a >>>> specific >>>>>>>> container (something similar to @TargetsContainer). This way >>>> tests >>>>>>>> that require to be run in a full container could be executed >>>> only in >>>>>>>> JBoss and Glassfish but would be skipped in embedded Weld. >>>>>>>> >>>>>>>> Christian >>>>>>>> >>>>>>>> >>>>>>>> 2011/12/16 Jason Porter <[email protected]>: >>>>>>>>> Comments inline. >>>>>>>>> >>>>>>>>> On Fri, Dec 16, 2011 at 01:38, Mark Struberg >>>>>> <[email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi! >>>>>>>>>> >>>>>>>>>> Another round of discussion - this time about how to >>>> setup our >>>>>> unit >>>>>>>>>> testing and integration testing. >>>>>>>>>> >>>>>>>>>> I think it's pretty much clear that we will gonna >>>> use >>>>>> Arquillian for all >>>>>>>>>> our tests. That was also fundamental part of the >>>> incubator >>>>>> proposal. >>>>>>>>>> And instead of inventing yet another abstraction >>>> layer, I'd >>>>>> favour to just >>>>>>>>>> use Arquillian and contribute all things we need to >>>> this >>>>>> project. >>>>>>>>>> So here a formal vote >>>>>>>>>> >>>>>>>>>> 1.) >>>>>>>>>> >>>>>>>>>> for using Arquillian as test integration framework. >>>>>>>>>> +1 from me >>>>>>>>>> >>>>>>>>> >>>>>>>>> +1 from me >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> 2.) >>>>>>>>>> >>>>>>>>>> What do we like to use from the unit tests itself? >>>> TestNG or >>>>>> JUnit? >>>>>>>>>> JUnit has better Arquillian integration and TestNG is >>>> better >>>>>> for 'real >>>>>>>>>> world' projects, because it allows to define test >>>>>> dependencies. >>>>>>>>>> So which one to take? >>>>>>>>>> But we should definitely only use 1 of the two >>>> exclusively! >>>>>>>>>> >>>>>>>>> >>>>>>>>> +1 for using one exclusively >>>>>>>>> >>>>>>>>> As for which, I'm also +0 I like both for different >>>> reasons, >>>>>> but JUnit is >>>>>>>>> certainly the easier of the two to use with Arquillian. >>>>>>>>> >>>>>>>>> Also, are we open to having other languages for tests or >>>> using >>>>>> other >>>>>>>>> testing structures such as spock (it works with >>>> Arquillian, BTW), >>>>>> RSpec, >>>>>>>>> scala, etc.? >>>>>>>>> >>>>>>>>> >>>>>>>>>> 3.) >>>>>>>>>> 'Integrated' tests vs 'Integration >>>> Tests' >>>>>>>>>> We have 2 options to test our projects >>>>>>>>>> >>>>>>>>>> 3.a.) create a full unit test in each module (e.g. >>>>>> deltaspike/core/impl) >>>>>>>>>> and add a profile for each and every server (maybe we >>>> can trim >>>>>> this down >>>>>>>>>> with pom imports?). >>>>>>>>>> Then run the build with one after each other: >>>>>>>>>> >>>>>>>>>> $> mvn clean test -Powb >>>>>>>>>> $> mvn clean test -Pweld >>>>>>>>>> $> mvn clean test -Powb-tc >>>>>>>>>> $> mvn clean test -Pweld-tc >>>>>>>>>> $> mvn clean test -Pgf31 >>>>>>>>>> $> mvn clean test -Pas7 >>>>>>>>>> $> mvn clean test -Ptomee >>>>>>>>>> >>>>>>>>>> This might pretty much blow up our poms... >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> 3.b) create a full unit test in each module (e.g. >>>>>> deltaspike/core/impl) >>>>>>>>>> and add only 2 profiles directly ('weld' and >>>>>> 'owb' (probably resin later)) >>>>>>>>>> Then add a deltaspike/test/base/core with the very >>>> basic parent >>>>>> stuff and >>>>>>>>>> 1 module for each integrated container, e.g. >>>>>>>>>> deltaspike/test/weld-tc >>>>>>>>>> deltaspike/test/owb-tc >>>>>>>>>> deltaspike/test/weld-jetty >>>>>>>>>> deltaspike/test/owb-jetty >>>>>>>>>> deltaspike/test/tomee >>>>>>>>>> deltaspike/test/gf31 >>>>>>>>>> deltaspike/test/geronimo >>>>>>>>>> deltaspike/test/websphere >>>>>>>>>> deltaspike/test/as7 >>>>>>>>>> deltaspike/test/as6 >>>>>>>>>> etc. >>>>>>>>>> >>>>>>>>>> The initial setup costs are higher, but it would be >>>> pretty easy >>>>>> to add new >>>>>>>>>> containers that way. >>>>>>>>>> >>>>>>>>>> Which route shall we take? >>>>>>>>>> >>>>>>>>> >>>>>>>>> In Seam 3 we have an approach which is a little bit of a >>>> mixture of >>>>>> both. >>>>>>>>> We have regular unit tests (with mockito, stubs, etc) >>>> under >>>>>>>>> impl/src/test/... and then we have a separate module >>>> called >>>>>> testsuite which >>>>>>>>> contained all the integration tests. We used pom imports >>>> to get all >>>>>> the >>>>>>>>> containers setup correctly and then had different profiles >>>> which >>>>>> pulled in >>>>>>>>> the poms and also configured surefire for that server. >>>> Inside the >>>>>> code >>>>>>>>> structure we separated the servers by package: common, >>>> jbossas, >>>>>> glassfish, >>>>>>>>> etc. That way all of the tests were in one location, they >>>> were easy >>>>>> to >>>>>>>>> execute, the tests stayed DRY, and the IDE wasn't much >>>> of a >>>>>> problem. The >>>>>>>>> two downsides we had were 1) the extra config in the pom >>>> and 2) you >>>>>> can't >>>>>>>>> run the full integration suite in one maven execution. We >>>> spent >>>>>> quite a bit >>>>>>>>> of time examining options and trying some of them out. >>>> This was the >>>>>> best we >>>>>>>>> came up with, and also the one Aslak thought would work >>>> best as >>>>>> well. >>>>>>>>> >>>>>>>>> >>>>>>>>>> LieGrue, >>>>>>>>>> strub >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Jason Porter >>>>>>>>> http://lightguard-jp.blogspot.com >>>>>>>>> http://twitter.com/lightguardjp >>>>>>>>> >>>>>>>>> Software Engineer >>>>>>>>> Open Source Advocate >>>>>>>>> Author of Seam Catch - Next Generation Java Exception >>>> Handling >>>>>>>>> >>>>>>>>> PGP key id: 926CCFF5 >>>>>>>>> PGP key available at: keyserver.net, pgp.mit.edu >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Christian Kaltepoth >>>>>>>> Blog: http://chkal.blogspot.com/ >>>>>>>> Twitter: http://twitter.com/chkal >>>>>>>> >>>>>>>> >>>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Christian Kaltepoth >>>>>> Blog: http://chkal.blogspot.com/ >>>>>> Twitter: http://twitter.com/chkal >>>>>> >>>> >>>> >>>> >>>> -- >>>> Christian Kaltepoth >>>> Blog: http://chkal.blogspot.com/ >>>> Twitter: http://twitter.com/chkal >>>> >> >> >> > > > -- > Dan Allen > Principal Software Engineer, Red Hat | Author of Seam in Action > Registered Linux User #231597 > > http://www.google.com/profiles/dan.j.allen#about > http://mojavelinux.com > http://mojavelinux.com/seaminaction
