Re: http-based/teleporter unit tests vs. paxexam integraiton tests (was: RE: flaky integration tests with teleporte rule)
> >> 2. paxexam integration tests do not run in the IDE (at least not in >> eclipse/junit runner - the problem seems to be that the pom information is >> not available for version lookup). this is only nice-to-have. > > Neither do I use Eclipse nor do I run ITs from my IDE, but I'm sure that > issue > can be solved. > The simplest workaround for this is to manually add a few dependencies to your Eclipse project - you don't need to add the bundles that are launched by pax exam, but some of pax exams dependencies itself and then this works perfectly. You can also invoke the pax exam tests in debug mode from maven and then connect to it using remote debugging. I'm not saying this is perfect and if this could be simplified would be great, but it's not rocket science either. Carsten -- Carsten Ziegeler Adobe Research Switzerland cziege...@apache.org
Re: Fwd: http-based/teleporter unit tests vs. paxexam integraiton tests (was: RE: flaky integration tests with teleporte rule)
On Wednesday 07 September 2016 09:29:15 Andrei Dulvac wrote: > -- Forwarded message - > From: Stefan Seifert <sseif...@pro-vision.de> > Date: Tue, Sep 6, 2016 at 11:25 PM > Subject: http-based/teleporter unit tests vs. paxexam integraiton tests > (was: RE: flaky integration tests with teleporte rule) > To: dev@sling.apache.org <dev@sling.apache.org> > > Hi. > > I'd also like to recommend the newly added > https://github.com/apache/sling/tree/trunk/testing/junit/rules for > HTTP-based tests. Having HTTP-level tests make sense to me when you have > multiple instances, for example or want to have functional tests that cover > a whole bunch of functionality. hi Andrei, managing tests over HTTP and invoking functionality over HTTP in tests (which I do in Scripting Thymeleaf also) are different things. Regards, O. > >To be honest, I see only 2, maybe 3 cases where managing tests over HTTP > >makes > >sense: > > > >1. you do not want to start your (fat and slow) application under test from > >the tests itself > >2. you need a different runner than PaxExam (runners and rules are gone in > >JUnit 5) > >3. reuse of tests like done in o.a.s.karaf-launchpad-oak-tar-integration- > >tests > > > >In any other case I would use Pax Exam and @Inject to ensure required > >services > >are available before tests are executed. > > this may be true. > > before again using a http-based test (also with latest > slingstart/provisoining features and teleporter rule they are much nicer > than before) i played some minutes with paxexam and was drawn back by two > isses (without further thinking or researching on them): > > 1. within the codebase of org.apache.sling.testing.paxexam a long list of > dependencies was hard-coded. this is usually not what i want. using > slingstart i reference one sling launchpad version i want to target (e.g. > 8) and deploy only few additional bundles. then i can be sure that my > application works exactly with this versions of dependencies and felix > framework, not only with the latest and greatest versions. if paxexam could > be equipped with an "externalized" dependency management based on sling > provisioning files and their modularization/overlay/inheritance features > this would be great. > > 2. paxexam integration tests do not run in the IDE (at least not in > eclipse/junit runner - the problem seems to be that the pom information is > not available for version lookup). this is only nice-to-have. > > stefan
Re: http-based/teleporter unit tests vs. paxexam integraiton tests (was: RE: flaky integration tests with teleporte rule)
On Tuesday 06 September 2016 21:25:45 Stefan Seifert wrote: > >To be honest, I see only 2, maybe 3 cases where managing tests over HTTP > >makes > >sense: > > > >1. you do not want to start your (fat and slow) application under test from > >the tests itself > >2. you need a different runner than PaxExam (runners and rules are gone in > >JUnit 5) > >3. reuse of tests like done in o.a.s.karaf-launchpad-oak-tar-integration- > >tests > > > >In any other case I would use Pax Exam and @Inject to ensure required > >services > >are available before tests are executed. > > this may be true. > > before again using a http-based test (also with latest > slingstart/provisoining features and teleporter rule they are much nicer > than before) i played some minutes with paxexam and was drawn back by two > isses (without further thinking or researching on them): Note: I wrote Pax Exam, not Sling Testing PaxExam. I've added Sling Testing PaxExam, because all of our existing testing options have one or the other drawback which prevent me from using them with *my* requirements. > 1. within the codebase of org.apache.sling.testing.paxexam a long list of > dependencies was hard-coded. this is usually not what i want. using > slingstart i reference one sling launchpad version i want to target (e.g. > 8) and deploy only few additional bundles. then i can be sure that my > application works exactly with this versions of dependencies and felix > framework, not only with the latest and greatest versions. There is currently only Launchpad 8 with slingstart, you can not choose between different Launchpad versions. Slingstart is more or less *one* long list of hard-coded dependencies where Testing PaxExam provides a bunch of "features" (sets of dependencies) to use. A release of Testing PaxExam will reflect a point in time with a working (tested) set of dependencies and configurations. And there will be a release in the future which matches Launchpad 9 of course. Sightly, which occupies the .html extension, comes with Launchpad 8 and this prevents testing Scripting Thymeleaf with HTML scripts using .html extension. The removal of bundles from slingstart was added quite recently, right? Another drawback of slingstart is the requirement for an extra IT module. With Testing PaxExam I can have all in one module. > if paxexam could > be equipped with an "externalized" dependency management based on sling > provisioning files and their modularization/overlay/inheritance features > this would be great. You can override Testing PaxExam's "features" and use a different VersionResolver or override single versions. Bertrand came up with a requirement to pick up versions from project – added with SLING-6038. > 2. paxexam integration tests do not run in the IDE (at least not in > eclipse/junit runner - the problem seems to be that the pom information is > not available for version lookup). this is only nice-to-have. Neither do I use Eclipse nor do I run ITs from my IDE, but I'm sure that issue can be solved. Regards, O. > stefan
Fwd: http-based/teleporter unit tests vs. paxexam integraiton tests (was: RE: flaky integration tests with teleporte rule)
-- Forwarded message - From: Stefan Seifert <sseif...@pro-vision.de> Date: Tue, Sep 6, 2016 at 11:25 PM Subject: http-based/teleporter unit tests vs. paxexam integraiton tests (was: RE: flaky integration tests with teleporte rule) To: dev@sling.apache.org <dev@sling.apache.org> Hi. I'd also like to recommend the newly added https://github.com/apache/sling/tree/trunk/testing/junit/rules for HTTP-based tests. Having HTTP-level tests make sense to me when you have multiple instances, for example or want to have functional tests that cover a whole bunch of functionality. >To be honest, I see only 2, maybe 3 cases where managing tests over HTTP >makes >sense: > >1. you do not want to start your (fat and slow) application under test from >the tests itself >2. you need a different runner than PaxExam (runners and rules are gone in >JUnit 5) >3. reuse of tests like done in o.a.s.karaf-launchpad-oak-tar-integration- >tests > >In any other case I would use Pax Exam and @Inject to ensure required >services >are available before tests are executed. this may be true. before again using a http-based test (also with latest slingstart/provisoining features and teleporter rule they are much nicer than before) i played some minutes with paxexam and was drawn back by two isses (without further thinking or researching on them): 1. within the codebase of org.apache.sling.testing.paxexam a long list of dependencies was hard-coded. this is usually not what i want. using slingstart i reference one sling launchpad version i want to target (e.g. 8) and deploy only few additional bundles. then i can be sure that my application works exactly with this versions of dependencies and felix framework, not only with the latest and greatest versions. if paxexam could be equipped with an "externalized" dependency management based on sling provisioning files and their modularization/overlay/inheritance features this would be great. 2. paxexam integration tests do not run in the IDE (at least not in eclipse/junit runner - the problem seems to be that the pom information is not available for version lookup). this is only nice-to-have. stefan
http-based/teleporter unit tests vs. paxexam integraiton tests (was: RE: flaky integration tests with teleporte rule)
>To be honest, I see only 2, maybe 3 cases where managing tests over HTTP >makes >sense: > >1. you do not want to start your (fat and slow) application under test from >the tests itself >2. you need a different runner than PaxExam (runners and rules are gone in >JUnit 5) >3. reuse of tests like done in o.a.s.karaf-launchpad-oak-tar-integration- >tests > >In any other case I would use Pax Exam and @Inject to ensure required >services >are available before tests are executed. this may be true. before again using a http-based test (also with latest slingstart/provisoining features and teleporter rule they are much nicer than before) i played some minutes with paxexam and was drawn back by two isses (without further thinking or researching on them): 1. within the codebase of org.apache.sling.testing.paxexam a long list of dependencies was hard-coded. this is usually not what i want. using slingstart i reference one sling launchpad version i want to target (e.g. 8) and deploy only few additional bundles. then i can be sure that my application works exactly with this versions of dependencies and felix framework, not only with the latest and greatest versions. if paxexam could be equipped with an "externalized" dependency management based on sling provisioning files and their modularization/overlay/inheritance features this would be great. 2. paxexam integration tests do not run in the IDE (at least not in eclipse/junit runner - the problem seems to be that the pom information is not available for version lookup). this is only nice-to-have. stefan
