Yes, in the case of kernel, you have used PAX exam to install all the kernel bundles and then test the relevant features. Ideally, you would add mock bundles and then test just one or a few real bundles. So this is a unit test for OSGi bundles. That should have been the correct approach.
Also, does PAX exam have a mode to support bundles.info? If so it will be easy because providing all the bundles, including the kernel bundles, when you are testing a particular feature in a different bundle is cumbersome. Arun had mentioned that if the PAX exam tests run successfully, you can assume that the product will start successfully. That will not be the case because the way in which the PAX exam runtime starts up is different from how a product runtime starts up. On Thu, Dec 17, 2015 at 11:10 AM, Sameera Jayasoma <[email protected]> wrote: > Hi Azeez, > > Pax Exam is not a replacement for our integration tests. It is a way to > run unit tests which require an OSGi environment. > > Pax Exam is the most suitable solution to test Carbon Kernel and our > components (repos). We need to evaluate Pax Exam for testing our middleware > products. Carbon Kernel does not have transports, hence its bit hard to > execute integration tests. Carbon Kernel provides different OSGI utilities > for products. Therefore we need to run unit tests in an OSGI environment. > This is a must for Carbon kernel. > > On Thu, Dec 17, 2015 at 12:12 AM, Afkham Azeez <[email protected]> wrote: > >> 1. PAX exam tests being successful does not imply that the product pack >> will properly start >> This is because the way in which bundles are added to the PAX exam >> runtime is completely different from how the features are installed into >> product packs >> > > I don't agree with this point. :) Let me explain. > > We load all the bundles from the bundles.info file into a clean OSGI > environments . In PAX Exam we load all the bundles programmatically into a > clean environment plus Pax Exam bundles. Only difference is the Pax Exam > bundles. > > >> >> 2. Installing 100s of bundles programatically to test a product is not >> practical and a very cumbersome error prone approach >> I don't believe PAX exam was designed to do this. IMO, PAX exam should be >> used to test a bundle or may be just a few bundles, and also instead of a >> fully fledged runtime like a product, it is designed to have mock OSGi >> services & bundles along with the bundles under test. >> > > One option is to load all bundles from the bundles.info file. This will > make sure Pax Exam tests are up to date with product changes.. > > > >> 3. Correct approach for product integration testing is to use the product >> pack >> This is because of 1 & 2 above. In Carbon 4, we use the pack, extract it >> and run tests against it. I still believe that is the correct approach >> which needs to be followed in Carbon 5 based products as well. >> > > Yes this is true for products. We have to go with integration test > approach for our products. Pax Exam can be used to test functionalities > which requires OSGi in Carbon kernel and components. > > >> >> 4. PAX exam along with mock bundles & mock OSGi services for testing OSGi >> bundles >> I think PAX exam will be useful for unit testing individual bundles >> > > This is true. > > > Thanks, > Sameera. > > > >> >> Thanks >> Azeez >> >> >> On Wed, Dec 16, 2015 at 9:49 PM, Aruna Karunarathna <[email protected]> >> wrote: >> >>> >>> >>> On Wed, Dec 16, 2015 at 8:22 PM, Afkham Azeez <[email protected]> wrote: >>> >>>> I have been trying to develop PAX exam based tests for MSS and it seems >>>> very hard to use because there is little help for troubleshooting. MSS has >>>> just one feature and it has been a struggle to get the test setup. With >>>> much more complex products, it would be a total nightmare. A lot of time is >>>> wasted due to trial and error method of troubleshooting. >>>> >>>> Hi Azeez, >>> >>> Yes, agree on you that, it's not very easy to configure the test setup >>> at the beginning, when we were integrating it to kernel it took some time >>> to understand the anatomy of PAX-Exam test cases and the first test case to >>> up and running. >>> First what we did was, make the base test case solid, and make sure the >>> server is up and running, For example, when the test container starts it >>> does not start as we run the carbon.sh file, since the PAX-Test container >>> doesn't use the org.wso2.carbon.launcher.jar to boot up the server, so we >>> had to simulate that before starting the PAX-Exam container [1]. But after >>> that >>> people wrote test cases without much hassle. >>> >>> After those efforts, I think we make use of Pax to solve most of our in >>> container testing scenarios, We were able to make sure each OSGi services >>> are running solidly. So after code changes, it made very easy to identify >>> the broken OSGi services etc, without manual testing. also it helped to >>> simulate complex cases such as start up coordination. >>> >>> >>>> Either we are using PAX exam incorrectly or PAX exam is not best >>>> framework to be adopted. >>>> >>>> I'm not sure that PAX exam complicates things when it comes to >>> products, but for kernel I think we need in container OSGi test cases. So >>> we can verify the services are working accordingly. >>> >>> From what I have experienced so far, in most of our current products, >>> missing OSGi service or some service component issues can only be >>> identified after starting the product. We need to identify these issues at >>> the build time. Either using Pax or some other alternative. >>> >>> [1]. >>> https://github.com/wso2/carbon-kernel/blob/master/tests/osgi-tests/src/test/java/org/wso2/carbon/osgi/utils/OSGiTestUtils.java#L43 >>> >>> Regards, >>> Aruna >>> >>>> Thoughts welcome. >>>> >>>> -- >>>> *Afkham Azeez* >>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>> Member; Apache Software Foundation; http://www.apache.org/ >>>> * <http://www.apache.org/>* >>>> *email: **[email protected]* <[email protected]> >>>> * cell: +94 77 3320919 <%2B94%2077%203320919>blog: * >>>> *http://blog.afkham.org* <http://blog.afkham.org> >>>> *twitter: **http://twitter.com/afkham_azeez* >>>> <http://twitter.com/afkham_azeez> >>>> *linked-in: **http://lk.linkedin.com/in/afkhamazeez >>>> <http://lk.linkedin.com/in/afkhamazeez>* >>>> >>>> *Lean . Enterprise . Middleware* >>>> >>> >>> >>> >>> -- >>> >>> *Aruna Sujith Karunarathna *| Software Engineer >>> WSO2, Inc | lean. enterprise. middleware. >>> #20, Palm Grove, Colombo 03, Sri Lanka >>> Mobile: +94 71 9040362 | Work: +94 112145345 >>> Email: [email protected] | Web: www.wso2.com >>> >>> >> >> >> >> -- >> *Afkham Azeez* >> Director of Architecture; WSO2, Inc.; http://wso2.com >> Member; Apache Software Foundation; http://www.apache.org/ >> * <http://www.apache.org/>* >> *email: **[email protected]* <[email protected]> >> * cell: +94 77 3320919 <%2B94%2077%203320919>blog: * >> *http://blog.afkham.org* <http://blog.afkham.org> >> *twitter: **http://twitter.com/afkham_azeez* >> <http://twitter.com/afkham_azeez> >> *linked-in: **http://lk.linkedin.com/in/afkhamazeez >> <http://lk.linkedin.com/in/afkhamazeez>* >> >> *Lean . Enterprise . Middleware* >> > > > > -- > Sameera Jayasoma, > Software Architect, > > WSO2, Inc. (http://wso2.com) > email: [email protected] > blog: http://blog.sameera.org > twitter: https://twitter.com/sameerajayasoma > flickr: http://www.flickr.com/photos/sameera-jayasoma/collections > Mobile: 0094776364456 > > Lean . Enterprise . Middleware > > -- *Afkham Azeez* Director of Architecture; WSO2, Inc.; http://wso2.com Member; Apache Software Foundation; http://www.apache.org/ * <http://www.apache.org/>* *email: **[email protected]* <[email protected]> * cell: +94 77 3320919blog: **http://blog.afkham.org* <http://blog.afkham.org> *twitter: **http://twitter.com/afkham_azeez* <http://twitter.com/afkham_azeez> *linked-in: **http://lk.linkedin.com/in/afkhamazeez <http://lk.linkedin.com/in/afkhamazeez>* *Lean . Enterprise . Middleware*
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
