1. PAX exam tests being successful does not imply that the product pack will properly start This is because the way in which bundles are added to the PAX exam runtime is completely different from how the features are installed into product packs
2. Installing 100s of bundles programatically to test a product is not practical and a very cumbersome error prone approach I don't believe PAX exam was designed to do this. IMO, PAX exam should be used to test a bundle or may be just a few bundles, and also instead of a fully fledged runtime like a product, it is designed to have mock OSGi services & bundles along with the bundles under test. 3. Correct approach for product integration testing is to use the product pack This is because of 1 & 2 above. In Carbon 4, we use the pack, extract it and run tests against it. I still believe that is the correct approach which needs to be followed in Carbon 5 based products as well. 4. PAX exam along with mock bundles & mock OSGi services for testing OSGi bundles I think PAX exam will be useful for unit testing individual bundles Thanks Azeez On Wed, Dec 16, 2015 at 9:49 PM, Aruna Karunarathna <[email protected]> wrote: > > > On Wed, Dec 16, 2015 at 8:22 PM, Afkham Azeez <[email protected]> wrote: > >> I have been trying to develop PAX exam based tests for MSS and it seems >> very hard to use because there is little help for troubleshooting. MSS has >> just one feature and it has been a struggle to get the test setup. With >> much more complex products, it would be a total nightmare. A lot of time is >> wasted due to trial and error method of troubleshooting. >> >> Hi Azeez, > > Yes, agree on you that, it's not very easy to configure the test setup at > the beginning, when we were integrating it to kernel it took some time to > understand the anatomy of PAX-Exam test cases and the first test case to up > and running. > First what we did was, make the base test case solid, and make sure the > server is up and running, For example, when the test container starts it > does not start as we run the carbon.sh file, since the PAX-Test container > doesn't use the org.wso2.carbon.launcher.jar to boot up the server, so we > had to simulate that before starting the PAX-Exam container [1]. But after > that > people wrote test cases without much hassle. > > After those efforts, I think we make use of Pax to solve most of our in > container testing scenarios, We were able to make sure each OSGi services > are running solidly. So after code changes, it made very easy to identify > the broken OSGi services etc, without manual testing. also it helped to > simulate complex cases such as start up coordination. > > >> Either we are using PAX exam incorrectly or PAX exam is not best >> framework to be adopted. >> >> I'm not sure that PAX exam complicates things when it comes to products, > but for kernel I think we need in container OSGi test cases. So we can > verify the services are working accordingly. > > From what I have experienced so far, in most of our current products, > missing OSGi service or some service component issues can only be > identified after starting the product. We need to identify these issues at > the build time. Either using Pax or some other alternative. > > [1]. > https://github.com/wso2/carbon-kernel/blob/master/tests/osgi-tests/src/test/java/org/wso2/carbon/osgi/utils/OSGiTestUtils.java#L43 > > Regards, > Aruna > >> Thoughts welcome. >> >> -- >> *Afkham Azeez* >> Director of Architecture; WSO2, Inc.; http://wso2.com >> Member; Apache Software Foundation; http://www.apache.org/ >> * <http://www.apache.org/>* >> *email: **[email protected]* <[email protected]> >> * cell: +94 77 3320919 <%2B94%2077%203320919>blog: * >> *http://blog.afkham.org* <http://blog.afkham.org> >> *twitter: **http://twitter.com/afkham_azeez* >> <http://twitter.com/afkham_azeez> >> *linked-in: **http://lk.linkedin.com/in/afkhamazeez >> <http://lk.linkedin.com/in/afkhamazeez>* >> >> *Lean . Enterprise . Middleware* >> > > > > -- > > *Aruna Sujith Karunarathna *| Software Engineer > WSO2, Inc | lean. enterprise. middleware. > #20, Palm Grove, Colombo 03, Sri Lanka > Mobile: +94 71 9040362 | Work: +94 112145345 > Email: [email protected] | Web: www.wso2.com > > -- *Afkham Azeez* Director of Architecture; WSO2, Inc.; http://wso2.com Member; Apache Software Foundation; http://www.apache.org/ * <http://www.apache.org/>* *email: **[email protected]* <[email protected]> * cell: +94 77 3320919blog: **http://blog.afkham.org* <http://blog.afkham.org> *twitter: **http://twitter.com/afkham_azeez* <http://twitter.com/afkham_azeez> *linked-in: **http://lk.linkedin.com/in/afkhamazeez <http://lk.linkedin.com/in/afkhamazeez>* *Lean . Enterprise . Middleware*
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
