On Thu, Dec 17, 2015 at 12:17 AM, Afkham Azeez <[email protected]> wrote:
> Check the attached class which programatically adds all the bundles > necessary just to get MSS, our smallest product running & its complexity. > It is not practical to do this for all products. The correct way to > integration test products is to use the product packs itself, not an full > product environment created using PAX exam. > What karaf people are doing is, first start the bare bone karaf distribution[1], then they install and uninstall new features to that karaf distribution while running the OSGi test cases [2] ,[3] [1]. https://github.com/apache/karaf/blob/master/itests/src/test/java/org/apache/karaf/itests/KarafTestSupport.java#L131 [2]. https://github.com/apache/karaf/blob/master/itests/src/test/java/org/apache/karaf/itests/JndiTest.java#L37 [3]. https://github.com/apache/karaf/blob/master/itests/src/test/java/org/apache/karaf/itests/JmsTest.java#L42 > > On Thu, Dec 17, 2015 at 12:12 AM, Afkham Azeez <[email protected]> wrote: > >> 1. PAX exam tests being successful does not imply that the product pack >> will properly start >> This is because the way in which bundles are added to the PAX exam >> runtime is completely different from how the features are installed into >> product packs >> >> 2. Installing 100s of bundles programatically to test a product is not >> practical and a very cumbersome error prone approach >> I don't believe PAX exam was designed to do this. IMO, PAX exam should be >> used to test a bundle or may be just a few bundles, and also instead of a >> fully fledged runtime like a product, it is designed to have mock OSGi >> services & bundles along with the bundles under test. >> >> 3. Correct approach for product integration testing is to use the product >> pack >> This is because of 1 & 2 above. In Carbon 4, we use the pack, extract it >> and run tests against it. I still believe that is the correct approach >> which needs to be followed in Carbon 5 based products as well. >> >> 4. PAX exam along with mock bundles & mock OSGi services for testing OSGi >> bundles >> I think PAX exam will be useful for unit testing individual bundles >> >> Thanks >> Azeez >> >> >> On Wed, Dec 16, 2015 at 9:49 PM, Aruna Karunarathna <[email protected]> >> wrote: >> >>> >>> >>> On Wed, Dec 16, 2015 at 8:22 PM, Afkham Azeez <[email protected]> wrote: >>> >>>> I have been trying to develop PAX exam based tests for MSS and it seems >>>> very hard to use because there is little help for troubleshooting. MSS has >>>> just one feature and it has been a struggle to get the test setup. With >>>> much more complex products, it would be a total nightmare. A lot of time is >>>> wasted due to trial and error method of troubleshooting. >>>> >>>> Hi Azeez, >>> >>> Yes, agree on you that, it's not very easy to configure the test setup >>> at the beginning, when we were integrating it to kernel it took some time >>> to understand the anatomy of PAX-Exam test cases and the first test case to >>> up and running. >>> First what we did was, make the base test case solid, and make sure the >>> server is up and running, For example, when the test container starts it >>> does not start as we run the carbon.sh file, since the PAX-Test container >>> doesn't use the org.wso2.carbon.launcher.jar to boot up the server, so we >>> had to simulate that before starting the PAX-Exam container [1]. But after >>> that >>> people wrote test cases without much hassle. >>> >>> After those efforts, I think we make use of Pax to solve most of our in >>> container testing scenarios, We were able to make sure each OSGi services >>> are running solidly. So after code changes, it made very easy to identify >>> the broken OSGi services etc, without manual testing. also it helped to >>> simulate complex cases such as start up coordination. >>> >>> >>>> Either we are using PAX exam incorrectly or PAX exam is not best >>>> framework to be adopted. >>>> >>>> I'm not sure that PAX exam complicates things when it comes to >>> products, but for kernel I think we need in container OSGi test cases. So >>> we can verify the services are working accordingly. >>> >>> From what I have experienced so far, in most of our current products, >>> missing OSGi service or some service component issues can only be >>> identified after starting the product. We need to identify these issues at >>> the build time. Either using Pax or some other alternative. >>> >>> [1]. >>> https://github.com/wso2/carbon-kernel/blob/master/tests/osgi-tests/src/test/java/org/wso2/carbon/osgi/utils/OSGiTestUtils.java#L43 >>> >>> Regards, >>> Aruna >>> >>>> Thoughts welcome. >>>> >>>> -- >>>> *Afkham Azeez* >>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>> Member; Apache Software Foundation; http://www.apache.org/ >>>> * <http://www.apache.org/>* >>>> *email: **[email protected]* <[email protected]> >>>> * cell: +94 77 3320919 <%2B94%2077%203320919>blog: * >>>> *http://blog.afkham.org* <http://blog.afkham.org> >>>> *twitter: **http://twitter.com/afkham_azeez* >>>> <http://twitter.com/afkham_azeez> >>>> *linked-in: **http://lk.linkedin.com/in/afkhamazeez >>>> <http://lk.linkedin.com/in/afkhamazeez>* >>>> >>>> *Lean . Enterprise . Middleware* >>>> >>> >>> >>> >>> -- >>> >>> *Aruna Sujith Karunarathna *| Software Engineer >>> WSO2, Inc | lean. enterprise. middleware. >>> #20, Palm Grove, Colombo 03, Sri Lanka >>> Mobile: +94 71 9040362 | Work: +94 112145345 >>> Email: [email protected] | Web: www.wso2.com >>> >>> >> >> >> >> -- >> *Afkham Azeez* >> Director of Architecture; WSO2, Inc.; http://wso2.com >> Member; Apache Software Foundation; http://www.apache.org/ >> * <http://www.apache.org/>* >> *email: **[email protected]* <[email protected]> >> * cell: +94 77 3320919 <%2B94%2077%203320919>blog: * >> *http://blog.afkham.org* <http://blog.afkham.org> >> *twitter: **http://twitter.com/afkham_azeez* >> <http://twitter.com/afkham_azeez> >> *linked-in: **http://lk.linkedin.com/in/afkhamazeez >> <http://lk.linkedin.com/in/afkhamazeez>* >> >> *Lean . Enterprise . Middleware* >> > > > > -- > *Afkham Azeez* > Director of Architecture; WSO2, Inc.; http://wso2.com > Member; Apache Software Foundation; http://www.apache.org/ > * <http://www.apache.org/>* > *email: **[email protected]* <[email protected]> > * cell: +94 77 3320919 <%2B94%2077%203320919>blog: * > *http://blog.afkham.org* <http://blog.afkham.org> > *twitter: **http://twitter.com/afkham_azeez* > <http://twitter.com/afkham_azeez> > *linked-in: **http://lk.linkedin.com/in/afkhamazeez > <http://lk.linkedin.com/in/afkhamazeez>* > > *Lean . Enterprise . Middleware* > -- *Aruna Sujith Karunarathna *| Software Engineer WSO2, Inc | lean. enterprise. middleware. #20, Palm Grove, Colombo 03, Sri Lanka Mobile: +94 71 9040362 | Work: +94 112145345 Email: [email protected] | Web: www.wso2.com
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
