On Fri, Jan 20, 2012 at 12:59 PM, Krishantha Samaraweera < krishan...@wso2.com> wrote:
> > On Fri, Jan 20, 2012 at 9:51 AM, Muhammed Shariq <sha...@wso2.com> wrote: > >> Hi Krishantha et al, >> >> If I am not mistaken we are running these test for products only right, >> at the moment? How about the possibility of running these test on the >> stratos services too ? I had a chat with Dhrashana and we most of the tests >> that covers multitenancy aspect, so its a matter of setting up a stratos >> deployment in the bamboo machins right? I can work with the automation team >> to get this running. IIRC we even used these test suite to smoke test SLive >> after migrations etc, so we already have it. >> > > +1, we have written automation test to run on Stratos services and > products. These tests cover tenant and multi-tenancy aspects as well. Yes > it is matter to setting the things in bamboo machine. But we need to solve > resource problems first. You cannot run startos setup on a single machine. > How about using the machines used to run the Dev environment to set up the trunk stratos builds. If I am not mistaken we have the scripts etc for Dev too. Now that we are moving to the trunk, the Dev environment should run on the trunk builds right? Also it would be ideal since any failures can be patched and verified then and there, so it will save cycles as well. Only issue is we need a to copy the packs to these machines and then set it up (would some scripts, scp etc work ?). Its important to automate this task, else we will be wasting cycles.! If we can somehow figure out a way to set this packs up remotely (ideally in the dev setup) then we could run the tests from the bamboo machines, and generate the results, notification etc. Setting up the dev cloud once a week should be sufficient IMO? > >> My concern is that, once we start developing new features we usually do >> development and testing cycles with the product. Ultimately the services >> gets side lined until quite late in the release cycle. If we can run a full >> test round once a week maybe and get notifications of the failures (or on >> the build status dashboard itself), we wouldn't need to worry about the >> status of the trunk at any time. WDYT ? >> > > +1, As far as you admit that failures are not false positives :)) > Of course I admit that there are issues ;)) > > >> >> On a related note, there was a thread on adding TestNG integration test >> to cover tenant aspects as well, wonder if we have made any progress in >> that front? Thinking about it, we first need to create a tenant, and also >> some of the tenant related test would require a proper setup (with mounting >> etc) prior to running the tests. Guess we need to put more thought into >> that? >> > > Integration tests are different from platform tests. Integration test > covers features that grouped in to products. But Platform test covers > integrated product deployment (whole platform). We already have test to > check tenant aspects. Even you can specify your tenant on the > test configuration and run tests on your tenant. Setting up stratos evn > would be to call startos setup script though the automation framework. We > may need to find a way for distributed deployment of each service. > Yes we need to test the status of the current trunk with exact setup, configs etc. If not we can't be sure the trunks is stable in terms of functionality. Shall we try to come up with a plan and set this up ASAP, as part of the stabilization effort it self? > Thanks, > Krishantha. > > >> On Fri, Jan 20, 2012 at 12:23 AM, Krishantha Samaraweera < >> krishan...@wso2.com> wrote: >> >>> Hi, >>> >>> We have done the $subject on Bamboo machine and it is working. You can >>> find the last successful test build at [1]. >>> >>> Test framework has been configured to use packs generated by trunk >>> build. The test setup consists of five products (ESB, G-Reg, AS, BPS and >>> DSS) with registry mounts on top of Mysql. This include Selenium test for >>> G-Reg as well. I had to exclude Ravana tests due to few issues after >>> updating the Ravana framework. Will get them fix by tomorrow. >>> >>> Thanks, >>> Krishantha. >>> >>> [1] http://wso2.org/bamboo/browse/WSO2CARBONTUNK-AUTOMATIONTEST-13/ >>> >>> _______________________________________________ >>> Carbon-dev mailing list >>> Carbon-dev@wso2.org >>> http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev >>> >>> >> >> >> -- >> Thanks, >> Shariq. >> Phone: +94 777 202 225 >> >> >> _______________________________________________ >> Carbon-dev mailing list >> Carbon-dev@wso2.org >> http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev >> >> > > _______________________________________________ > Carbon-dev mailing list > Carbon-dev@wso2.org > http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev > > -- Thanks, Shariq. Phone: +94 777 202 225
_______________________________________________ Carbon-dev mailing list Carbon-dev@wso2.org http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev