Hi team, here is a short update:
1) The QA user story for destructive testing of OpenStack cloud is on review [1]. 2) The spec for a new framework which will focus on HA/failover and destructive testing is no review [2]. 3) The commit for the new repository is on review [3] as well. [1] https://review.openstack.org/#/c/396142 [2] https://review.openstack.org/#/c/399618 [3] https://review.openstack.org/#/c/374667 On Fri, Oct 14, 2016 at 3:47 AM, Ghanshyam Mann < [email protected]> wrote: > I like os-faults library which can provide the abstraction over different > destructive actions like reboot/poweroff node etc. > > > > But not much clear about Stepler that what all tests it will contain. > Tempest do have scenario tests which can hit the production env with > significant way of testing scenario. > > It can cover the end user scenario also which involves the interaction of > public OpenStack APIs and always in enhancement state by adding more and > more scenario tests. > > > > Few query over Stepler as separate project: > > 1. Is Stepler will cover only tests which hits the node level > action(reboot, HA etc)? > > 2. This should not mix the scenario tests which are in Tempest > scope because that can make confusion for developers (where to add scenario > tests) as well as for tester(from where to run what scenario tests). > > 3. How we make sure those tests run fine or at least run while > adding. > > a. I think devstack enhancement for multi-node etc can do this as > mentioned by you also. > > b. If so then why not adding those tests in Tempest only using > os-faults lib ? > > > > Overall I feel os-faults as lib is really nice idea but tests scope can > go in Tempest under HW_scenario (or something else) till it does not break > basic principle of Tempest. > > > > Thanks > > gmann > > > > *From:* Timur Nurlygayanov [mailto:[email protected]] > *Sent:* 06 October 2016 20:09 > *To:* OpenStack Development Mailing List (not for usage questions) < > [email protected]> > *Subject:* Re: [openstack-dev] [QA] The end-user test suite for OpenStack > clusters > > > > Ken, it is a good idea! > > > > The plan is to develop os-faults as a library which will be able > > to manage cluster nodes and OpenStack services on these nodes. > > It is a good idea to add some Tempest tests which will use > > os-faults library as well for some API tests. > > > > The Stepler framework [1] will use os-faults to perform all destructive > > actions in the clouds (the reboot of nodes, restart of OpenStack services, > > enable/disable network interfaces or some firewall rules and etc). > > > > We need to get +1 from you in [1] to create the repository with > > advanced end-user scenario tests. > > > > Thank you! > > > > [1] https://review.openstack.org/#/c/374667/ > > > > On Tue, Oct 4, 2016 at 8:53 PM, Yaroslav Lobankov <[email protected]> > wrote: > > Hi Ken, > > > > OS-Faults doesn't have any scenarios in the tree yet (the project is two > months old), but you can find some examples of the use in the > os-faults/examples directory. > > > > Regards, > > Yaroslav Lobankov. > > > > On Tue, Oct 4, 2016 at 8:02 PM, Ken'ichi Ohmichi <[email protected]> > wrote: > > Hi Timur, > > Thanks for your explanation. > > > 2016-09-29 6:22 GMT-07:00 Timur Nurlygayanov <[email protected]>: > > > >> I am guessing the above "restart nodes" is for verifying each > >> OpenStack service restarts successfully, right? > > > > Yes, this is right. And we also will check that HA logic for these > > services works correctly (for example, rescheduling of L3 Neutron > > agents for networks). > > > >> But these service scripts are provided by distributors, and Devstack > >> itself doesn't contain service scripts IIUC. > >> So I'd like to know how to verify it on Devstack clouds. > > > > Yes, DevStack doesn't support many scenarios which are actual > > and should be supported on the production clouds. > > It will be not possible to run all advanced test scenarios for DevStack > > clouds, > > just because DevStack can't deploy OpenStack cloud with 3 controllers > > now (so, probably it will be possible in the future). > > > > Of course, some advanced scenarios will support DevStack clouds, > > for example, some test scenarios which are based on customer-found > > issues from the real production clouds, like upload of the large images > > (100+ Gb) > > to Glance with Swift backend. Such cases are important for verification > of > > pre-production environments, but not very important for CI gate jobs. > > > > It is also important to note that in these advanced cases we are > targeting > > to check not only the logic of Python code, but also the correct > > configuration > > of all OpenStack components on some pre-production OpenStack clusters. > > I guessed some part of os-faults can be moved to Tempest if os-faults > contains API tests for enabling/disabling OpenStack services. > Then, os-faults would be able to concentrate on more destructive tests > like rebooting physical nodes, etc. > However, I could not find any actual scenarios on current os-faults > (https://github.com/openstack/os-faults). > That seems to just contain some abstraction layers and unit tests. Can > we see actual test scenarios of os-faults ? > Maybe I missed something. > > Thanks > Ken Ohmichi > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: [email protected]?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: [email protected]?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > > -- > > > > Timur, > > Senior QA Manager > > OpenStack Projects > > Mirantis Inc > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: [email protected]?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > -- Timur, QA Manager OpenStack Projects Mirantis Inc
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
