Hi Dims, Hi gmann, Thanks guys. Slowly recovering from PTG jet lag....
OK then.., I will start working on this. I would also like to coordinate this work with LCOO [1] team, where we can get valuable feedback and contributions.. I will give you update on next QA IRC on 0900 UTC meeting, which is 9th of March? (1700 UTC meeting is difficult for me, however I will definitely be there if needed) [1] https://wiki.openstack.org/wiki/LCOO --- Regards, Sampath On Fri, Feb 24, 2017 at 8:57 PM, Ghanshyam Mann <[email protected]> wrote: > Yea, agree with dims. > > Sampath , > > Thanks for taking over this, it is really great help. Please update the > current spec with approaches you have. Timur help will be great if he show > up sometime. > > Also we will add destructive testing as one of weekly meeting agenda and > make sure you will get all help & required attention from QA team. > > -gmann > > > On Fri, Feb 24, 2017 at 7:26 AM, Davanum Srinivas <[email protected]> wrote: >> >> Sampath, >> >> I am not sure if you will hear back from Timur soon as he may not be >> working on this stuff anymore (in Mirantis). So i'd recommend picking >> up the work if you don't hear from him soon. >> >> Thanks, >> Dims >> >> On Thu, Feb 23, 2017 at 3:41 PM, Sam P <[email protected]> wrote: >> > Hi Timur, >> > >> > The current status of this work is, >> > 1) The QA user story for destructive testing of OpenStack cloud [1] is >> > merged . >> > 2) The spec for a new framework which will focus on HA/failover and >> > destructive testing [2] has no update since Nov 30 2016. >> > 3) The commit for the new repository [3] has abandoned due to no >> > update after Nov 29 2016. >> > >> > Currently, I am working on 3rd party destructive/HA testing CI for >> > Masakari[4] and very much interested in this work. >> > I will keep working on [1] with PWG. >> > Please let me know your plans for [2], and [3]. >> > If it is difficult for you to continue, I would love to continue your >> > work on [2], and [3]. >> > >> > [1] https://review.openstack.org/#/c/396142 >> > [2] https://review.openstack.org/#/c/399618 >> > [3] https://review.openstack.org/#/c/374667 >> > [4] https://wiki.openstack.org/wiki/Masakari >> > --- Regards, >> > Sampath >> > >> > >> > >> > On Mon, Nov 28, 2016 at 6:37 AM, Timur Nurlygayanov >> > <[email protected]> wrote: >> >> Hi team, >> >> >> >> here is a short update: >> >> >> >> 1) The QA user story for destructive testing of OpenStack cloud is on >> >> review >> >> [1]. >> >> 2) The spec for a new framework which will focus on HA/failover and >> >> destructive testing is no review [2]. >> >> 3) The commit for the new repository is on review [3] as well. >> >> >> >> [1] https://review.openstack.org/#/c/396142 >> >> [2] https://review.openstack.org/#/c/399618 >> >> [3] https://review.openstack.org/#/c/374667 >> >> >> >> >> >> On Fri, Oct 14, 2016 at 3:47 AM, Ghanshyam Mann >> >> <[email protected]> wrote: >> >>> >> >>> I like os-faults library which can provide the abstraction over >> >>> different >> >>> destructive actions like reboot/poweroff node etc. >> >>> >> >>> >> >>> >> >>> But not much clear about Stepler that what all tests it will contain. >> >>> Tempest do have scenario tests which can hit the production env with >> >>> significant way of testing scenario. >> >>> >> >>> It can cover the end user scenario also which involves the interaction >> >>> of >> >>> public OpenStack APIs and always in enhancement state by adding more >> >>> and >> >>> more scenario tests. >> >>> >> >>> >> >>> >> >>> Few query over Stepler as separate project: >> >>> >> >>> 1. Is Stepler will cover only tests which hits the node level >> >>> action(reboot, HA etc)? >> >>> >> >>> 2. This should not mix the scenario tests which are in Tempest >> >>> scope >> >>> because that can make confusion for developers (where to add scenario >> >>> tests) >> >>> as well as for tester(from where to run what scenario tests). >> >>> >> >>> 3. How we make sure those tests run fine or at least run while >> >>> adding. >> >>> >> >>> a. I think devstack enhancement for multi-node etc can do this >> >>> as >> >>> mentioned by you also. >> >>> >> >>> b. If so then why not adding those tests in Tempest only using >> >>> os-faults lib ? >> >>> >> >>> >> >>> >> >>> Overall I feel os-faults as lib is really nice idea but tests scope >> >>> can >> >>> go in Tempest under HW_scenario (or something else) till it does not >> >>> break >> >>> basic principle of Tempest. >> >>> >> >>> >> >>> >> >>> Thanks >> >>> >> >>> gmann >> >>> >> >>> >> >>> >> >>> From: Timur Nurlygayanov [mailto:[email protected]] >> >>> Sent: 06 October 2016 20:09 >> >>> To: OpenStack Development Mailing List (not for usage questions) >> >>> <[email protected]> >> >>> Subject: Re: [openstack-dev] [QA] The end-user test suite for >> >>> OpenStack >> >>> clusters >> >>> >> >>> >> >>> >> >>> Ken, it is a good idea! >> >>> >> >>> >> >>> >> >>> The plan is to develop os-faults as a library which will be able >> >>> >> >>> to manage cluster nodes and OpenStack services on these nodes. >> >>> >> >>> It is a good idea to add some Tempest tests which will use >> >>> >> >>> os-faults library as well for some API tests. >> >>> >> >>> >> >>> >> >>> The Stepler framework [1] will use os-faults to perform all >> >>> destructive >> >>> >> >>> actions in the clouds (the reboot of nodes, restart of OpenStack >> >>> services, >> >>> >> >>> enable/disable network interfaces or some firewall rules and etc). >> >>> >> >>> >> >>> >> >>> We need to get +1 from you in [1] to create the repository with >> >>> >> >>> advanced end-user scenario tests. >> >>> >> >>> >> >>> >> >>> Thank you! >> >>> >> >>> >> >>> >> >>> [1] https://review.openstack.org/#/c/374667/ >> >>> >> >>> >> >>> >> >>> On Tue, Oct 4, 2016 at 8:53 PM, Yaroslav Lobankov >> >>> <[email protected]> >> >>> wrote: >> >>> >> >>> Hi Ken, >> >>> >> >>> >> >>> >> >>> OS-Faults doesn't have any scenarios in the tree yet (the project is >> >>> two >> >>> months old), but you can find some examples of the use in the >> >>> os-faults/examples directory. >> >>> >> >>> >> >>> >> >>> Regards, >> >>> >> >>> Yaroslav Lobankov. >> >>> >> >>> >> >>> >> >>> On Tue, Oct 4, 2016 at 8:02 PM, Ken'ichi Ohmichi >> >>> <[email protected]> >> >>> wrote: >> >>> >> >>> Hi Timur, >> >>> >> >>> Thanks for your explanation. >> >>> >> >>> >> >>> 2016-09-29 6:22 GMT-07:00 Timur Nurlygayanov >> >>> <[email protected]>: >> >>> > >> >>> >> I am guessing the above "restart nodes" is for verifying each >> >>> >> OpenStack service restarts successfully, right? >> >>> > >> >>> > Yes, this is right. And we also will check that HA logic for these >> >>> > services works correctly (for example, rescheduling of L3 Neutron >> >>> > agents for networks). >> >>> > >> >>> >> But these service scripts are provided by distributors, and >> >>> >> Devstack >> >>> >> itself doesn't contain service scripts IIUC. >> >>> >> So I'd like to know how to verify it on Devstack clouds. >> >>> > >> >>> > Yes, DevStack doesn't support many scenarios which are actual >> >>> > and should be supported on the production clouds. >> >>> > It will be not possible to run all advanced test scenarios for >> >>> > DevStack >> >>> > clouds, >> >>> > just because DevStack can't deploy OpenStack cloud with 3 >> >>> > controllers >> >>> > now (so, probably it will be possible in the future). >> >>> > >> >>> > Of course, some advanced scenarios will support DevStack clouds, >> >>> > for example, some test scenarios which are based on customer-found >> >>> > issues from the real production clouds, like upload of the large >> >>> > images >> >>> > (100+ Gb) >> >>> > to Glance with Swift backend. Such cases are important for >> >>> > verification >> >>> > of >> >>> > pre-production environments, but not very important for CI gate >> >>> > jobs. >> >>> > >> >>> > It is also important to note that in these advanced cases we are >> >>> > targeting >> >>> > to check not only the logic of Python code, but also the correct >> >>> > configuration >> >>> > of all OpenStack components on some pre-production OpenStack >> >>> > clusters. >> >>> >> >>> I guessed some part of os-faults can be moved to Tempest if os-faults >> >>> contains API tests for enabling/disabling OpenStack services. >> >>> Then, os-faults would be able to concentrate on more destructive tests >> >>> like rebooting physical nodes, etc. >> >>> However, I could not find any actual scenarios on current os-faults >> >>> (https://github.com/openstack/os-faults). >> >>> That seems to just contain some abstraction layers and unit tests. Can >> >>> we see actual test scenarios of os-faults ? >> >>> Maybe I missed something. >> >>> >> >>> Thanks >> >>> Ken Ohmichi >> >>> >> >>> >> >>> >> >>> __________________________________________________________________________ >> >>> OpenStack Development Mailing List (not for usage questions) >> >>> Unsubscribe: >> >>> [email protected]?subject:unsubscribe >> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> __________________________________________________________________________ >> >>> OpenStack Development Mailing List (not for usage questions) >> >>> Unsubscribe: >> >>> [email protected]?subject:unsubscribe >> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> -- >> >>> >> >>> >> >>> >> >>> Timur, >> >>> >> >>> Senior QA Manager >> >>> >> >>> OpenStack Projects >> >>> >> >>> Mirantis Inc >> >>> >> >>> >> >>> >> >>> __________________________________________________________________________ >> >>> OpenStack Development Mailing List (not for usage questions) >> >>> Unsubscribe: >> >>> [email protected]?subject:unsubscribe >> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >>> >> >> >> >> >> >> >> >> -- >> >> >> >> Timur, >> >> QA Manager >> >> OpenStack Projects >> >> Mirantis Inc >> >> >> >> >> >> __________________________________________________________________________ >> >> OpenStack Development Mailing List (not for usage questions) >> >> Unsubscribe: >> >> [email protected]?subject:unsubscribe >> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> > >> > >> > __________________________________________________________________________ >> > OpenStack Development Mailing List (not for usage questions) >> > Unsubscribe: >> > [email protected]?subject:unsubscribe >> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> >> -- >> Davanum Srinivas :: https://twitter.com/dims >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: [email protected]?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: [email protected]?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
