Hi, My first impression is "It's interesting" :)
I actually don't read whole of this threads, but, I re-read the QA mission statement[1]. I feel os-faults is a little bit far from the mission statement. Because os-faults seems like focusing on operator testing. But I think this can be under the QA project. Because tempest scope has also operator testing. [1] https://wiki.openstack.org/wiki/QA#Project_Team_Definition On Sat, Oct 8, 2016 at 4:43 AM, Ken'ichi Ohmichi <ken1ohmi...@gmail.com> wrote: > Hi Timur, > > 2016-10-06 4:08 GMT-07:00 Timur Nurlygayanov <tnurlygaya...@mirantis.com>: >> Ken, it is a good idea! >> >> The plan is to develop os-faults as a library which will be able >> to manage cluster nodes and OpenStack services on these nodes. >> It is a good idea to add some Tempest tests which will use >> os-faults library as well for some API tests. > > Sorry for my misleading, that is not I wanted to say. > I am thinking Tempest doesn't use os-faults as library. > I just wanted to say some destructive tests which will use REST APIs > can be implemented in Tempest instead of os-faults. > > For example, the compute service client contains disable_service which > disables nova service but Tempest doesn't contain the corresponding > test cases because Tempest tests need to run in parallel. > https://github.com/openstack/tempest/blob/master/tempest/lib/services/compute/services_client.py#L54 > > If some tests use this method, the other tests which run in parallel > can be failed as unexpected on the gate. > Such tests also are destructive and I am hoping these tests also will > be able to run the gate by finding better way. > >> The Stepler framework [1] will use os-faults to perform all destructive >> actions in the clouds (the reboot of nodes, restart of OpenStack services, >> enable/disable network interfaces or some firewall rules and etc). >> >> We need to get +1 from you in [1] to create the repository with >> advanced end-user scenario tests. > > OK, I'd like to summarize my thinking here for adding os-faults under > QA project: > > Under QA project, the first target of most programs(tempest, devstack, > etc) is for the gate testing. > Of course, some of them are available on production clouds also as the > design, but the first is for the gate. > That means devstack clouds as the first target/purpose. > At this time, this os-faults doesn't seem the gate as the first > purpose based on current implementation. > So I feel os-faults seems different from the existing programs. > > os-faults is very young and we will be able to extend it for devstack > clouds also later, I hope. > In addition, I heard this kind of tests(destructive, HA, etc) are > requested from the other users. > So this os-faults seems very valuable for users. > > Just in my opinion, I am find to add this os-faults under QA project. > But before that, I'd like to get opinions from the other people of QA team. > > Thanks > Ken Ohmichi > > > > > > > > > > > > >> On Tue, Oct 4, 2016 at 8:53 PM, Yaroslav Lobankov <yloban...@mirantis.com> >> wrote: >>> >>> Hi Ken, >>> >>> OS-Faults doesn't have any scenarios in the tree yet (the project is two >>> months old), but you can find some examples of the use in the >>> os-faults/examples directory. >>> >>> Regards, >>> Yaroslav Lobankov. >>> >>> On Tue, Oct 4, 2016 at 8:02 PM, Ken'ichi Ohmichi <ken1ohmi...@gmail.com> >>> wrote: >>>> >>>> Hi Timur, >>>> >>>> Thanks for your explanation. >>>> >>>> 2016-09-29 6:22 GMT-07:00 Timur Nurlygayanov >>>> <tnurlygaya...@mirantis.com>: >>>> > >>>> >> I am guessing the above "restart nodes" is for verifying each >>>> >> OpenStack service restarts successfully, right? >>>> > >>>> > Yes, this is right. And we also will check that HA logic for these >>>> > services works correctly (for example, rescheduling of L3 Neutron >>>> > agents for networks). >>>> > >>>> >> But these service scripts are provided by distributors, and Devstack >>>> >> itself doesn't contain service scripts IIUC. >>>> >> So I'd like to know how to verify it on Devstack clouds. >>>> > >>>> > Yes, DevStack doesn't support many scenarios which are actual >>>> > and should be supported on the production clouds. >>>> > It will be not possible to run all advanced test scenarios for DevStack >>>> > clouds, >>>> > just because DevStack can't deploy OpenStack cloud with 3 controllers >>>> > now (so, probably it will be possible in the future). >>>> > >>>> > Of course, some advanced scenarios will support DevStack clouds, >>>> > for example, some test scenarios which are based on customer-found >>>> > issues from the real production clouds, like upload of the large images >>>> > (100+ Gb) >>>> > to Glance with Swift backend. Such cases are important for verification >>>> > of >>>> > pre-production environments, but not very important for CI gate jobs. >>>> > >>>> > It is also important to note that in these advanced cases we are >>>> > targeting >>>> > to check not only the logic of Python code, but also the correct >>>> > configuration >>>> > of all OpenStack components on some pre-production OpenStack clusters. >>>> >>>> I guessed some part of os-faults can be moved to Tempest if os-faults >>>> contains API tests for enabling/disabling OpenStack services. >>>> Then, os-faults would be able to concentrate on more destructive tests >>>> like rebooting physical nodes, etc. >>>> However, I could not find any actual scenarios on current os-faults >>>> (https://github.com/openstack/os-faults). >>>> That seems to just contain some abstraction layers and unit tests. Can >>>> we see actual test scenarios of os-faults ? >>>> Maybe I missed something. >>>> >>>> Thanks >>>> Ken Ohmichi >>>> >>>> >>>> __________________________________________________________________________ >>>> OpenStack Development Mailing List (not for usage questions) >>>> Unsubscribe: >>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> >>> >>> __________________________________________________________________________ >>> OpenStack Development Mailing List (not for usage questions) >>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >> >> >> >> -- >> >> Timur, >> Senior QA Manager >> OpenStack Projects >> Mirantis Inc >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev