On Wed, Mar 16, 2016 at 10:41 PM, Jim Rollenhagen <j...@jimrollenhagen.com> wrote: > On Wed, Mar 16, 2016 at 06:20:11PM -0700, Ken'ichi Ohmichi wrote: >> Hi >> >> I have one proposal[1] related to negative tests in Tempest, and >> hoping opinions before doing that. >> >> Now Tempest contains negative tests and sometimes patches are being >> posted for adding more negative tests, but I'd like to propose >> removing them from Tempest instead. >> >> Negative tests verify surfaces of REST APIs for each component without >> any integrations between components. That doesn't seem integration >> tests which are scope of Tempest. >> In addition, we need to spend the test operating time on different >> component's gate if adding negative tests into Tempest. For example, >> we are operating negative tests of Keystone and more >> components on the gate of Nova. That is meaningless, so we need to >> avoid more negative tests into Tempest now. >> >> If wanting to add negative tests, it is a nice option to implement >> these tests on each component repo with Tempest plugin interface. We >> can avoid operating negative tests on different component gates and >> each component team can decide what negative tests are valuable on the >> gate. >> >> In long term, all negative tests will be migrated into each component >> repo with Tempest plugin interface. We will be able to operate >> valuable negative tests only on each gate. > > So, positive tests in tempest, negative tests as a plugin. > > Is there any longer term goal to have all tests for all projects in a > plugin for that project? Seems odd to separate them.
I'd love to see this idea explored further. What happens if Tempest ends up without tests, as a library for shared code as well as a centralized place to run tests from via plugins? > > // jim > >> >> Any thoughts? >> >> Thanks >> Ken Ohmichi >> >> --- >> [1]: https://review.openstack.org/#/c/293197/ >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev