2016-03-16 19:41 GMT-07:00 Jim Rollenhagen <j...@jimrollenhagen.com>: > On Wed, Mar 16, 2016 at 06:20:11PM -0700, Ken'ichi Ohmichi wrote: >> Hi >> >> I have one proposal[1] related to negative tests in Tempest, and >> hoping opinions before doing that. >> >> Now Tempest contains negative tests and sometimes patches are being >> posted for adding more negative tests, but I'd like to propose >> removing them from Tempest instead. >> >> Negative tests verify surfaces of REST APIs for each component without >> any integrations between components. That doesn't seem integration >> tests which are scope of Tempest. >> In addition, we need to spend the test operating time on different >> component's gate if adding negative tests into Tempest. For example, >> we are operating negative tests of Keystone and more >> components on the gate of Nova. That is meaningless, so we need to >> avoid more negative tests into Tempest now. >> >> If wanting to add negative tests, it is a nice option to implement >> these tests on each component repo with Tempest plugin interface. We >> can avoid operating negative tests on different component gates and >> each component team can decide what negative tests are valuable on the >> gate. >> >> In long term, all negative tests will be migrated into each component >> repo with Tempest plugin interface. We will be able to operate >> valuable negative tests only on each gate. > > So, positive tests in tempest, negative tests as a plugin. > > Is there any longer term goal to have all tests for all projects in a > plugin for that project? Seems odd to separate them.
Yeah, from implementation viewpoint, that seems a little odd. but from the main scope of Tempest and to avoid unnecessary gate operation time, that can be acceptable, I feel. Negative tests can be corner cases in most cases, they don't seem integration tests. Thanks Ken Ohmichi __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
