On Wed, Mar 16, 2016 at 10:29:48PM -0400, Adam Young wrote: > On 03/16/2016 09:20 PM, Ken'ichi Ohmichi wrote: > >Hi > > > >I have one proposal[1] related to negative tests in Tempest, and > >hoping opinions before doing that. > > > >Now Tempest contains negative tests and sometimes patches are being > >posted for adding more negative tests, but I'd like to propose > >removing them from Tempest instead. > > > >Negative tests verify surfaces of REST APIs for each component without > >any integrations between components. That doesn't seem integration > >tests which are scope of Tempest. > >In addition, we need to spend the test operating time on different > >component's gate if adding negative tests into Tempest. For example, > >we are operating negative tests of Keystone and more > >components on the gate of Nova. That is meaningless, so we need to > >avoid more negative tests into Tempest now. > > > >If wanting to add negative tests, it is a nice option to implement > >these tests on each component repo with Tempest plugin interface. We > >can avoid operating negative tests on different component gates and > >each component team can decide what negative tests are valuable on the > >gate. > > Hear hear! For Keystone, please put them in the Keystone Client Functional > tests.
I'd recommend putting the negative tests in a plugin in the keystone tree itself. Otherwise, how do you ensure that a change to keystone doesn't suddenly make a bad API call go from 400 to 200, for instance? (this assumes that keystoneclient functional tests don't run against keystone changes, of course) // jim __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
