On 10/30/2014 10:47 AM, Eoghan Glynn wrote: > >> Matthew wrote: >> >> This would have the advantage of making tempest slimmer for every project >> and begin the process of getting projects to take responsibility for their >> functional testing rather than relying on tempest. > > [much snipping] > >> Sean wrote: >> >> Ok, so part of this remains to be seen about what the biggest bang for the >> buck is. The class of bugs I feel like we need to nail in Nova right now are >> going to require tests that bring up pieces of the wsgi stack, but are >> probably not runable on a real deploy. Again, this is about debugability. > > So this notion of the biggest bang for our buck is an aspect of the drive > for in-tree functional tests, that's not entirely clear to me as yet. > > i.e. whether individual projects should be prioritizing within this effort: > > (a) the creation of net-new coverage for scenarios (especially known or > suspected bugs) that were not previously tested, in a non-unit sense > > (b) the relocation of existing integration test coverage from Tempest to > the project trees, in order to make the management of Tempest more > tractable > > It feels like there may be a tension between (a) and (b) in terms of the > pay-off for this effort. I'd interested in hearing other opinions on this, > on what aspect projects are expecting (and expected) to concentrate on > initially.
For what it's worth I have a bunch of early targets listed for Nova for our summit session - https://etherpad.openstack.org/p/kilo-nova-functional-testing My focus in kilo is going to be first about A), as that provides value out of the gate (pun intended). Then peel off some stuff from B as makes sense. -Sean -- Sean Dague http://dague.net _______________________________________________ OpenStack-dev mailing list OpenStackfirstname.lastname@example.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev