On Mon, Aug 4, 2014 at 3:29 AM, Chris Dent <[email protected]> wrote:
> > In the "Thoughts on the patch test failure rate and moving forward" > thread[1] there's discussion of moving some of the burden for > functional testing to the individual projects. This seems like a good > idea to me, but also seems like it could be a source of confusion so I > thought I'd start another thread to focus on the details of just this > topic, separate from the gate-oriented discussion in the other. > > In a couple of messages[2] Sean mentions "the vision". Is there a wiki > page or spec or other kind of document where this nascent vision is > starting to form? Even if we can't quite get started just yet, it > would be good to have an opporunity to think about the constraints and > goals that we'll be working with. > There is no single document on this that I know of. But two good places to start are the functional testing systems we have for swift and neutron: http://logs.openstack.org/70/116570/3/check/check-swift-dsvm-functional/4894055/console.html http://logs.openstack.org/70/116570/3/check/check-neutron-dsvm-functional/bf68c61/console.html http://git.openstack.org/cgit/openstack/neutron/tree/tox.ini#n29 > > Not just the goal of moving tests around, but what, for example, makes > a good functional test? > Here is a non-exhaustive list of some differences between what belongs in tempest and functional tests * functional tests should ideally not need external services (external services makes it closer to an integration test) * functional tests can be whitebox tests * a functional test environment should be easier to set up then devstack > > For constraints: Will tempest be available as a stable library? Is using > tempest (or other same library across all projects) a good or bad thing? > Seems there's some disagreement on both of these. > Yes, there is a separate thread on spinning out a tempest-lib (not sure on what final name will be yet) that functional tests can use. Although I think there is a lot to be done before needing the tempest-lib. > > Personally I'm quite eager to to vastly increase the amount of testing > I can do on my own machine(s) before letting the gate touch my code. > Why can't you run devstack locally? Maybe there are some changes we can make so its easier to run devstack locally first. > > [1] > http://lists.openstack.org/pipermail/openstack-dev/2014- > July/thread.html#41057 > [2] > http://lists.openstack.org/pipermail/openstack-dev/2014-July/041188.html > http://lists.openstack.org/pipermail/openstack-dev/2014-July/041252.html > > -- > Chris Dent tw:@anticdent freenode:cdent > https://tank.peermore.com/tanks/cdent > > _______________________________________________ > OpenStack-dev mailing list > [email protected] > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >
_______________________________________________ OpenStack-dev mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
