I (and a few others) have been using gabbi[1] for a couple of months now and it has proven very useful and evolved a bit so I thought it would be worthwhile to followup my original message and give an update.
Some recent reviews[1] give a sample of how it can be used to validate an existing API as well as search for less than perfect HTTP behavior (e.g sending a 404 when a 405 would be correct). Regular use has lead to some important changes: * It can now be integrated with other tox targets so it can run alongside other functional tests. * Individual tests can be xfailed and skipped. An entire YAML test file can be skipped. * For those APIs which provide insufficient hypermedia support, the ability to inspect and reference the prior test and use template variables in the current request has been expanded (with support for environment variables pending a merge). My original motivation for creating the tool was to make it easier to learn APIs by causing a body of readable YAML files to exist. This remains important but what I've found is that writing the tests is itself an incredible tool. Not only is it very easy to write tests (throw some stuff at a URL and see what happen) and find (many) bugs as a result, the exploratory nature of test writing drives a learning process. You'll note that the reviews below are just the YAML files. That's because the test loading and fixture python code is already merged. Adding tests is just a matter of adding more YAML. An interesting trick is to run a small segment of the gabbi tests in a project (e.g. just one file that represents one type of resource) while producing coverage data. Reviewing the coverage of just the controller for that resource can help drive test creation and separation. [1] http://gabbi.readthedocs.org/en/latest/ [2] https://review.openstack.org/#/c/159945/ https://review.openstack.org/#/c/159204/ -- Chris Dent tw:@anticdent freenode:cdent https://tank.peermore.com/tanks/cdent __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
