On 03/03/15 00:56, Chris Dent wrote:

I (and a few others) have been using gabbi[1] for a couple of months now
and it has proven very useful and evolved a bit so I thought it would be
worthwhile to followup my original message and give an update.

Some recent reviews[1] give a sample of how it can be used to validate
an existing API as well as search for less than perfect HTTP behavior
(e.g sending a 404 when a 405 would be correct).

Regular use has lead to some important changes:

* It can now be integrated with other tox targets so it can run
  alongside other functional tests.
* Individual tests can be xfailed and skipped. An entire YAML test
  file can be skipped.
* For those APIs which provide insufficient hypermedia support, the
  ability to inspect and reference the prior test and use template
  variables in the current request has been expanded (with support for
  environment variables pending a merge).

My original motivation for creating the tool was to make it easier to
learn APIs by causing a body of readable YAML files to exist. This
remains important but what I've found is that writing the tests is
itself an incredible tool. Not only is it very easy to write tests
(throw some stuff at a URL and see what happen) and find (many) bugs
as a result, the exploratory nature of test writing drives a
learning process.

You'll note that the reviews below are just the YAML files. That's
because the test loading and fixture python code is already merged.
Adding tests is just a matter of adding more YAML. An interesting
trick is to run a small segment of the gabbi tests in a project (e.g.
just one file that represents one type of resource) while producing
coverage data. Reviewing the coverage of just the controller for that
resource can help drive test creation and separation.

[1] http://gabbi.readthedocs.org/en/latest/
[2] https://review.openstack.org/#/c/159945/
    https://review.openstack.org/#/c/159204/

This looks very useful, I'd like to use this in the heat functional tests job.

Is it possible to write tests which do a POST/PUT then a loop of GETs until some condition is met (a response_json_paths match on IN_PROGRESS -> COMPLETE)

This would allow for testing of non-atomic PUT/POST operations for entities like nova servers, heat stacks etc.

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Reply via email to