On Wed, 21 Jun 2017, Shewale, Bhagyashri wrote:
I would like to write functional tests to check the exact req/resp for each placement API for all supported versions similar to what is already done for other APIs under nova/tests/functional/api_sample_tests/api_samples/*. These request/response json samples can be used by the api.openstack.org and in the manuals.There are already functional tests written for placement APIs under nova/tests/functional/api/openstack/placement, but these tests doesn't check the entire HTTP response for each API for all supported versions. I think adding such functional tests for checking response for each placement API would be beneficial to the project. If there is an interest to create such functional tests, I can file a new blueprint for this activity.
At Matt points out elsewhere, we made a choice to not use the api_samples format when developing placement. There were a few different reasons for this: * we wanted to limit the amount of nova code used in placement, to ease the eventual extraction of placement to its own code repository * some of us wanted to use gabbi [1] as the testing framework as it is nicely declarative [2] and keeps the request and response in the same place * we were building the api framework from scratch and doing what amounts to test driven development [2] using functional tests and gabbi works well for that * testing the full response isn't actually a great way to test an API in a granular way; the info is useful to have but it isn't a useful test (from a development standpoint) But, as you've noted, this means there isn't a single place to go to see a collection of a full request and response bodies. That information can be extracted from the gabbi tests, but it's a) not immediately obvious, b) requires interpretation. Quite some time ago I started a gabbi-based full request and response suite of tests [3] but it was never finished and now is very out of date. If the end goal is to have a set of documents that pair all the possible requests (with bodies) with all possible responses (with bodies), gabbi could easily create this in its "verbose" mode [4] when run as functional tests or with the gabbi-run [5] command that can run against a running service. So I would suggest that we more completely explain the goal or goals that you're trying to satisfy and then see how we can use the existing tooling to fill them. Some questions related to that: * who or what needs to consume these JSON samples? * do they need to be tests against current code, or are they primarily reference info? * what are the value propositions associated with fully validating the structure and content of the response bodies? We can relatively easily figure out some way to drive gabbi to produce the desired information, but first we want to make sure that the information produced is going to be the right info (that is, will satisfy the needs of whoever wants it). I am, as Matt mentioned, on holiday at the moment so my response to any future messages may be delayed, but I'll catch up as I'm able. [1] https://gabbi.readthedocs.io/en/latest/ [2] https://github.com/openstack/nova/tree/master/nova/tests/functional/api/openstack/placement/gabbits [3] https://review.openstack.org/#/c/370204/ [4] verbose mode can print out request and response headers, bodies, or both. If the bodies are JSON, it will be pretty printed. [5] https://gabbi.readthedocs.io/en/latest/runner.html -- Chris Dent ┬──┬◡ﾉ(° -°ﾉ) https://anticdent.org/ freenode: cdent tw: @anticdent
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
