>> * who or what needs to consume these JSON samples? The users of placement API can rely on the request/response for different supported placement versions based on some tests running on the OpenStack CI infrastructure. Right now, most of the placement APIs are well documented and others are in progress but there are no tests to verify these APIs. We would like to write new functional test to consume these json samples to verify each placement API for all supported versions.
>>do they need to be tests against current code, or are they primarily >>reference info? Yes, the request/response json files for each supported version should be run against the current master code. >>what are the value propositions associated with fully validating the >>structure and content of the response bodies? It's kind of validation and verification mechanism available on OpenStack CI which can be fully trusted by the users of placement API and some of the other value propositions are mentioned by Rochelle Grober in his reply. Thanks for your response. Regards, Bhagyashri Shewale -----Original Message----- From: Chris Dent [mailto:[email protected]] Sent: Thursday, June 22, 2017 3:00 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [nova][placement] Add API Sample tests for Placement APIs On Wed, 21 Jun 2017, Shewale, Bhagyashri wrote: > I would like to write functional tests to check the exact req/resp > for each placement API for all supported versions similar to what is already > done for other APIs under > nova/tests/functional/api_sample_tests/api_samples/*. > These request/response json samples can be used by the api.openstack.org and > in the manuals. > > There are already functional tests written for placement APIs under > nova/tests/functional/api/openstack/placement, > but these tests doesn't check the entire HTTP response for each API for all > supported versions. > > I think adding such functional tests for checking response for each placement > API would be beneficial to the project. > If there is an interest to create such functional tests, I can file a new > blueprint for this activity. At Matt points out elsewhere, we made a choice to not use the api_samples format when developing placement. There were a few different reasons for this: * we wanted to limit the amount of nova code used in placement, to ease the eventual extraction of placement to its own code repository * some of us wanted to use gabbi [1] as the testing framework as it is nicely declarative [2] and keeps the request and response in the same place * we were building the api framework from scratch and doing what amounts to test driven development [2] using functional tests and gabbi works well for that * testing the full response isn't actually a great way to test an API in a granular way; the info is useful to have but it isn't a useful test (from a development standpoint) But, as you've noted, this means there isn't a single place to go to see a collection of a full request and response bodies. That information can be extracted from the gabbi tests, but it's a) not immediately obvious, b) requires interpretation. Quite some time ago I started a gabbi-based full request and response suite of tests [3] but it was never finished and now is very out of date. If the end goal is to have a set of documents that pair all the possible requests (with bodies) with all possible responses (with bodies), gabbi could easily create this in its "verbose" mode [4] when run as functional tests or with the gabbi-run [5] command that can run against a running service. So I would suggest that we more completely explain the goal or goals that you're trying to satisfy and then see how we can use the existing tooling to fill them. Some questions related to that: * who or what needs to consume these JSON samples? * do they need to be tests against current code, or are they primarily reference info? * what are the value propositions associated with fully validating the structure and content of the response bodies? We can relatively easily figure out some way to drive gabbi to produce the desired information, but first we want to make sure that the information produced is going to be the right info (that is, will satisfy the needs of whoever wants it). I am, as Matt mentioned, on holiday at the moment so my response to any future messages may be delayed, but I'll catch up as I'm able. [1] https://gabbi.readthedocs.io/en/latest/ [2] https://github.com/openstack/nova/tree/master/nova/tests/functional/api/openstack/placement/gabbits [3] https://review.openstack.org/#/c/370204/ [4] verbose mode can print out request and response headers, bodies, or both. If the bodies are JSON, it will be pretty printed. [5] https://gabbi.readthedocs.io/en/latest/runner.html -- Chris Dent ┬──┬◡ﾉ(° -°ﾉ) https://anticdent.org/ freenode: cdent tw: @anticdent ______________________________________________________________________ Disclaimer: This email and any attachments are sent in strictest confidence for the sole use of the addressee and may contain legally privileged, confidential, and proprietary data. If you are not the intended recipient, please advise the sender by replying promptly to this email and then delete and destroy this email and any attachments without any further use, copying or forwarding. __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
