>>So I'd say the api is verified. What is missing, and could be useful, is
>>using those tests to get accurate and up to date representations of the JSON
>>request and response bodies. If that's something we'd like to pursue as I
>>said in my other message the 'verbose' functionality that can be provided in
>>gabbi->>based tests should be able to help.
I have checked with the 'verbose' functionality, but that doesn't verify the
entire response, it's simply prints header and response on console.
On the current master code, few of the gabbi tests verify the entire response
and the remaining verify’s only specific attributes from the response.
For example:
1. The test "update a resource provider" which is under
"nova/tests/functional/api/openstack/placement/gabbits/resource-provider.yaml"
does not check all the key-value pairs of "links" attribute, it only checks the
"links[?rel = "self"].href".
Response return after update
{
"links": [
{
"href": "/resource_providers/8e8b14f1-1360-4b61-8ff0-aeefc73a1808",
"rel": "self"
},
{
"href":
"/resource_providers/8e8b14f1-1360-4b61-8ff0-aeefc73a1808/inventories",
"rel": "inventories"
},
{
"href": "/resource_providers/8e8b14f1-1360-4b61-8ff0-aeefc73a1808/usages",
"rel": "usages"
}
],
"generation": 0,
"uuid": "8e8b14f1-1360-4b61-8ff0-aeefc73a1808",
"name": "new name"
}
2. The test "list one resource providers" under
nova/tests/functional/api/openstack/placement/gabbits/resource-provider.yaml
checks all the key-value pairs of response object.
Response return in case of list one resource providers
{
"resource_providers": [
{
"links": [
{
"href": "/resource_providers/958f0cb7-8fa2-4521-95df-e4cff1cc0d07",
"rel": "self"
},
{
"href":
"/resource_providers/958f0cb7-8fa2-4521-95df-e4cff1cc0d07/inventories",
"rel": "inventories"
},
{
"href":
"/resource_providers/958f0cb7-8fa2-4521-95df-e4cff1cc0d07/usages",
"rel": "usages"
}
],
"name": "0e6980e1-501a-468c-9d9a-bce6caecffb3",
"generation": 0,
"uuid": "958f0cb7-8fa2-4521-95df-e4cff1cc0d07"
}
]
}
So instead of verifying each and every key-value pair from the response object,
it would nice if gabbit can add a support to
accept response.json file and during execution of the test, it can verify
whether all key-value pairs response.json file are matching with the actual
response.
In the current master code, for a specific micro-version, if nova is returning
extra parameters, then the gabbit test won't fail because it is only validating
required attributes from the response.
I think, this issue can be handled once we do strict validation of actual
response with the response.json.
Please let me know your thoughts of adding support in gabbi project to accept
response.json file for each test.
Regards,
Bhagyashri Shewale
-----Original Message-----
From: Chris Dent [mailto:[email protected]]
Sent: Friday, June 23, 2017 5:35 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [nova][placement] Add API Sample tests for
Placement APIs
On Thu, 22 Jun 2017, Shewale, Bhagyashri wrote:
>>> * who or what needs to consume these JSON samples?
> The users of placement API can rely on the request/response for different
> supported placement versions based on some tests running on the OpenStack CI
> infrastructure.
> Right now, most of the placement APIs are well documented and others are in
> progress but there are no tests to verify these APIs.
Either we are misunderstanding each other, or you're not aware of what the
gabbi tests are doing. They verify the placement API and provide extensive
coverage of the entire placement HTTP framework, including the accuracy of
response codes in edge cases not on the "happy path". Coverage is well over 90%
for the group of files in nova/api/openstack/placement (except for the wsgi
deployment script
itself) when the
nova/tests/functional/api/openstack/placement/test_placement_api.py
functionl tests runs all the gabbi files in
nova/tests/functional/api/openstack/placement/gabbits/.
So I'd say the api is verified. What is missing, and could be useful, is using
those tests to get accurate and up to date representations of the JSON request
and response bodies. If that's something we'd like to pursue as I said in my
other message the 'verbose' functionality that can be provided in gabbi-based
tests should be able to help.
> We would like to write new functional test to consume these json samples to
> verify each placement API for all supported versions.
Those gabbi files also test functionality at micorversion boundaries.
--
Chris Dent ┬──┬◡ﾉ(° -°ﾉ) https://anticdent.org/
freenode: cdent tw: @anticdent
______________________________________________________________________
Disclaimer: This email and any attachments are sent in strictest confidence
for the sole use of the addressee and may contain legally privileged,
confidential, and proprietary data. If you are not the intended recipient,
please advise the sender by replying promptly to this email and then delete
and destroy this email and any attachments without any further use, copying
or forwarding.
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [email protected]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
