Re: [openstack-dev] [qa][tempest] Where to do response body validation
I disagree to moving this logic to tempest/services/*. The idea of these modules - assemble requests and return responses. Testing and verification should be wrapped over it. Either base class or tests, it depends on situation... -- Kind Regards Valeriy Ponomaryov On Thu, Mar 13, 2014 at 6:55 AM, Christopher Yeoh cbky...@gmail.com wrote: Hi, The new tempest body response validation is being added to individual testcases. See this as an example: https://review.openstack.org/#/c/78149 After having a look at https://review.openstack.org/#/c/80174/ I'm now thinking that perhaps we should be doing the response validation in the tempest/services/compute classes. And only check the response body if the status code is a success code (and then check that it is an appropriate success code). I think this will lead to fewer changes needed in the end as the response body checking will not needed to be added to individual tests. There may be some complications with handling extensions, but I think they are all implement backwards compatible behaviour so should be ok. Anyone have any thoughts about this alternative approach? Regards, Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa][tempest] Where to do response body validation
Hi Chris, Thank you for picking it up, -Original Message- From: Christopher Yeoh [mailto:cbky...@gmail.com] Sent: Thursday, March 13, 2014 1:56 PM To: OpenStack Development Mailing List Subject: [openstack-dev] [qa][tempest] Where to do response body validation The new tempest body response validation is being added to individual testcases. See this as an example: https://review.openstack.org/#/c/78149 After having a look at https://review.openstack.org/#/c/80174/ I'm now thinking that perhaps we should be doing the response validation in the tempest/services/compute classes. And only check the response body if the status code is a success code (and then check that it is an appropriate success code). I think this will lead to fewer changes needed in the end as the response body checking will not needed to be added to individual tests. There may be some complications with handling extensions, but I think they are all implement backwards compatible behaviour so should be ok. Anyone have any thoughts about this alternative approach? I like the above idea that the response body validation will be operated in REST client. Tempest will be able to check response body anytime and reduce the test code. One concern is that Nova API returns different response body when admin user or not. I'd like to check response body containing attributes what admin user can get. For example, get server info API returns a response including OS-EXT-STS and OS-EXT-SRV-ATT attributes when an admin user. So how about operating the basic validation(without special attributes) only in each REST client? The the special validation(such as the above admin info) would be operated in each test. The schema size for the special cases could be reduced. Thanks Ken'ichi Ohmichi ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
