On Mon, 12 Jan 2015, Anne Gentle wrote:
I'm interested, sure. What did you use to write the HTTP tests, as in, what was the source of truth for what the requests and responses should be?
That is an _extremely_ good question and one I really struggled with as I started integrating gabbi with ceilometer. Initially I thought "I'll just use the API docs as the source of truth" but I found they were a bit incomplete on some of the nuances, so I asked around for other sources of truth, but got little in the way of response. So then I tried to use the api controller code, but not to put too fine a point on it, but the combination of WSME and Pecan makes for utterly inscrutable code if you're interested in the actual structure of the HTTP requests and response and the URIs being used. So then I tried to use the existing api unit tests and was able to extract a bit there, but it wasn't smooth sailing. So finally what I did was decide that I would do the work in phases and with collaborators: I'd get the initial framework in place and then impose upon those more familiar with the API than I to do subsequent dependent patchsets that cover the API more completely. I have to admit that the concept of API truth is part of the reason I wanted to create this kind of testing. None of the resources I could find in the ceilometer code tree gave any clear overview that mapped URIs to methods, allowing easy discovery of how the code works. I wanted to find some kind of map. Gabbi itself doesn't solve this problem (there's no map between URI and python method) but it can at least show the API, there in the code. It's a step in the right direction. I know that there are discussions in progress about formalizing APIs with tools like RAML (for example the thread Ian just extended). I think these have their place, especially for declaring truth, but they aren't necessarily good learning aids for new developers or good assistants for enabling and maintaining transparency.  I started at: http://docs.openstack.org/developer/ceilometer/webapi/v2.html but I think I should have used: http://developer.openstack.org/api-ref-telemetry-v2.html  https://github.com/tiddlyweb/tiddlyweb/blob/master/tiddlyweb/urls.map  http://lists.openstack.org/pipermail/openstack-dev/2015-January/054153.html -- Chris Dent tw:@anticdent freenode:cdent https://tank.peermore.com/tanks/cdent __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev