Re: [openstack-dev] [all] [api] gabbi: A tool for declarative testing of APIs

On Apr 20, 2015, at 2:45 PM, Chris Dent wrote: > I wanted to make a quick update on the latest happenings with > gabbi[0], the tool I've created to do "declarative" testing of > OpenStack APIs (starting with Ceilometer and Gnocchi). > > * Jay Pipes and I are doing a presentation "API Matters" at

Re: [openstack-dev] [all] [api] gabbi: A tool for declarative testing of APIs

I wanted to make a quick update on the latest happenings with gabbi[0], the tool I've created to do "declarative" testing of OpenStack APIs (starting with Ceilometer and Gnocchi). * Jay Pipes and I are doing a presentation "API Matters" at summit. The latter of half of that will be me noodling

Re: [openstack-dev] [all] [api] gabbi: A tool for declarative testing of APIs

On Tue, 3 Mar 2015, Steve Baker wrote: This looks very useful, I'd like to use this in the heat functional tests job. Awesome, let me know if you need any assistance getting started. Is it possible to write tests which do a POST/PUT then a loop of GETs until some condition is met (a response

Re: [openstack-dev] [all] [api] gabbi: A tool for declarative testing of APIs

On 03/03/15 00:56, Chris Dent wrote: I (and a few others) have been using gabbi[1] for a couple of months now and it has proven very useful and evolved a bit so I thought it would be worthwhile to followup my original message and give an update. Some recent reviews[1] give a sample of how it ca

Re: [openstack-dev] [all] [api] gabbi: A tool for declarative testing of APIs

On 03/02/2015 03:56 AM, Chris Dent wrote: I (and a few others) have been using gabbi[1] for a couple of months now and it has proven very useful and evolved a bit so I thought it would be worthwhile to followup my original message and give an update. Some recent reviews[1] give a sample of how

Re: [openstack-dev] [all] [api] gabbi: A tool for declarative testing of APIs

I (and a few others) have been using gabbi[1] for a couple of months now and it has proven very useful and evolved a bit so I thought it would be worthwhile to followup my original message and give an update. Some recent reviews[1] give a sample of how it can be used to validate an existing API

Re: [openstack-dev] [all] [api] gabbi: A tool for declarative testing of APIs

On 01/12/2015 02:20 PM, Chris Dent wrote: After some discussion with Sean Dague and a few others it became clear that it would be a good idea to introduce a new tool I've been working on to the list to get a sense of its usefulness generally, work towards getting it into global requirements, and

Re: [openstack-dev] [all] [api] gabbi: A tool for declarative testing of APIs

On Mon, 12 Jan 2015, Sean Dague wrote: I think it's important to look at this in the narrower context, we're not testing full environments here, this is custom crafting HTTP req / resp in a limited context to make sure components are completing a contract. Yes, exactly. In fact one of the thi

Re: [openstack-dev] [all] [api] gabbi: A tool for declarative testing of APIs

On Tue, 13 Jan 2015, Boris Pavlovic wrote: Having separated engine seems like a good idea. It will really simplify stuff I'm not certain that's the case, but it may be worth exploration. This seems like a huge duplication of efforts. I mean operators will write own tools developers own... Wh

Re: [openstack-dev] [all] [api] gabbi: A tool for declarative testing of APIs

Sean, So I'd say let's focus on that problem right now, and get some traction > on this as part of functional test suites in OpenStack. Genericizing it > too much just turns this back into a version of every other full stack > testing tool, which we know isn't sufficient for having quality > comp

Re: [openstack-dev] [all] [api] gabbi: A tool for declarative testing of APIs

On Mon, 12 Jan 2015, Anne Gentle wrote: I'm interested, sure. What did you use to write the HTTP tests, as in, what was the source of truth for what the requests and responses should be? That is an _extremely_ good question and one I really struggled with as I started integrating gabbi with ce

Re: [openstack-dev] [all] [api] gabbi: A tool for declarative testing of APIs

On Mon, Jan 12, 2015 at 1:20 PM, Chris Dent wrote: > > After some discussion with Sean Dague and a few others it became > clear that it would be a good idea to introduce a new tool I've been > working on to the list to get a sense of its usefulness generally, > work towards getting it into global

Re: [openstack-dev] [all] [api] gabbi: A tool for declarative testing of APIs

On 01/12/2015 05:00 PM, Boris Pavlovic wrote: > Hi Chris, > > If there's sufficient motivation and time it might make sense to > separate the part of gabbi that builds TestCases from the part that > runs (and evaluates) HTTP requests and responses. If that happens then > integrati

Re: [openstack-dev] [all] [api] gabbi: A tool for declarative testing of APIs

Hi Chris, If there's sufficient motivation and time it might make sense to > separate the part of gabbi that builds TestCases from the part that > runs (and evaluates) HTTP requests and responses. If that happens then > integration with tools like Rally and runners is probably possible. Having

Re: [openstack-dev] [all] [api] gabbi: A tool for declarative testing of APIs

On Tue, 13 Jan 2015, Boris Pavlovic wrote: The Idea is brilliant. I may steal it! =) Feel free. But there are some issues that will be faced: 1) Using as a base unittest: python -m subunit.run discover -f gabbi | subunit2pyunit So rally team won't be able to reuse it for load testing (i

Re: [openstack-dev] [all] [api] gabbi: A tool for declarative testing of APIs

On Mon, 12 Jan 2015, Gregory Haynes wrote: Awesome! I was discussing trying to add extensions to RAML[1] so we could do something like this the other day. Is there any reason you didnt use an existing modeling language like this? Glad you like it. I chose to go with my own model in the YAML f

Re: [openstack-dev] [all] [api] gabbi: A tool for declarative testing of APIs

Excerpts from Chris Dent's message of 2015-01-12 19:20:18 +: > > After some discussion with Sean Dague and a few others it became > clear that it would be a good idea to introduce a new tool I've been > working on to the list to get a sense of its usefulness generally, > work towards getting i

Re: [openstack-dev] [all] [api] gabbi: A tool for declarative testing of APIs

Sean, I definitely like the direction that gabbi seems to be headed. It feels > like a much cleaner version of what nova tried to do with API samples. > As long as multiple projects think this is an interesting direction, I > think it's probably fine to add it to global-requirements and let them

Re: [openstack-dev] [all] [api] gabbi: A tool for declarative testing of APIs

On 01/12/2015 03:18 PM, Boris Pavlovic wrote: > Chris, > > The Idea is brilliant. I may steal it! =) > > But there are some issues that will be faced: > > 1) Using as a base unittest: > > python -m subunit.run discover -f gabbi | subunit2pyunit > > > So rally team won't be able to reus

Re: [openstack-dev] [all] [api] gabbi: A tool for declarative testing of APIs

On 01/12/2015 03:11 PM, Dean Troyer wrote: > Thanks for this Chris, I'm hoping to get my fingers dirty with it Real > Soon Now. > > On Mon, Jan 12, 2015 at 1:54 PM, Eoghan Glynn > wrote: > > I'd be interested in hearing the api-wg viewpoint, specifically whether >

Re: [openstack-dev] [all] [api] gabbi: A tool for declarative testing of APIs

Chris, The Idea is brilliant. I may steal it! =) But there are some issues that will be faced: 1) Using as a base unittest: > python -m subunit.run discover -f gabbi | subunit2pyunit So rally team won't be able to reuse it for load testing (if we directly integrate it) because we will have hu

Re: [openstack-dev] [all] [api] gabbi: A tool for declarative testing of APIs

Thanks for this Chris, I'm hoping to get my fingers dirty with it Real Soon Now. On Mon, Jan 12, 2015 at 1:54 PM, Eoghan Glynn wrote: > > I'd be interested in hearing the api-wg viewpoint, specifically whether > that working group intends to recommend any best practices around the > approach to A

Re: [openstack-dev] [all] [api] gabbi: A tool for declarative testing of APIs

> After some discussion with Sean Dague and a few others it became > clear that it would be a good idea to introduce a new tool I've been > working on to the list to get a sense of its usefulness generally, > work towards getting it into global requirements, and get the > documentation fleshed ou

[openstack-dev] [all] [api] gabbi: A tool for declarative testing of APIs

After some discussion with Sean Dague and a few others it became clear that it would be a good idea to introduce a new tool I've been working on to the list to get a sense of its usefulness generally, work towards getting it into global requirements, and get the documentation fleshed out so that