On 10/23/2014 06:27 AM, Chris Dent wrote:
I've proposed a spec to Ceilometer
https://review.openstack.org/#/c/129669/
for a suite of declarative HTTP tests that would be runnable both in
gate check jobs and in local dev environments.
There's been some discussion that this may be generally applicable
and could be best served by a generic tool. My original assertion
was "let's make something work and then see if people like it" but I
thought I also better check with the larger world:
* Is this a good idea?
I think so
* Do other projects have similar ideas in progress?
Tempest faced a similar problem around negative tests in particular. We
have code in tempest that automatically generates a series of negative
test cases based on illegal variations of a schema. If you want to look
at it the NegativeAutoTest class is probably a good place to start. We have
discussed using a similar methodology for positive test cases but never
did anything with that.
Currently only a few of the previous negative tests have been replaced
with auto-gen tests. In addition to the issue of how to represent the
schema, the other major issue we encountered was the need to create
resources used by the auto-generated tests and a way to integrate a
resource description into the schema. We use json for the schema and
hoped one day to be able to receive base schemas from the projects
themselves.
* Is this concept something for which a generic tool should be
created _prior_ to implementation in an individual project?
* Is there prior art? What's a good format?
Marc Koderer and I did a lot of searching and asking folks if there was
some python code that we could use as a starting point but in the end
did not find anything. I do not have a list of what we considered and
rejected.
-David
Thanks.
_______________________________________________
OpenStack-dev mailing list
[email protected]
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
