<snip> > > Last, is a question, is it possible to currently run the full API an build a > > json schema for it all? Or to query these validating schemas? We *really* > > want that over in tempest, so we can completely drop the manual creation of > > negative testing, and just fuzz known bad against the schema definitions. > > Sorry, I'm not sure I understand this question correctly. > We need to define schemas for each API with the separated schema patches. > It is impossible to define one schema for all APIs. > > > Hi David, Marc, > > I guess the negative test generator of Tempest would need each API definition. > Glance can provide API definitions through API with jsonschema format, but > Nova does not have such feature. > We need to port these API schema from Nova to Tempest, I guess. right? >
As I understand things after reviewing the first versions of the negative test generator patch is that we have to hard code the schema into a file (right now it's in the test file, but eventually it'll be an external input file). One of my issues with doing that is it's highly manual process, essentially a copy and paste from the nova tree. I think what we're looking for from this jsonschema validation work is an API which we can query the API and get the jsonschema definitions; similar to what the glance API offers. I wouldn't advocate making the negative test generator be fully dynamic for the same reason we don't autodetect which API versions and extensions/features are enabled in tempest, but rather rely on the config file. But, instead have an additional tool which could query the schema for all the endpoints in nova and generate an input file for the negative test generator. That way we'll still catch breaking API changes in the gate, but it's not a manual process to update the input file with the schema definitions in tempest when there is a breaking API change. (Which hopefully should almost never happen) -Matt Treinish _______________________________________________ OpenStack-dev mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
