On 03/05/2014 10:25 PM, Christopher Yeoh wrote:
On Tue, 04 Mar 2014 13:31:07 -0500
David Kranz <[email protected]> wrote:
I think it would be a good time to have at least an initial
discussion about the requirements for theses schemas and where they
will live. The next step in tempest around this is to replace the
existing negative test files with auto-gen versions, and most of the
work in doing that is to define the schemas.
The tempest framework needs to know the http method, url part,
expected error codes, and payload description. I believe only the
last is covered by the current nova schema definitions, with the
others being some kind of attribute or data associated with the
method that is doing the validation. Ideally the information being
used to do the validation could be auto-converted to a more general
schema that could be used by tempest. I'm interested in what folks
have to say about this and especially from the folks who are core
members of both nova and tempest. See below for one example (note
that the tempest generator does not yet handle "pattern").
So as you've seen a lot of what is wanted for the tempest framework is
implicitly known already within the method context which is why its not
again explicitly stated in the schema. Not actually having thought
about it a lot, but I suspect the expected errors decorator is
something that would fit just as well in the validation framework
however.
Some of the other stuff such as url part, descriptions etc, not so much
as it would be purely duplicate information that would get out of date.
However for documentation auto generation it is something that we do
also want to have available in an automated fashion. I did a bit of
exploration early in Icehouse in generating this within the context of
api samples tests where we have access to this sort of stuff and I think
together we'd have all the info we need, I'm just not sure mashing them
together is the right way to do it.
To be clear, I was advocating for some solution where all the
information needed to create negative tests, api docs, (part of client
libraries?), etc. could be derived from the source code. The pieces of
information could come from explicit schema definitions, scanning the
code, python introspection. I have done this sort of thing in other
languages but am not enough of a Python wiz to say exactly how this
could work. Certainly the schema should not have duplicate information.
And from the documentation point of view we need to have a bit of a
think about whether doc strings on methods should be the canonical way
we produce descriptional information about API methods. One hand its
appealing, on the other hand they tend to be not very useful or very
internals Nova focussed. But we could get much better at it.
Agreed. IMO, doc strings on methods have far more benefits (automation,
not getting out of sync with code) than real costs (non-coders have to
edit the code to review or improve the docs). This works best when
developers write at least drafts of doc strings. The REST API case is a
little more tricky because it does not map as directly to methods.
-David
Short version - yea I think we want to get to the point where tempest
doesn't generate these manually. But I'm not sure about how we
should do it.
Chris
-David
From nova:
get_console_output = {
'type': 'object',
'properties': {
'get_console_output': {
'type': 'object',
'properties': {
'length': {
'type': ['integer', 'string'],
'minimum': 0,
'pattern': '^[0-9]+$',
},
},
'additionalProperties': False,
},
},
'required': ['get_console_output'],
'additionalProperties': False,
}
From tempest:
{
"name": "get-console-output",
"http-method": "POST",
"url": "servers/%s/action",
"resources": [
{"name":"server", "expected_result": 404}
],
"json-schema": {
"type": "object",
"properties": {
"os-getConsoleOutput": {
"type": "object",
"properties": {
"length": {
"type": ["integer", "string"],
"minimum": 0
}
}
}
},
"additionalProperties": false
}
}
_______________________________________________
OpenStack-dev mailing list
[email protected]
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________
OpenStack-dev mailing list
[email protected]
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
