Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
2015-03-24 23:20 GMT+09:00 Joe Gordon joe.gord...@gmail.com: On Tue, Mar 24, 2015 at 7:10 AM, Jay Pipes jaypi...@gmail.com wrote: On Tue, Mar 24, 2015 at 01:04:46PM +, Jeremy Stanley wrote: On 2015-03-23 21:31:30 -0400 (-0400), Jay Pipes wrote: On Mon, Mar 23, 2015 at 09:35:50PM +, Jeremy Stanley wrote: On 2015-03-23 15:15:18 -0400 (-0400), Jay Pipes wrote: [...] I don't want it suppressed. I want the use of API extensions and the extension framework(s) to be completely dropped for all future API-affecting work. [...] Perhaps controversial, but would it be worthwhile to propose to the Defcore working group that future compliance requirements include the absence of extensions to officially covered APIs? I don't understand what you're getting at, Jeremy. Could you elaborate? What do extensions have to do with future compliance requirements? Defcore's focus is on establishing interoperability standards for OpenStack deployments, to ease the end-user experience. Right now its model depends on a whitelist of API features. As discussed many times before and brought up again in this thread, when providers or distributors augment OpenStack APIs to add their own special features without implementing them upstream, this necessarily creates interoperability issues. Defcore's focus is on determining what is OpenStack, w.r.t. what is brandable as OpenStack. It's focus is not on establishing interoperability standards. I am not sure how you got to that conclusion, yes the defcore process has been very confusing and I am still not really sure what it was, but some part of it it *is* about interoperability/ Although our wiki does get out of date very easily, I think this still holds true: DefCore sets base requirements by defining 1) capabilities, 2) code and 3) must-pass tests for all OpenStack products. This definition uses community resources and involvement to drive interoperability by creating the minimum standards for products labeled OpenStack. https://wiki.openstack.org/wiki/Governance/DefCoreCommittee Related to this topic, Nova v2.1 API defines core APIs by itself[1], but I feel now it is better to remove the definition from Nova. On current implementation, the boot of nova-api fails if the above core APIs are not loaded. but that behavior seems conflict to Defcore process, and it would be nice to concentrate on Defcore to define what are core APIs. Thanks Ken Ohmichi --- [1]: https://github.com/openstack/nova/blob/master/nova/api/openstack/__init__.py#L66 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
On 2015-03-23 21:31:30 -0400 (-0400), Jay Pipes wrote: On Mon, Mar 23, 2015 at 09:35:50PM +, Jeremy Stanley wrote: On 2015-03-23 15:15:18 -0400 (-0400), Jay Pipes wrote: [...] I don't want it suppressed. I want the use of API extensions and the extension framework(s) to be completely dropped for all future API-affecting work. [...] Perhaps controversial, but would it be worthwhile to propose to the Defcore working group that future compliance requirements include the absence of extensions to officially covered APIs? I don't understand what you're getting at, Jeremy. Could you elaborate? What do extensions have to do with future compliance requirements? Defcore's focus is on establishing interoperability standards for OpenStack deployments, to ease the end-user experience. Right now its model depends on a whitelist of API features. As discussed many times before and brought up again in this thread, when providers or distributors augment OpenStack APIs to add their own special features without implementing them upstream, this necessarily creates interoperability issues. What I'm suggesting is that Defcore should not just specify which API features need to be supported, but also specify that nonstandard API features must not be added to the APIs its covering. -- Jeremy Stanley __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
On Tue, Mar 24, 2015 at 7:10 AM, Jay Pipes jaypi...@gmail.com wrote: On Tue, Mar 24, 2015 at 01:04:46PM +, Jeremy Stanley wrote: On 2015-03-23 21:31:30 -0400 (-0400), Jay Pipes wrote: On Mon, Mar 23, 2015 at 09:35:50PM +, Jeremy Stanley wrote: On 2015-03-23 15:15:18 -0400 (-0400), Jay Pipes wrote: [...] I don't want it suppressed. I want the use of API extensions and the extension framework(s) to be completely dropped for all future API-affecting work. [...] Perhaps controversial, but would it be worthwhile to propose to the Defcore working group that future compliance requirements include the absence of extensions to officially covered APIs? I don't understand what you're getting at, Jeremy. Could you elaborate? What do extensions have to do with future compliance requirements? Defcore's focus is on establishing interoperability standards for OpenStack deployments, to ease the end-user experience. Right now its model depends on a whitelist of API features. As discussed many times before and brought up again in this thread, when providers or distributors augment OpenStack APIs to add their own special features without implementing them upstream, this necessarily creates interoperability issues. Defcore's focus is on determining what is OpenStack, w.r.t. what is brandable as OpenStack. It's focus is not on establishing interoperability standards. I am not sure how you got to that conclusion, yes the defcore process has been very confusing and I am still not really sure what it was, but some part of it it *is* about interoperability/ Although our wiki does get out of date very easily, I think this still holds true: *DefCore sets base requirements by defining 1) capabilities, 2) code and 3) must-pass tests for all OpenStack products. This definition uses community resources and involvement to drive interoperability by creating the minimum standards for products labeled OpenStack.* *https://wiki.openstack.org/wiki/Governance/DefCoreCommittee https://wiki.openstack.org/wiki/Governance/DefCoreCommittee* Here is another source: DefCore has a single goal expressed from two sides: 1) defining the “what is OpenStack” brand for Vendors and 2) driving interoperability between OpenStack installations. http://robhirschfeld.com/2015/02/26/openstack-defcore-accelerates-simplifies-with-clear-and-timely-guidelines-feedback/ What I'm suggesting is that Defcore should not just specify which API features need to be supported, but also specify that nonstandard API features must not be added to the APIs its covering. We're perilously close to re-igniting the is OpenStack a set of APIs, or is OpenStack a set of implementations of those APIs debate. A debate I'm happy to have, of course :) I'm really not a fan of the Defcore effort. This should come as no surprise to anyone. I've been quite blunt about my disdain for the focus on identifying which API things are mandatory and which are optional, in order to say some vendor's product is OpenStack. That said, I'm not going to get in its way. If you think that Defcore should amend its compliancy guidelines to include the above, fine by me. Best, -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
On Tue, Mar 24, 2015 at 01:04:46PM +, Jeremy Stanley wrote: On 2015-03-23 21:31:30 -0400 (-0400), Jay Pipes wrote: On Mon, Mar 23, 2015 at 09:35:50PM +, Jeremy Stanley wrote: On 2015-03-23 15:15:18 -0400 (-0400), Jay Pipes wrote: [...] I don't want it suppressed. I want the use of API extensions and the extension framework(s) to be completely dropped for all future API-affecting work. [...] Perhaps controversial, but would it be worthwhile to propose to the Defcore working group that future compliance requirements include the absence of extensions to officially covered APIs? I don't understand what you're getting at, Jeremy. Could you elaborate? What do extensions have to do with future compliance requirements? Defcore's focus is on establishing interoperability standards for OpenStack deployments, to ease the end-user experience. Right now its model depends on a whitelist of API features. As discussed many times before and brought up again in this thread, when providers or distributors augment OpenStack APIs to add their own special features without implementing them upstream, this necessarily creates interoperability issues. Defcore's focus is on determining what is OpenStack, w.r.t. what is brandable as OpenStack. It's focus is not on establishing interoperability standards. What I'm suggesting is that Defcore should not just specify which API features need to be supported, but also specify that nonstandard API features must not be added to the APIs its covering. We're perilously close to re-igniting the is OpenStack a set of APIs, or is OpenStack a set of implementations of those APIs debate. A debate I'm happy to have, of course :) I'm really not a fan of the Defcore effort. This should come as no surprise to anyone. I've been quite blunt about my disdain for the focus on identifying which API things are mandatory and which are optional, in order to say some vendor's product is OpenStack. That said, I'm not going to get in its way. If you think that Defcore should amend its compliancy guidelines to include the above, fine by me. Best, -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
Jeremy Stanley on March 24, 2015 07:28 wrote: On 2015-03-24 10:10:07 -0400 (-0400), Jay Pipes wrote: [...] I'm really not a fan of the Defcore effort. This should come as no surprise to anyone. I've been quite blunt about my disdain for the focus on identifying which API things are mandatory and which are optional, in order to say some vendor's product is OpenStack. [...] Understandable--and I'm certainly no fan of bureaucracy and politicking--just pointing out that in the carrots-and-sticks bin we have trademarks as a potential carrot to encourage people running OpenStack commercially to further the goals of our community. If there is broad community support for the idea that standard OpenStack APIs should not be arbitrarily extended downstream because it hurts interoperability, then this might be one way to improve the situation. If there are other means to achieve this then they too should be considered, obviously. [Rockyg] Tl;dr. Propose it to DefCore. They might agree. But they might have action items for developers to make it actionable on DefCore's part. Hey, it sounds like a reasonable request to me, and it's a large operator making the request for DefCore to recognize that extensions break interoperability. Plus, Defcore specifically states that vendor specific code within OpenStack is not considered for inclusion in the Defcore capabilities/test/designated code sets. So then the questions become: * Are there any currently included capabilities that assume extensions work? * Are there any likely candidates for future Defcore sets that assume extensions work? * Will extensions be deprecated before capabilities requiring extensions are in the candidate pool? * Once extensions are deprecated, or it is known that nothing in Defcore sets include the ability to use extensions, are there tests which will validate that extensions are not part of the cloud under test? Something to think about, and if you are serious about limiting the negative effects of extensions, it is certainly worth proposing their elimination and/or restrictions on them to Defcore. And an FYI, APIs and tests are not chosen by DefCore because they are meaningful, but because they are measurable. DefCore uses User Surveys (and who knows what else) to determine adoption rates of OpenStack capabilities that are then converted to APIs and tests, etc. If you have a implementable ideas on how better to measure individual clouds/cloud products against the user communities' utilization, they'd love to hear from you. No, really. DefCore wants to hear of better ways to ensure that clouds with the OpenStack trademark mean that user implementations on those clouds are portable across compliant vendors within the scope of what those vendors advertise. --rocky -- Jeremy Stanley __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
On 2015-03-24 10:10:07 -0400 (-0400), Jay Pipes wrote: [...] I'm really not a fan of the Defcore effort. This should come as no surprise to anyone. I've been quite blunt about my disdain for the focus on identifying which API things are mandatory and which are optional, in order to say some vendor's product is OpenStack. [...] Understandable--and I'm certainly no fan of bureaucracy and politicking--just pointing out that in the carrots-and-sticks bin we have trademarks as a potential carrot to encourage people running OpenStack commercially to further the goals of our community. If there is broad community support for the idea that standard OpenStack APIs should not be arbitrarily extended downstream because it hurts interoperability, then this might be one way to improve the situation. If there are other means to achieve this then they too should be considered, obviously. -- Jeremy Stanley __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
On Tue, Mar 24, 2015 at 5:45 AM, Jay Pipes jaypi...@gmail.com wrote:
On Tue, Mar 10, 2015 at 09:52:14PM +1030, Christopher Yeoh wrote:
On Mon, 09 Mar 2015 16:14:21 -0400
Sean Dague s...@dague.net wrote:
On 03/09/2015 03:37 PM, Jay Pipes wrote:
On 03/08/2015 08:10 AM, Alex Xu wrote:
Thanks for Jay point this out! If we have agreement on this and
document it, that will be great for guiding developer how to add
new API.
I know we didn't want extension for API. But I think we still
need modularity. I don't think we should put everything in a single
file, that file will become huge in the future and hard to
maintenance.
I don't think everything should be in a single file either. In fact,
I've never advocated for that.
We can make the 'extension' not configurable. Replace 'extension'
with another name, deprecate the extension info api int he
future But that is not mean we should put everything in a file.
I didn't say that in my email. I'm not sure where you got the
impression I want to put everything in one file?
For modularity, we need define what should be in a separated
module(it is extension now.) There are three cases:
1. Add new resource
This is totally worth to put in a separated module.
Agreed.
2. Add new sub-resource
like server-tags, I prefer to put in a separated module, I
don't think put another 100 lines code in the servers.py is good
choice.
Agreed, which is exactly what I said in my email:
Similarly, new microversion API functionality should live in a
module, as a top-level (or subcollection) Controller in
/nova/api/openstack/compute/, and should not be in the
/nova/api/openstack/compute/plugins/ directory. Why? Because it's
not a plugin.
Ok so I'm getting confused about what we're disagreeing about then.
I wasn't disagreeing with you. I was disagreeing with Alex Xu :) Perhaps
your mail client was confusing you?
However, the first microversion change
https://review.openstack.org/#/c/140313/32 is one case where we didn't
need to create a new extension, relying only on microversions to add a
new parameter to the response, whereas server tags does add a new
conroller (os-server-tags) which is non trivia so I think it does. Do
we disagree about that?
The server tags patch does add more functionality to the API, for sure.
But, as I've written about in this thread, there's no reason at all that
it needed to use the extension framework. It could better have been
written as a simple Controller object in a new file in
/nova/api/openstack/compute/server_tags.py, with no use of the
extension mechanism whatsoever.
btw in a situation now where (I think) we are saying we are not going
to do any work for 3rd parties to add their own API plugins and have a
well planned API we don't need the prefixes as we don't need the prefix
on parameter names as there won't be name clashes without us realising
during testing. And we certainly don't need the os- prefix is the
plugin alias, but we do neeed them unique across the API I believe
because of the way we store information about them.
Right. There won't be any more name clashes if we don't allow API
extensions any more. Which is what I'm asking for.
3. extend attributes and methods for a existed resource
like add new attributes for servers, we can choice one of
existed module to put it in. Just like this patch
https://review.openstack.org/#/c/155853/
But for servers-tags, it's sub-resource, we can put it in
its-own module.
Agreed, and that's what I put in my email.
If we didn't want to support extension right now, we can begin
from not show servers-tags in extension info API first. That means
extension info is freeze now. We deprecated the extension info api
in later version.
It can be surpressed from /extensions by adding it to the suppress list
in the extensions code. Probably a good idea to stop v2.1+microversion
code REST API users not accidentally relying on it.
I don't want it suppressed. I want the use of API extensions and the
extension framework(s) to be completely dropped for all future
API-affecting work.
I don't understand what you're saying here. Could you elaborate?
What I am asking for is for new functionality (like the server-tags
subcollection resource), just add a new module called
/nova/api/openstack/compute/server_tags.py, create a Controller
object in that file with the new server tags resource, and don't
use any of the API extensions framework whatsoever.
In addition to that, for the changes to the main GET
/servers/{server_id} resource, use microversions to decorate the
/nova/api/openstack/compute/servers.py.Controller.show() method for
2.4 and add a tags key to the dict (note: NOT a
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
On 2015-03-23 15:15:18 -0400 (-0400), Jay Pipes wrote: [...] I don't want it suppressed. I want the use of API extensions and the extension framework(s) to be completely dropped for all future API-affecting work. [...] Perhaps controversial, but would it be worthwhile to propose to the Defcore working group that future compliance requirements include the absence of extensions to officially covered APIs? -- Jeremy Stanley __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
On Tue, Mar 10, 2015 at 09:52:14PM +1030, Christopher Yeoh wrote:
On Mon, 09 Mar 2015 16:14:21 -0400
Sean Dague s...@dague.net wrote:
On 03/09/2015 03:37 PM, Jay Pipes wrote:
On 03/08/2015 08:10 AM, Alex Xu wrote:
Thanks for Jay point this out! If we have agreement on this and
document it, that will be great for guiding developer how to add
new API.
I know we didn't want extension for API. But I think we still
need modularity. I don't think we should put everything in a single
file, that file will become huge in the future and hard to
maintenance.
I don't think everything should be in a single file either. In fact,
I've never advocated for that.
We can make the 'extension' not configurable. Replace 'extension'
with another name, deprecate the extension info api int he
future But that is not mean we should put everything in a file.
I didn't say that in my email. I'm not sure where you got the
impression I want to put everything in one file?
For modularity, we need define what should be in a separated
module(it is extension now.) There are three cases:
1. Add new resource
This is totally worth to put in a separated module.
Agreed.
2. Add new sub-resource
like server-tags, I prefer to put in a separated module, I
don't think put another 100 lines code in the servers.py is good
choice.
Agreed, which is exactly what I said in my email:
Similarly, new microversion API functionality should live in a
module, as a top-level (or subcollection) Controller in
/nova/api/openstack/compute/, and should not be in the
/nova/api/openstack/compute/plugins/ directory. Why? Because it's
not a plugin.
Ok so I'm getting confused about what we're disagreeing about then.
I wasn't disagreeing with you. I was disagreeing with Alex Xu :) Perhaps
your mail client was confusing you?
However, the first microversion change
https://review.openstack.org/#/c/140313/32 is one case where we didn't
need to create a new extension, relying only on microversions to add a
new parameter to the response, whereas server tags does add a new
conroller (os-server-tags) which is non trivia so I think it does. Do
we disagree about that?
The server tags patch does add more functionality to the API, for sure.
But, as I've written about in this thread, there's no reason at all that
it needed to use the extension framework. It could better have been
written as a simple Controller object in a new file in
/nova/api/openstack/compute/server_tags.py, with no use of the
extension mechanism whatsoever.
btw in a situation now where (I think) we are saying we are not going
to do any work for 3rd parties to add their own API plugins and have a
well planned API we don't need the prefixes as we don't need the prefix
on parameter names as there won't be name clashes without us realising
during testing. And we certainly don't need the os- prefix is the
plugin alias, but we do neeed them unique across the API I believe
because of the way we store information about them.
Right. There won't be any more name clashes if we don't allow API
extensions any more. Which is what I'm asking for.
3. extend attributes and methods for a existed resource
like add new attributes for servers, we can choice one of
existed module to put it in. Just like this patch
https://review.openstack.org/#/c/155853/
But for servers-tags, it's sub-resource, we can put it in
its-own module.
Agreed, and that's what I put in my email.
If we didn't want to support extension right now, we can begin
from not show servers-tags in extension info API first. That means
extension info is freeze now. We deprecated the extension info api
in later version.
It can be surpressed from /extensions by adding it to the suppress list
in the extensions code. Probably a good idea to stop v2.1+microversion
code REST API users not accidentally relying on it.
I don't want it suppressed. I want the use of API extensions and the
extension framework(s) to be completely dropped for all future
API-affecting work.
I don't understand what you're saying here. Could you elaborate?
What I am asking for is for new functionality (like the server-tags
subcollection resource), just add a new module called
/nova/api/openstack/compute/server_tags.py, create a Controller
object in that file with the new server tags resource, and don't
use any of the API extensions framework whatsoever.
In addition to that, for the changes to the main GET
/servers/{server_id} resource, use microversions to decorate the
/nova/api/openstack/compute/servers.py.Controller.show() method for
2.4 and add a tags key to the dict (note: NOT a
os-server-tags:tags key) returned by GET /servers/{server_id}. No
use of API extensions needed.
So I that's doabe but I think if we compared to the two wsgi.extends
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
Sorry for the delay in responding all. Comments inline. On Mon, Mar 09, 2015 at 03:04:59PM -0700, melanie witt wrote: On Mar 9, 2015, at 13:14, Sean Dague s...@dague.net wrote: So possibly another way to think about this is our prior signaling of what was supported by Nova was signaled by the extension list. Our code was refactored into a way that supported optional loading by that unit. As we're making things less optional it probably makes sense to evolve the API code tree to look more like our REST resource tree. Yes, that means servers.py ends up being big, but it is less confusing that all servers related code is in that file vs all over a bunch of other files. So I'd agree that in this case server tags probably should just be in servers.py. I also think long term we should do some plugin collapse for stuff that's all really just features on one resource tree so that the local filesystem code structure looks a bit more like the REST url tree. I think this makes a lot of sense. When I read the question, why is server tags being added as an extension the answer that comes to mind first is, because the extension framework is there and that's how things have been done so far. I think the original thinking on extensions was, make everything optional so users can enable/disable as they please, operators can disable any feature by removing the extension. Another benefit is the ability for anyone to add a (non-useful to the community at-large) feature without having to patch in several places. I used to be for extensions for the aforementioned benefits, but now I tend to think it's too flexible and complex. It's so flexible that you can easily get yourself into a situation where your deployment can't work with other useful tools/libraries/etc which expect a certain contract from the Nova API. It doesn't make sense to let the API we provide be so arbitrary. It's certainly not friendly to API users. Right. This is the number one reason API extensions need to go bye bye. We still have the ability to disable or limit features based on policy -- I don't think we need to do it via extensions. Agreed. The only problem that seems to be left is, how can we allow people to add un-upstreamable features to the API in their internal deployments? I know the ideal answer is don't do that but the reality is some things will never be agreed upon upstream and I do see value in the extension framework for that. I don't think anything in-tree should be implemented as extensions, though. IMO, all of the above functionality belongs as an entirely different endpoint and REST API. If it's not in the Nova /nova/api/ directory, it's not the Nova API. As I've written a number of times before, just because achieving some modicum of consensus about some new API resource or feature is hard, does not mean that we should spend our time enabling vendors and others to circumvent the API writing process by using API extensions. It's hard work coming to consensus about a new API feature. But it's necessary work. Best, -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
On Mon, Mar 23, 2015 at 2:35 PM, Jeremy Stanley fu...@yuggoth.org wrote: On 2015-03-23 15:15:18 -0400 (-0400), Jay Pipes wrote: [...] I don't want it suppressed. I want the use of API extensions and the extension framework(s) to be completely dropped for all future API-affecting work. [...] Perhaps controversial, but would it be worthwhile to propose to the Defcore working group that future compliance requirements include the absence of extensions to officially covered APIs? ++ -- Jeremy Stanley __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
On Mon, Mar 23, 2015 at 09:35:50PM +, Jeremy Stanley wrote: On 2015-03-23 15:15:18 -0400 (-0400), Jay Pipes wrote: [...] I don't want it suppressed. I want the use of API extensions and the extension framework(s) to be completely dropped for all future API-affecting work. [...] Perhaps controversial, but would it be worthwhile to propose to the Defcore working group that future compliance requirements include the absence of extensions to officially covered APIs? I don't understand what you're getting at, Jeremy. Could you elaborate? What do extensions have to do with future compliance requirements? Best, -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
On Mon, 09 Mar 2015 16:14:21 -0400
Sean Dague s...@dague.net wrote:
On 03/09/2015 03:37 PM, Jay Pipes wrote:
On 03/08/2015 08:10 AM, Alex Xu wrote:
Thanks for Jay point this out! If we have agreement on this and
document it, that will be great for guiding developer how to add
new API.
I know we didn't want extension for API. But I think we still
need modularity. I don't think we should put everything in a single
file, that file will become huge in the future and hard to
maintenance.
I don't think everything should be in a single file either. In fact,
I've never advocated for that.
We can make the 'extension' not configurable. Replace 'extension'
with another name, deprecate the extension info api int he
future But that is not mean we should put everything in a file.
I didn't say that in my email. I'm not sure where you got the
impression I want to put everything in one file?
For modularity, we need define what should be in a separated
module(it is extension now.) There are three cases:
1. Add new resource
This is totally worth to put in a separated module.
Agreed.
2. Add new sub-resource
like server-tags, I prefer to put in a separated module, I
don't think put another 100 lines code in the servers.py is good
choice.
Agreed, which is exactly what I said in my email:
Similarly, new microversion API functionality should live in a
module, as a top-level (or subcollection) Controller in
/nova/api/openstack/compute/, and should not be in the
/nova/api/openstack/compute/plugins/ directory. Why? Because it's
not a plugin.
Ok so I'm getting confused about what we're disagreeing about then.
However, the first microversion change
https://review.openstack.org/#/c/140313/32 is one case where we didn't
need to create a new extension, relying only on microversions to add a
new parameter to the response, whereas server tags does add a new
conroller (os-server-tags) which is non trivia so I think it does. Do
we disagree about that?
btw in a situation now where (I think) we are saying we are not going
to do any work for 3rd parties to add their own API plugins and have a
well planned API we don't need the prefixes as we don't need the prefix
on parameter names as there won't be name clashes without us realising
during testing. And we certainly don't need the os- prefix is the
plugin alias, but we do neeed them unique across the API I believe
because of the way we store information about them.
3. extend attributes and methods for a existed resource
like add new attributes for servers, we can choice one of
existed module to put it in. Just like this patch
https://review.openstack.org/#/c/155853/
But for servers-tags, it's sub-resource, we can put it in
its-own module.
Agreed, and that's what I put in my email.
If we didn't want to support extension right now, we can begin
from not show servers-tags in extension info API first. That means
extension info is freeze now. We deprecated the extension info api
in later version.
It can be surpressed from /extensions by adding it to the suppress list
in the extensions code. Probably a good idea to stop v2.1+microversion
code REST API users not accidentally relying on it.
I don't understand what you're saying here. Could you elaborate?
What I am asking for is for new functionality (like the server-tags
subcollection resource), just add a new module called
/nova/api/openstack/compute/server_tags.py, create a Controller
object in that file with the new server tags resource, and don't
use any of the API extensions framework whatsoever.
In addition to that, for the changes to the main GET
/servers/{server_id} resource, use microversions to decorate the
/nova/api/openstack/compute/servers.py.Controller.show() method for
2.4 and add a tags key to the dict (note: NOT a
os-server-tags:tags key) returned by GET /servers/{server_id}. No
use of API extensions needed.
So I that's doabe but I think if we compared to the two wsgi.extends
is cleaner and less code and we have to have the separate module file
anyway for the controller. Can discuss this more later
Incidentally as has been mentioned before we need new names for the
API and where the files need to live needs cleaning up. For example v3
and v2.1 needs to be worked out of the directory paths.This clenaup not
on involves but directly affects the all the related unit and
functional tests. Nor should we have contrib or should v2 live directly
in in nova/api/openstack/compute. But we also need a name for v2.1
microversions and should spend some time on that (does api modules work
for anyone?)
I think this dicussion indicates we should start with a bit of planning
first rather than just jump in and start shuffling this around now.
Work out what we want the final thing to look like so we can see what
the dependencies look like and minimise the number of times
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
On 03/09/2015 06:04 PM, melanie witt wrote: On Mar 9, 2015, at 13:14, Sean Dague s...@dague.net wrote: So possibly another way to think about this is our prior signaling of what was supported by Nova was signaled by the extension list. Our code was refactored into a way that supported optional loading by that unit. As we're making things less optional it probably makes sense to evolve the API code tree to look more like our REST resource tree. Yes, that means servers.py ends up being big, but it is less confusing that all servers related code is in that file vs all over a bunch of other files. So I'd agree that in this case server tags probably should just be in servers.py. I also think long term we should do some plugin collapse for stuff that's all really just features on one resource tree so that the local filesystem code structure looks a bit more like the REST url tree. I think this makes a lot of sense. When I read the question, why is server tags being added as an extension the answer that comes to mind first is, because the extension framework is there and that's how things have been done so far. I think the original thinking on extensions was, make everything optional so users can enable/disable as they please, operators can disable any feature by removing the extension. Another benefit is the ability for anyone to add a (non-useful to the community at-large) feature without having to patch in several places. I used to be for extensions for the aforementioned benefits, but now I tend to think it's too flexible and complex. It's so flexible that you can easily get yourself into a situation where your deployment can't work with other useful tools/libraries/etc which expect a certain contract from the Nova API. It doesn't make sense to let the API we provide be so arbitrary. It's certainly not friendly to API users. We still have the ability to disable or limit features based on policy -- I don't think we need to do it via extensions. The only problem that seems to be left is, how can we allow people to add un-upstreamable features to the API in their internal deployments? I know the ideal answer is don't do that but the reality is some things will never be agreed upon upstream and I do see value in the extension framework for that. I don't think anything in-tree should be implemented as extensions, though. At the moment this is provided for by an experimental flag in the response headers: https://git.openstack.org/cgit/openstack/nova-specs/tree/specs/kilo/approved/api-microversions.rst#n182 . It is intended to be used for transitioning from the current state of extensions to a place where optional API extensions aren't allowed, but that discussion can continue if there's a case for allowing some optional components for deployers. I'm in favor of having a mechanism for adding features to an deployment as long as it's exposed in a way that makes it clear that it's separate from the standard API, e.g. an entirely separate tree, not just resource prefixes. melanie (melwitt) __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
I have done the first version, follow this discussion I separated them into two patches: 1. The discussion about eliminated extension: https://review.openstack.org/162912 2. The discussion about modularity: https://review.openstack.org/162913 After begin the writefound I loss some confidence for my English and view. So please feel free to comment and update the doc to add your thought and your view, and add yourself as co-author.(If you feel hopeless, you also can start new one) Thanks Alex 2015-03-09 23:37 GMT+08:00 Alex Xu sou...@gmail.com: ok, no problem, will take a look it tomorrow. 2015-03-09 20:18 GMT+08:00 Christopher Yeoh cbky...@gmail.com: On Mon, Mar 9, 2015 at 10:08 PM, John Garbutt j...@johngarbutt.com wrote: +1 Please could you submit a dev ref for this? We can argue on the review, a bit like this one: https://github.com/openstack/nova/blob/master/doc/source/devref/policy_enforcement.rst I think it'd also be a good idea to add a testcase (use test_plugins directory where you can define your own controller which is never plubished) for each example so they don't get out of date Regards, Chris __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
2015-03-10 3:37 GMT+08:00 Jay Pipes jaypi...@gmail.com:
On 03/08/2015 08:10 AM, Alex Xu wrote:
Thanks for Jay point this out! If we have agreement on this and document
it, that will be great for guiding developer how to add new API.
I know we didn't want extension for API. But I think we still
need modularity. I don't think we should put everything in a single
file, that file will become huge in the future and hard to maintenance.
I don't think everything should be in a single file either. In fact, I've
never advocated for that.
We can make the 'extension' not configurable. Replace 'extension' with
another name, deprecate the extension info api int he future But
that is not mean we should put everything in a file.
I didn't say that in my email. I'm not sure where you got the impression I
want to put everything in one file?
For modularity, we need define what should be in a separated module(it
is extension now.) There are three cases:
1. Add new resource
This is totally worth to put in a separated module.
Agreed.
2. Add new sub-resource
like server-tags, I prefer to put in a separated module, I don't
think put another 100 lines code in the servers.py is good choice.
Agreed, which is exactly what I said in my email:
Similarly, new microversion API functionality should live in a
module, as a top-level (or subcollection) Controller in
/nova/api/openstack/compute/, and should not be in the
/nova/api/openstack/compute/plugins/ directory. Why? Because it's
not a plugin.
3. extend attributes and methods for a existed resource
like add new attributes for servers, we can choice one of existed
module to put it in. Just like this patch
https://review.openstack.org/#/c/155853/
But for servers-tags, it's sub-resource, we can put it in its-own
module.
Agreed, and that's what I put in my email.
If we didn't want to support extension right now, we can begin from not
show servers-tags in extension info API first. That means extension info
is freeze now. We deprecated the extension info api in later version.
I don't understand what you're saying here. Could you elaborate? What I am
asking for is for new functionality (like the server-tags subcollection
resource), just add a new module called
/nova/api/openstack/compute/server_tags.py,
create a Controller object in that file with the new server tags resource,
and don't use any of the API extensions framework whatsoever.
Ok, I think I see you now. Forgive me for any misunderstood.
Actually I mean we can freeze and depreciated the '/extensions' API at
here. We can freeze the '/extensions' API now, didn't show server-tags
extension in that API.
Follow the discussion, I think there is two things related to 'extension'.
One is '/extensions' API, another one is plugin framework. And I added
devref about that https://review.openstack.org/162912
For the modularity, I put in separated patch
https://review.openstack.org/162913
Those two things I think we can discussion them separately. (Emm.This
is kind of evidence about I misunderstood you)
In addition to that, for the changes to the main GET /servers/{server_id}
resource, use microversions to decorate the
/nova/api/openstack/compute/servers.py.Controller.show()
method for 2.4 and add a tags key to the dict (note: NOT a
os-server-tags:tags key) returned by GET /servers/{server_id}. No use of
API extensions needed.
Best,
-jay
Thanks
Alex
2015-03-08 8:31 GMT+08:00 Jay Pipes jaypi...@gmail.com
mailto:jaypi...@gmail.com:
Hi Stackers,
Now that microversions have been introduced to the Nova API (meaning
we can now have novaclient request, say, version 2.3 of the Nova API
using the special X-OpenStack-Nova-API-Version HTTP header), is
there any good reason to require API extensions at all for *new*
functionality.
Sergey Nikitin is currently in the process of code review for the
final patch that adds server instance tagging to the Nova API:
https://review.openstack.org/#__/c/128940
https://review.openstack.org/#/c/128940
Unfortunately, for some reason I really don't understand, Sergey is
being required to create an API extension called os-server-tags in
order to add the server tag functionality to the API. The patch
implements the 2.4 Nova API microversion, though, as you can see
from this part of the patch:
https://review.openstack.org/#__/c/128940/43/nova/api/__
openstack/compute/plugins/v3/__server_tags.py
https://review.openstack.org/#/c/128940/43/nova/api/
openstack/compute/plugins/v3/server_tags.py
What is the point of creating a new plugin/API extension for this
new functionality? Why can't we just modify the
nova/api/openstack/compute/__server.py Controller.show() method and
decorate it with a 2.4 microversion that adds a tags attribute to
the returned server dictionary?
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
On Mon, Mar 9, 2015 at 10:08 PM, John Garbutt j...@johngarbutt.com wrote: Hi, I think I agree with Jay here, but let me explain... On 8 March 2015 at 12:10, Alex Xu sou...@gmail.com wrote: Thanks for Jay point this out! If we have agreement on this and document it, that will be great for guiding developer how to add new API. +1 Please could you submit a dev ref for this? We can argue on the review, a bit like this one: https://github.com/openstack/nova/blob/master/doc/source/devref/policy_enforcement.rst For modularity, we need define what should be in a separated module(it is extension now.) There are three cases: 1. Add new resource This is totally worth to put in a separated module. +1 2. Add new sub-resource like server-tags, I prefer to put in a separated module, I don't think put another 100 lines code in the servers.py is good choice. -1 I hate the idea of show instance extension code for version 2.4 living separately to the rest of the instance show logic, when it really doesn't have to. It feels too heavyweight in its current form. If the only thing server-tags did was to add a parameter then we wouldn't need a new extension, but its not, it adds another resource with associated actions Maybe we need a more modular way of expressing the extension within the same file? I think servers.py is simply to big. Its much harder to read and debug than any other plugin just because of its size - or maybe I just need a 50 monitor :) I'd rather ensure functionality common server-tags and the API is kept together rather than spread through servers.py 3. extend attributes and methods for a existed resource like add new attributes for servers, we can choice one of existed module to put it in. Just like this patch https://review.openstack.org/#/c/155853/ +1 I wish it was easier to read, but I hope thats fixable long term. 2015-03-08 8:31 GMT+08:00 Jay Pipes jaypi...@gmail.com: Now that microversions have been introduced to the Nova API (meaning we can now have novaclient request, say, version 2.3 of the Nova API using the special X-OpenStack-Nova-API-Version HTTP header), is there any good reason to require API extensions at all for *new* functionality. As above, a new resource probably should get a new plugins/v3 module right? It feels (at worst) borderline in the os-server-tags case, due to the extra actions. What is the point of creating a new plugin/API extension for this new functionality? Why can't we just modify the nova/api/openstack/compute/server.py Controller.show() method and decorate it with a 2.4 microversion that adds a tags attribute to the returned server dictionary? Similarly, new microversion API functionality should live in a module, as a top-level (or subcollection) Controller in /nova/api/openstack/compute/, and should not be in the /nova/api/openstack/compute/plugins/ directory. Why? Because it's not a plugin. Everything is a plugin in v3, no more distinction between core vs plugin. It needs renaming really. It should look just like servers, I guess, which is a top level item: https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/plugins/v3/servers.py Why are we continuing to use these awkward, messy, and cumbersome API extensions? We certainly should never be forced to add an extension to advertise new functionality anymore. Its a big reason why I want to see the API micro-versions succeed. Yep, there is I think no reason except to support /extensions for now and I don't really think its worth having two entry points, one for modules which will appear in /extensions and one for modules which won't appear in /extensioins. The overhead is low. We should warn v2.1+ users to ignore /extensions unless they are legacy v2 api users and they should remove their use of it anyway as soon as they get off v2.1. They key to dumping it all is when people tell us v2.1 really is behaving just like v2 so we can remove the old v2 code and then later have a microversion that doesn't support /extensions. I hope all the json-home stuff is in by then :-) Please, I am begging the Nova core team. Let us stop this madness. No more API extensions. Lets try get something agreed in devref, so we are ready to go when Liberty opens. It would be nice to look at ways to fold back the existing extensions into the main code. I know there are v2.0 compatibility issues there, but I think/hope thats mostly cosmetic at this point. Yea we already did a lot of that in v3 and had to separate some of them out again for v2.1 (argh!). Others we have just faked (eg you load module X you get module Y for free which doesn't really exist anymore - but only for those that we were very sure that the extension only existed to notify users that some functionality was present. Thanks, John
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
On Mon, Mar 9, 2015 at 10:08 PM, John Garbutt j...@johngarbutt.com wrote: +1 Please could you submit a dev ref for this? We can argue on the review, a bit like this one: https://github.com/openstack/nova/blob/master/doc/source/devref/policy_enforcement.rst I think it'd also be a good idea to add a testcase (use test_plugins directory where you can define your own controller which is never plubished) for each example so they don't get out of date Regards, Chris __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
On Mon, Mar 9, 2015 at 9:35 PM, Sean Dague s...@dague.net wrote: On 03/07/2015 07:31 PM, Jay Pipes wrote: Hi Stackers, Now that microversions have been introduced to the Nova API (meaning we can now have novaclient request, say, version 2.3 of the Nova API using the special X-OpenStack-Nova-API-Version HTTP header), is there any good reason to require API extensions at all for *new* functionality. Sergey Nikitin is currently in the process of code review for the final patch that adds server instance tagging to the Nova API: https://review.openstack.org/#/c/128940 Unfortunately, for some reason I really don't understand, Sergey is being required to create an API extension called os-server-tags in order to add the server tag functionality to the API. The patch implements the 2.4 Nova API microversion, though, as you can see from this part of the patch: https://review.openstack.org/#/c/128940/43/nova/api/openstack/compute/plugins/v3/server_tags.py What is the point of creating a new plugin/API extension for this new functionality? Why can't we just modify the nova/api/openstack/compute/server.py Controller.show() method and decorate it with a 2.4 microversion that adds a tags attribute to the returned server dictionary? https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/servers.py#L369 Because we're using an API extension for this new server tags functionality, we are instead having the extension extend the server dictionary with an os-server-tags:tags key containing the list of string tags. This is ugly and pointless. We don't need to use API extensions any more for this stuff. A client knows that server tags are supported by the 2.4 API microversion. If the client requests the 2.4+ API, then we should just include the tags attribute in the server dictionary. Similarly, new microversion API functionality should live in a module, as a top-level (or subcollection) Controller in /nova/api/openstack/compute/, and should not be in the /nova/api/openstack/compute/plugins/ directory. Why? Because it's not a plugin. Why are we continuing to use these awkward, messy, and cumbersome API extensions? Please, I am begging the Nova core team. Let us stop this madness. No more API extensions. Agreed, the current extensions list exists to explain the base v2 functionality. I think we should consider that frozen and deprecated as of v2.1 as we have a better way to express features. -Sean So I think we can a microversion ASAP to remove support for /extensions. Obviously we'll need to keep the actual code to support v2.1 for quite a while though. I think we still want some fields in the controller like we do because we want to automate JSON-HOME/Schema stuff (maybe not sure) -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
Hi, Apologies for the slow reply, long weekend because of a public holiday over here. I'm probably going to end up repeating part of what Alex has mentioned as well. So the first thing I think we want to distinguish between plugins being a REST API user or operator concept and it being a tool developers use as a framework to support the Nova REST API. As I've mentioned before I've no problem with the feature set of the API being fixed (per microversion) across all Nova deployments. Get back to me when we have consensus on that and its trivial to implement and we'll no longer have the concept of core and extension/plugin. But plugin like implementations using Stevedore as a tool for developers to keep good modularity has proven to be very useful to keep complexity level lower and interactions between modules much clearer. servers.py is an example of this where in v2 I think we have/had the most complex method and even with all the fix up work which has been done on it it is still very complicated to understand. On Sun, Mar 8, 2015 at 11:01 AM, Jay Pipes jaypi...@gmail.com wrote: Hi Stackers, Now that microversions have been introduced to the Nova API (meaning we can now have novaclient request, say, version 2.3 of the Nova API using the special X-OpenStack-Nova-API-Version HTTP header), is there any good reason to require API extensions at all for *new* functionality. Sergey Nikitin is currently in the process of code review for the final patch that adds server instance tagging to the Nova API: https://review.openstack.org/#/c/128940 Unfortunately, for some reason I really don't understand, Sergey is being required to create an API extension called os-server-tags in order to add the server tag functionality to the API. The patch implements the 2.4 Nova API microversion, though, as you can see from this part of the patch: https://review.openstack.org/#/c/128940/43/nova/api/ openstack/compute/plugins/v3/server_tags.py What is the point of creating a new plugin/API extension for this new functionality? Why can't we just modify the nova/api/openstack/compute/server.py Controller.show() method and decorate it with a 2.4 microversion that adds a tags attribute to the returned server dictionary? Actually I think it does more than just add extra reponse information: - it adds extra tags parameter to show - it doesn't add it to index, but it probably should add the response information to detail to be consistent with the rest of the API - It adds a new resource /servers/server_id/tags - with create, delete and delete all supported. I don't think that these belong in servers.py https://github.com/openstack/nova/blob/master/nova/api/ openstack/compute/servers.py#L369 Because we're using an API extension for this new server tags functionality, we are instead having the extension extend the server dictionary with an os-server-tags:tags key containing the list of string tags. This is ugly and pointless. We don't need to use API extensions any more for this stuff. So we had a prefix rule originally in V2 to allow for extensions and guarantee no name clashes. I'd be happy removing this requirement, even removing old ones as long as we have consensus. A client knows that server tags are supported by the 2.4 API microversion. If the client requests the 2.4+ API, then we should just include the tags attribute in the server dictionary. Similarly, new microversion API functionality should live in a module, as a top-level (or subcollection) Controller in /nova/api/openstack/compute/, and should not be in the /nova/api/openstack/compute/plugins/ directory. Why? Because it's not a plugin. So I don't see how that changes whether we're using plugins (from a user point of view) or not. The good news for you is that there is fixing the shambles of a directory structure for the api is on the list of things to do, it just wasn't a high prioirty things for us in Kilo, get v2.1 and microversions out. For example, we have v3 in the directory path as well for historical reasons and we also have a contrib directory in compute and none of those are really contrib now either. Now the nova/api/openstack/compute/ directory where you want to put all the v2 microversions code is currently full of v2 core code already. It just makes more sense to me to wait unti the old v2 core code can be removed because the v2.1 api is considered equivalent and then move the v2.1 microversions code into its final place , rather than a shuffle now to move the old v2 code (along with all the changes need to the unittests) and then just have to delete it again not much longer. Why are we continuing to use these awkward, messy, and cumbersome API extensions? Please, I am begging the Nova core team. Let us stop this madness. No more API extensions. It is still not clear to me exactly what you mean by use of an extension. None of us ours are built for real runtime loading/unloading of
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
- Original Message - From: Christopher Yeoh cbky...@gmail.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Monday, March 9, 2015 1:04:15 PM Subject: Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality? On Mon, Mar 9, 2015 at 10:08 PM, John Garbutt j...@johngarbutt.com wrote: Hi, I think I agree with Jay here, but let me explain... On 8 March 2015 at 12:10, Alex Xu sou...@gmail.com wrote: Thanks for Jay point this out! If we have agreement on this and document it, that will be great for guiding developer how to add new API. +1 Please could you submit a dev ref for this? We can argue on the review, a bit like this one: https://github.com/openstack/nova/blob/master/doc/source/devref/policy_enforcement.rst For modularity, we need define what should be in a separated module(it is extension now.) There are three cases: 1. Add new resource This is totally worth to put in a separated module. +1 2. Add new sub-resource like server-tags, I prefer to put in a separated module, I don't think put another 100 lines code in the servers.py is good choice. -1 I hate the idea of show instance extension code for version 2.4 living separately to the rest of the instance show logic, when it really doesn't have to. It feels too heavyweight in its current form. If the only thing server-tags did was to add a parameter then we wouldn't need a new extension, but its not, it adds another resource with associated actions Maybe we need a more modular way of expressing the extension within the same file? I think servers.py is simply to big. Its much harder to read and debug than any other plugin just because of its size - or maybe I just need a 50 monitor :) I'd rather ensure functionality common server-tags and the API is kept together rather than spread through servers.py No, it isn't. It is bellow 2k line. I usually use low level tools even for python related debugging. For ex.: strace, gdb.. With the extension I get lot of files which may be involved may be not. This causes me additional headache, because more difficult to see which file is involved. After an strace I usually know what is the mistake, I just need to find it in the code. I do not like when I had to open more than 3 files, after I see what went wrong. I some cases I use gdb, just to get python stack traces just before the first incorrect step is detected, in other cases git grep is sufficient. Actually for me the extensions increases the required monitor number, some cases I also need to use more complicated approaches. I tied lot of python profiler tool as well, but there is no single all cases win version, extra custom hack is required in many cases to get something close what I want. 3. extend attributes and methods for a existed resource like add new attributes for servers, we can choice one of existed module to put it in. Just like this patch https://review.openstack.org/#/c/155853/ +1 I wish it was easier to read, but I hope thats fixable long term. 2015-03-08 8:31 GMT+08:00 Jay Pipes jaypi...@gmail.com : Now that microversions have been introduced to the Nova API (meaning we can now have novaclient request, say, version 2.3 of the Nova API using the special X-OpenStack-Nova-API-Version HTTP header), is there any good reason to require API extensions at all for *new* functionality. As above, a new resource probably should get a new plugins/v3 module right? It feels (at worst) borderline in the os-server-tags case, due to the extra actions. What is the point of creating a new plugin/API extension for this new functionality? Why can't we just modify the nova/api/openstack/compute/server.py Controller.show() method and decorate it with a 2.4 microversion that adds a tags attribute to the returned server dictionary? Similarly, new microversion API functionality should live in a module, as a top-level (or subcollection) Controller in /nova/api/openstack/compute/, and should not be in the /nova/api/openstack/compute/plugins/ directory. Why? Because it's not a plugin. Everything is a plugin in v3, no more distinction between core vs plugin. It needs renaming really. It should look just like servers, I guess, which is a top level item: https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/plugins/v3/servers.py Why are we continuing to use these awkward, messy, and cumbersome API extensions? We certainly should never be forced to add an extension to advertise new functionality anymore. Its a big reason why I want to see the API micro-versions succeed. Yep, there is I think no reason except to support /extensions for now and I don't really think its worth having two entry points, one for modules which will appear in /extensions
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
Hi, I think I agree with Jay here, but let me explain... On 8 March 2015 at 12:10, Alex Xu sou...@gmail.com wrote: Thanks for Jay point this out! If we have agreement on this and document it, that will be great for guiding developer how to add new API. +1 Please could you submit a dev ref for this? We can argue on the review, a bit like this one: https://github.com/openstack/nova/blob/master/doc/source/devref/policy_enforcement.rst For modularity, we need define what should be in a separated module(it is extension now.) There are three cases: 1. Add new resource This is totally worth to put in a separated module. +1 2. Add new sub-resource like server-tags, I prefer to put in a separated module, I don't think put another 100 lines code in the servers.py is good choice. -1 I hate the idea of show instance extension code for version 2.4 living separately to the rest of the instance show logic, when it really doesn't have to. It feels too heavyweight in its current form. Maybe we need a more modular way of expressing the extension within the same file? 3. extend attributes and methods for a existed resource like add new attributes for servers, we can choice one of existed module to put it in. Just like this patch https://review.openstack.org/#/c/155853/ +1 I wish it was easier to read, but I hope thats fixable long term. 2015-03-08 8:31 GMT+08:00 Jay Pipes jaypi...@gmail.com: Now that microversions have been introduced to the Nova API (meaning we can now have novaclient request, say, version 2.3 of the Nova API using the special X-OpenStack-Nova-API-Version HTTP header), is there any good reason to require API extensions at all for *new* functionality. As above, a new resource probably should get a new plugins/v3 module right? It feels (at worst) borderline in the os-server-tags case, due to the extra actions. What is the point of creating a new plugin/API extension for this new functionality? Why can't we just modify the nova/api/openstack/compute/server.py Controller.show() method and decorate it with a 2.4 microversion that adds a tags attribute to the returned server dictionary? Similarly, new microversion API functionality should live in a module, as a top-level (or subcollection) Controller in /nova/api/openstack/compute/, and should not be in the /nova/api/openstack/compute/plugins/ directory. Why? Because it's not a plugin. Everything is a plugin in v3, no more distinction between core vs plugin. It needs renaming really. It should look just like servers, I guess, which is a top level item: https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/plugins/v3/servers.py Why are we continuing to use these awkward, messy, and cumbersome API extensions? We certainly should never be forced to add an extension to advertise new functionality anymore. Its a big reason why I want to see the API micro-versions succeed. Please, I am begging the Nova core team. Let us stop this madness. No more API extensions. Lets try get something agreed in devref, so we are ready to go when Liberty opens. It would be nice to look at ways to fold back the existing extensions into the main code. I know there are v2.0 compatibility issues there, but I think/hope thats mostly cosmetic at this point. Thanks, John __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
On 03/07/2015 07:31 PM, Jay Pipes wrote: Hi Stackers, Now that microversions have been introduced to the Nova API (meaning we can now have novaclient request, say, version 2.3 of the Nova API using the special X-OpenStack-Nova-API-Version HTTP header), is there any good reason to require API extensions at all for *new* functionality. Sergey Nikitin is currently in the process of code review for the final patch that adds server instance tagging to the Nova API: https://review.openstack.org/#/c/128940 Unfortunately, for some reason I really don't understand, Sergey is being required to create an API extension called os-server-tags in order to add the server tag functionality to the API. The patch implements the 2.4 Nova API microversion, though, as you can see from this part of the patch: https://review.openstack.org/#/c/128940/43/nova/api/openstack/compute/plugins/v3/server_tags.py What is the point of creating a new plugin/API extension for this new functionality? Why can't we just modify the nova/api/openstack/compute/server.py Controller.show() method and decorate it with a 2.4 microversion that adds a tags attribute to the returned server dictionary? https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/servers.py#L369 Because we're using an API extension for this new server tags functionality, we are instead having the extension extend the server dictionary with an os-server-tags:tags key containing the list of string tags. This is ugly and pointless. We don't need to use API extensions any more for this stuff. A client knows that server tags are supported by the 2.4 API microversion. If the client requests the 2.4+ API, then we should just include the tags attribute in the server dictionary. Similarly, new microversion API functionality should live in a module, as a top-level (or subcollection) Controller in /nova/api/openstack/compute/, and should not be in the /nova/api/openstack/compute/plugins/ directory. Why? Because it's not a plugin. Why are we continuing to use these awkward, messy, and cumbersome API extensions? Please, I am begging the Nova core team. Let us stop this madness. No more API extensions. Agreed, the current extensions list exists to explain the base v2 functionality. I think we should consider that frozen and deprecated as of v2.1 as we have a better way to express features. -Sean -- Sean Dague http://dague.net signature.asc Description: OpenPGP digital signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
ok, no problem, will take a look it tomorrow. 2015-03-09 20:18 GMT+08:00 Christopher Yeoh cbky...@gmail.com: On Mon, Mar 9, 2015 at 10:08 PM, John Garbutt j...@johngarbutt.com wrote: +1 Please could you submit a dev ref for this? We can argue on the review, a bit like this one: https://github.com/openstack/nova/blob/master/doc/source/devref/policy_enforcement.rst I think it'd also be a good idea to add a testcase (use test_plugins directory where you can define your own controller which is never plubished) for each example so they don't get out of date Regards, Chris __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
On Mar 9, 2015, at 13:14, Sean Dague s...@dague.net wrote: So possibly another way to think about this is our prior signaling of what was supported by Nova was signaled by the extension list. Our code was refactored into a way that supported optional loading by that unit. As we're making things less optional it probably makes sense to evolve the API code tree to look more like our REST resource tree. Yes, that means servers.py ends up being big, but it is less confusing that all servers related code is in that file vs all over a bunch of other files. So I'd agree that in this case server tags probably should just be in servers.py. I also think long term we should do some plugin collapse for stuff that's all really just features on one resource tree so that the local filesystem code structure looks a bit more like the REST url tree. I think this makes a lot of sense. When I read the question, why is server tags being added as an extension the answer that comes to mind first is, because the extension framework is there and that's how things have been done so far. I think the original thinking on extensions was, make everything optional so users can enable/disable as they please, operators can disable any feature by removing the extension. Another benefit is the ability for anyone to add a (non-useful to the community at-large) feature without having to patch in several places. I used to be for extensions for the aforementioned benefits, but now I tend to think it's too flexible and complex. It's so flexible that you can easily get yourself into a situation where your deployment can't work with other useful tools/libraries/etc which expect a certain contract from the Nova API. It doesn't make sense to let the API we provide be so arbitrary. It's certainly not friendly to API users. We still have the ability to disable or limit features based on policy -- I don't think we need to do it via extensions. The only problem that seems to be left is, how can we allow people to add un-upstreamable features to the API in their internal deployments? I know the ideal answer is don't do that but the reality is some things will never be agreed upon upstream and I do see value in the extension framework for that. I don't think anything in-tree should be implemented as extensions, though. melanie (melwitt) signature.asc Description: Message signed with OpenPGP using GPGMail __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
On 03/08/2015 08:10 AM, Alex Xu wrote:
Thanks for Jay point this out! If we have agreement on this and document
it, that will be great for guiding developer how to add new API.
I know we didn't want extension for API. But I think we still
need modularity. I don't think we should put everything in a single
file, that file will become huge in the future and hard to maintenance.
I don't think everything should be in a single file either. In fact,
I've never advocated for that.
We can make the 'extension' not configurable. Replace 'extension' with
another name, deprecate the extension info api int he future But
that is not mean we should put everything in a file.
I didn't say that in my email. I'm not sure where you got the impression
I want to put everything in one file?
For modularity, we need define what should be in a separated module(it
is extension now.) There are three cases:
1. Add new resource
This is totally worth to put in a separated module.
Agreed.
2. Add new sub-resource
like server-tags, I prefer to put in a separated module, I don't
think put another 100 lines code in the servers.py is good choice.
Agreed, which is exactly what I said in my email:
Similarly, new microversion API functionality should live in a
module, as a top-level (or subcollection) Controller in
/nova/api/openstack/compute/, and should not be in the
/nova/api/openstack/compute/plugins/ directory. Why? Because it's
not a plugin.
3. extend attributes and methods for a existed resource
like add new attributes for servers, we can choice one of existed
module to put it in. Just like this patch
https://review.openstack.org/#/c/155853/
But for servers-tags, it's sub-resource, we can put it in its-own
module.
Agreed, and that's what I put in my email.
If we didn't want to support extension right now, we can begin from not
show servers-tags in extension info API first. That means extension info
is freeze now. We deprecated the extension info api in later version.
I don't understand what you're saying here. Could you elaborate? What I
am asking for is for new functionality (like the server-tags
subcollection resource), just add a new module called
/nova/api/openstack/compute/server_tags.py, create a Controller object
in that file with the new server tags resource, and don't use any of the
API extensions framework whatsoever.
In addition to that, for the changes to the main GET
/servers/{server_id} resource, use microversions to decorate the
/nova/api/openstack/compute/servers.py.Controller.show() method for 2.4
and add a tags key to the dict (note: NOT a os-server-tags:tags key)
returned by GET /servers/{server_id}. No use of API extensions needed.
Best,
-jay
Thanks
Alex
2015-03-08 8:31 GMT+08:00 Jay Pipes jaypi...@gmail.com
mailto:jaypi...@gmail.com:
Hi Stackers,
Now that microversions have been introduced to the Nova API (meaning
we can now have novaclient request, say, version 2.3 of the Nova API
using the special X-OpenStack-Nova-API-Version HTTP header), is
there any good reason to require API extensions at all for *new*
functionality.
Sergey Nikitin is currently in the process of code review for the
final patch that adds server instance tagging to the Nova API:
https://review.openstack.org/#__/c/128940
https://review.openstack.org/#/c/128940
Unfortunately, for some reason I really don't understand, Sergey is
being required to create an API extension called os-server-tags in
order to add the server tag functionality to the API. The patch
implements the 2.4 Nova API microversion, though, as you can see
from this part of the patch:
https://review.openstack.org/#__/c/128940/43/nova/api/__openstack/compute/plugins/v3/__server_tags.py
https://review.openstack.org/#/c/128940/43/nova/api/openstack/compute/plugins/v3/server_tags.py
What is the point of creating a new plugin/API extension for this
new functionality? Why can't we just modify the
nova/api/openstack/compute/__server.py Controller.show() method and
decorate it with a 2.4 microversion that adds a tags attribute to
the returned server dictionary?
https://github.com/openstack/__nova/blob/master/nova/api/__openstack/compute/servers.py#__L369
https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/servers.py#L369
Because we're using an API extension for this new server tags
functionality, we are instead having the extension extend the
server dictionary with an os-server-tags:tags key containing the
list of string tags.
This is ugly and pointless. We don't need to use API extensions any
more for this stuff.
A client knows that server tags are supported by the 2.4 API
microversion. If the client requests the 2.4+ API, then we should
just include the tags attribute in the server dictionary.
Similarly, new microversion API functionality should live in
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
On 03/09/2015 07:32 AM, Christopher Yeoh wrote: snip So the first thing I think we want to distinguish between plugins being a REST API user or operator concept and it being a tool developers use as a framework to support the Nova REST API. As I've mentioned before I've no problem with the feature set of the API being fixed (per microversion) across all Nova deployments. Get back to me when we have consensus on that and its trivial to implement and we'll no longer have the concept of core andextension/plugin. Why are we continuing to add API extensions for new stuff? But plugin like implementations using Stevedore as a tool for developers to keep good modularity has proven to be very useful to keep complexity level lower and interactions between modules much clearer. Uhm, I beg to differ. The plugin implementation using Stevedore has made things way more complicated than they need to be. It would be much simpler to have a single directory call /nova/api/openstack/compute/ with modules representing each resource collection controller. servers.py is an example of this where in v2 I think we have/had the most complex method and even with all the fix up work which has been done on it it is still very complicated to understand. It doesn't *need* to be more complicated. The microversions framework decorators allow us to encapsulate the differences between microversions for a particular method. Best, -jay On Sun, Mar 8, 2015 at 11:01 AM, Jay Pipes jaypi...@gmail.com mailto:jaypi...@gmail.com wrote: Hi Stackers, Now that microversions have been introduced to the Nova API (meaning we can now have novaclient request, say, version 2.3 of the Nova API using the special X-OpenStack-Nova-API-Version HTTP header), is there any good reason to require API extensions at all for *new* functionality. Sergey Nikitin is currently in the process of code review for the final patch that adds server instance tagging to the Nova API: https://review.openstack.org/#__/c/128940 https://review.openstack.org/#/c/128940 Unfortunately, for some reason I really don't understand, Sergey is being required to create an API extension called os-server-tags in order to add the server tag functionality to the API. The patch implements the 2.4 Nova API microversion, though, as you can see from this part of the patch: https://review.openstack.org/#__/c/128940/43/nova/api/__openstack/compute/plugins/v3/__server_tags.py https://review.openstack.org/#/c/128940/43/nova/api/openstack/compute/plugins/v3/server_tags.py What is the point of creating a new plugin/API extension for this new functionality? Why can't we just modify the nova/api/openstack/compute/__server.py Controller.show() method and decorate it with a 2.4 microversion that adds a tags attribute to the returned server dictionary? Actually I think it does more than just add extra reponse information: - it adds extra tags parameter to show - it doesn't add it to index, but it probably should add the response information to detail to be consistent with the rest of the API - It adds a new resource /servers/server_id/tags - with create, delete and delete all supported. I don't think that these belong in servers.py https://github.com/openstack/__nova/blob/master/nova/api/__openstack/compute/servers.py#__L369 https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/servers.py#L369 Because we're using an API extension for this new server tags functionality, we are instead having the extension extend the server dictionary with an os-server-tags:tags key containing the list of string tags. This is ugly and pointless. We don't need to use API extensions any more for this stuff. So we had a prefix rule originally in V2 to allow for extensions and guarantee no name clashes. I'd be happy removing this requirement, even removing old ones as long as we have consensus. A client knows that server tags are supported by the 2.4 API microversion. If the client requests the 2.4+ API, then we should just include the tags attribute in the server dictionary. Similarly, new microversion API functionality should live in a module, as a top-level (or subcollection) Controller in /nova/api/openstack/compute/, and should not be in the /nova/api/openstack/compute/__plugins/ directory. Why? Because it's not a plugin. So I don't see how that changes whether we're using plugins (from a user point of view) or not. The good news for you is that there is fixing the shambles of a directory structure for the api is on the list of things to do, it just wasn't a high prioirty things for us in Kilo, get v2.1 and microversions out. For example, we have v3 in the directory path as well for historical reasons and we also have a contrib directory in compute and none of those are really contrib now either. Now the
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
On 03/09/2015 03:37 PM, Jay Pipes wrote:
On 03/08/2015 08:10 AM, Alex Xu wrote:
Thanks for Jay point this out! If we have agreement on this and document
it, that will be great for guiding developer how to add new API.
I know we didn't want extension for API. But I think we still
need modularity. I don't think we should put everything in a single
file, that file will become huge in the future and hard to maintenance.
I don't think everything should be in a single file either. In fact,
I've never advocated for that.
We can make the 'extension' not configurable. Replace 'extension' with
another name, deprecate the extension info api int he future But
that is not mean we should put everything in a file.
I didn't say that in my email. I'm not sure where you got the impression
I want to put everything in one file?
For modularity, we need define what should be in a separated module(it
is extension now.) There are three cases:
1. Add new resource
This is totally worth to put in a separated module.
Agreed.
2. Add new sub-resource
like server-tags, I prefer to put in a separated module, I don't
think put another 100 lines code in the servers.py is good choice.
Agreed, which is exactly what I said in my email:
Similarly, new microversion API functionality should live in a
module, as a top-level (or subcollection) Controller in
/nova/api/openstack/compute/, and should not be in the
/nova/api/openstack/compute/plugins/ directory. Why? Because it's
not a plugin.
3. extend attributes and methods for a existed resource
like add new attributes for servers, we can choice one of existed
module to put it in. Just like this patch
https://review.openstack.org/#/c/155853/
But for servers-tags, it's sub-resource, we can put it in its-own
module.
Agreed, and that's what I put in my email.
If we didn't want to support extension right now, we can begin from not
show servers-tags in extension info API first. That means extension info
is freeze now. We deprecated the extension info api in later version.
I don't understand what you're saying here. Could you elaborate? What I
am asking for is for new functionality (like the server-tags
subcollection resource), just add a new module called
/nova/api/openstack/compute/server_tags.py, create a Controller object
in that file with the new server tags resource, and don't use any of the
API extensions framework whatsoever.
In addition to that, for the changes to the main GET
/servers/{server_id} resource, use microversions to decorate the
/nova/api/openstack/compute/servers.py.Controller.show() method for 2.4
and add a tags key to the dict (note: NOT a os-server-tags:tags key)
returned by GET /servers/{server_id}. No use of API extensions needed.
So possibly another way to think about this is our prior signaling of
what was supported by Nova was signaled by the extension list. Our code
was refactored into a way that supported optional loading by that unit.
As we're making things less optional it probably makes sense to evolve
the API code tree to look more like our REST resource tree. Yes, that
means servers.py ends up being big, but it is less confusing that all
servers related code is in that file vs all over a bunch of other files.
So I'd agree that in this case server tags probably should just be in
servers.py. I also think long term we should do some plugin collapse
for stuff that's all really just features on one resource tree so that
the local filesystem code structure looks a bit more like the REST url tree.
-Sean
--
Sean Dague
http://dague.net
signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
2015-03-09 20:36 GMT+09:00 Christopher Yeoh cbky...@gmail.com: On Mon, Mar 9, 2015 at 9:35 PM, Sean Dague s...@dague.net wrote: On 03/07/2015 07:31 PM, Jay Pipes wrote: Hi Stackers, Now that microversions have been introduced to the Nova API (meaning we can now have novaclient request, say, version 2.3 of the Nova API using the special X-OpenStack-Nova-API-Version HTTP header), is there any good reason to require API extensions at all for *new* functionality. Sergey Nikitin is currently in the process of code review for the final patch that adds server instance tagging to the Nova API: https://review.openstack.org/#/c/128940 Unfortunately, for some reason I really don't understand, Sergey is being required to create an API extension called os-server-tags in order to add the server tag functionality to the API. The patch implements the 2.4 Nova API microversion, though, as you can see from this part of the patch: https://review.openstack.org/#/c/128940/43/nova/api/openstack/compute/plugins/v3/server_tags.py What is the point of creating a new plugin/API extension for this new functionality? Why can't we just modify the nova/api/openstack/compute/server.py Controller.show() method and decorate it with a 2.4 microversion that adds a tags attribute to the returned server dictionary? https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/servers.py#L369 Because we're using an API extension for this new server tags functionality, we are instead having the extension extend the server dictionary with an os-server-tags:tags key containing the list of string tags. This is ugly and pointless. We don't need to use API extensions any more for this stuff. A client knows that server tags are supported by the 2.4 API microversion. If the client requests the 2.4+ API, then we should just include the tags attribute in the server dictionary. Similarly, new microversion API functionality should live in a module, as a top-level (or subcollection) Controller in /nova/api/openstack/compute/, and should not be in the /nova/api/openstack/compute/plugins/ directory. Why? Because it's not a plugin. Why are we continuing to use these awkward, messy, and cumbersome API extensions? Please, I am begging the Nova core team. Let us stop this madness. No more API extensions. Agreed, the current extensions list exists to explain the base v2 functionality. I think we should consider that frozen and deprecated as of v2.1 as we have a better way to express features. -Sean So I think we can a microversion ASAP to remove support for /extensions. Obviously we'll need to keep the actual code to support v2.1 for quite a while though. big +1 for removing /extensions. Now /extensions API provides available futures on each cloud services, but provided information is difficult to use. In addition, there is a ugly trick[1] in the implementation for providing v2 full compatible information. I think we still want some fields in the controller like we do because we want to automate JSON-HOME/Schema stuff (maybe not sure) yeah, JSON-Home is one of standard ways for providing this kind of information. The nova-spec is https://review.openstack.org/#/c/130715/ I hope it will be implement in Liberty. Thanks Ken Ohmichi --- [1]: https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/plugins/v3/extension_info.py#L29 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
Hi, 2015-03-09 20:38 GMT+09:00 John Garbutt j...@johngarbutt.com: On 8 March 2015 at 12:10, Alex Xu sou...@gmail.com wrote: Thanks for Jay point this out! If we have agreement on this and document it, that will be great for guiding developer how to add new API. +1 Please could you submit a dev ref for this? We can argue on the review, a bit like this one: https://github.com/openstack/nova/blob/master/doc/source/devref/policy_enforcement.rst For modularity, we need define what should be in a separated module(it is extension now.) There are three cases: 1. Add new resource This is totally worth to put in a separated module. +1 yeah, +1 for simple/readable code. 2. Add new sub-resource like server-tags, I prefer to put in a separated module, I don't think put another 100 lines code in the servers.py is good choice. -1 I hate the idea of show instance extension code for version 2.4 living separately to the rest of the instance show logic, when it really doesn't have to. It feels too heavyweight in its current form. Maybe we need a more modular way of expressing the extension within the same file? That would depend on how size of a file. I'd like to avoid a single huge file for reviewing a patch on firefox. Sometimes, loading time is long due to the size(especially nova/compute/api.py). What is the point of creating a new plugin/API extension for this new functionality? Why can't we just modify the nova/api/openstack/compute/server.py Controller.show() method and decorate it with a 2.4 microversion that adds a tags attribute to the returned server dictionary? Similarly, new microversion API functionality should live in a module, as a top-level (or subcollection) Controller in /nova/api/openstack/compute/, and should not be in the /nova/api/openstack/compute/plugins/ directory. Why? Because it's not a plugin. Everything is a plugin in v3, no more distinction between core vs plugin. It needs renaming really. +1, but we need to remove current v2(not v2.1) code before doing that. To reduce cost for maintaining both v2 and v2.1 code, we will use v2.1 code only in long term and v2 code will be removed. Now v2 code lives under /nova/api/openstack/compute and /nova/api/openstack/compute/contrib, and v2.1 code does under /nova/api/openstack/compute/plugins/v3. So current directory structure would be easy to remove v2 code in long term. Why are we continuing to use these awkward, messy, and cumbersome API extensions? We certainly should never be forced to add an extension to advertise new functionality anymore. Its a big reason why I want to see the API micro-versions succeed. +1 I also don't want to add dummy extensions just for advertising a new functionalities more. When I saw the code first time, I was confused due to them. Thanks Ken Ohmichi __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
I agree with Jay. The extension layer is also expensive in CPU usage, and it also makes more difficult to troubleshoot issues. - Original Message - From: Jay Pipes jaypi...@gmail.com To: OpenStack Development Mailing List openstack-dev@lists.openstack.org, Sergey Nikitin sniki...@mirantis.com Sent: Sunday, March 8, 2015 1:31:34 AM Subject: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality? Hi Stackers, Now that microversions have been introduced to the Nova API (meaning we can now have novaclient request, say, version 2.3 of the Nova API using the special X-OpenStack-Nova-API-Version HTTP header), is there any good reason to require API extensions at all for *new* functionality. Sergey Nikitin is currently in the process of code review for the final patch that adds server instance tagging to the Nova API: https://review.openstack.org/#/c/128940 Unfortunately, for some reason I really don't understand, Sergey is being required to create an API extension called os-server-tags in order to add the server tag functionality to the API. The patch implements the 2.4 Nova API microversion, though, as you can see from this part of the patch: https://review.openstack.org/#/c/128940/43/nova/api/openstack/compute/plugins/v3/server_tags.py What is the point of creating a new plugin/API extension for this new functionality? Why can't we just modify the nova/api/openstack/compute/server.py Controller.show() method and decorate it with a 2.4 microversion that adds a tags attribute to the returned server dictionary? https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/servers.py#L369 Because we're using an API extension for this new server tags functionality, we are instead having the extension extend the server dictionary with an os-server-tags:tags key containing the list of string tags. This is ugly and pointless. We don't need to use API extensions any more for this stuff. A client knows that server tags are supported by the 2.4 API microversion. If the client requests the 2.4+ API, then we should just include the tags attribute in the server dictionary. Similarly, new microversion API functionality should live in a module, as a top-level (or subcollection) Controller in /nova/api/openstack/compute/, and should not be in the /nova/api/openstack/compute/plugins/ directory. Why? Because it's not a plugin. Why are we continuing to use these awkward, messy, and cumbersome API extensions? Please, I am begging the Nova core team. Let us stop this madness. No more API extensions. Best, -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
Thanks for Jay point this out! If we have agreement on this and document it, that will be great for guiding developer how to add new API. I know we didn't want extension for API. But I think we still need modularity. I don't think we should put everything in a single file, that file will become huge in the future and hard to maintenance. We can make the 'extension' not configurable. Replace 'extension' with another name, deprecate the extension info api int he future But that is not mean we should put everything in a file. For modularity, we need define what should be in a separated module(it is extension now.) There are three cases: 1. Add new resource This is totally worth to put in a separated module. 2. Add new sub-resource like server-tags, I prefer to put in a separated module, I don't think put another 100 lines code in the servers.py is good choice. 3. extend attributes and methods for a existed resource like add new attributes for servers, we can choice one of existed module to put it in. Just like this patch https://review.openstack.org/#/c/155853/ But for servers-tags, it's sub-resource, we can put it in its-own module. If we didn't want to support extension right now, we can begin from not show servers-tags in extension info API first. That means extension info is freeze now. We deprecated the extension info api in later version. Thanks Alex 2015-03-08 8:31 GMT+08:00 Jay Pipes jaypi...@gmail.com: Hi Stackers, Now that microversions have been introduced to the Nova API (meaning we can now have novaclient request, say, version 2.3 of the Nova API using the special X-OpenStack-Nova-API-Version HTTP header), is there any good reason to require API extensions at all for *new* functionality. Sergey Nikitin is currently in the process of code review for the final patch that adds server instance tagging to the Nova API: https://review.openstack.org/#/c/128940 Unfortunately, for some reason I really don't understand, Sergey is being required to create an API extension called os-server-tags in order to add the server tag functionality to the API. The patch implements the 2.4 Nova API microversion, though, as you can see from this part of the patch: https://review.openstack.org/#/c/128940/43/nova/api/ openstack/compute/plugins/v3/server_tags.py What is the point of creating a new plugin/API extension for this new functionality? Why can't we just modify the nova/api/openstack/compute/server.py Controller.show() method and decorate it with a 2.4 microversion that adds a tags attribute to the returned server dictionary? https://github.com/openstack/nova/blob/master/nova/api/ openstack/compute/servers.py#L369 Because we're using an API extension for this new server tags functionality, we are instead having the extension extend the server dictionary with an os-server-tags:tags key containing the list of string tags. This is ugly and pointless. We don't need to use API extensions any more for this stuff. A client knows that server tags are supported by the 2.4 API microversion. If the client requests the 2.4+ API, then we should just include the tags attribute in the server dictionary. Similarly, new microversion API functionality should live in a module, as a top-level (or subcollection) Controller in /nova/api/openstack/compute/, and should not be in the /nova/api/openstack/compute/plugins/ directory. Why? Because it's not a plugin. Why are we continuing to use these awkward, messy, and cumbersome API extensions? Please, I am begging the Nova core team. Let us stop this madness. No more API extensions. Best, -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?
Hi Stackers, Now that microversions have been introduced to the Nova API (meaning we can now have novaclient request, say, version 2.3 of the Nova API using the special X-OpenStack-Nova-API-Version HTTP header), is there any good reason to require API extensions at all for *new* functionality. Sergey Nikitin is currently in the process of code review for the final patch that adds server instance tagging to the Nova API: https://review.openstack.org/#/c/128940 Unfortunately, for some reason I really don't understand, Sergey is being required to create an API extension called os-server-tags in order to add the server tag functionality to the API. The patch implements the 2.4 Nova API microversion, though, as you can see from this part of the patch: https://review.openstack.org/#/c/128940/43/nova/api/openstack/compute/plugins/v3/server_tags.py What is the point of creating a new plugin/API extension for this new functionality? Why can't we just modify the nova/api/openstack/compute/server.py Controller.show() method and decorate it with a 2.4 microversion that adds a tags attribute to the returned server dictionary? https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/servers.py#L369 Because we're using an API extension for this new server tags functionality, we are instead having the extension extend the server dictionary with an os-server-tags:tags key containing the list of string tags. This is ugly and pointless. We don't need to use API extensions any more for this stuff. A client knows that server tags are supported by the 2.4 API microversion. If the client requests the 2.4+ API, then we should just include the tags attribute in the server dictionary. Similarly, new microversion API functionality should live in a module, as a top-level (or subcollection) Controller in /nova/api/openstack/compute/, and should not be in the /nova/api/openstack/compute/plugins/ directory. Why? Because it's not a plugin. Why are we continuing to use these awkward, messy, and cumbersome API extensions? Please, I am begging the Nova core team. Let us stop this madness. No more API extensions. Best, -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
