Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Ken'ichi Ohmichi]

2015-03-24 23:20 GMT+09:00 Joe Gordon :
> On Tue, Mar 24, 2015 at 7:10 AM, Jay Pipes  wrote:
>> On Tue, Mar 24, 2015 at 01:04:46PM +, Jeremy Stanley wrote:
>> > On 2015-03-23 21:31:30 -0400 (-0400), Jay Pipes wrote:
>> > > On Mon, Mar 23, 2015 at 09:35:50PM +, Jeremy Stanley wrote:
>> > > > On 2015-03-23 15:15:18 -0400 (-0400), Jay Pipes wrote:
>> > > > [...]
>> > > > > I don't want it suppressed. I want the use of API extensions and
>> > > > > the
>> > > > > extension framework(s) to be completely dropped for all future
>> > > > > API-affecting work.
>> > > > [...]
>> > > >
>> > > > Perhaps controversial, but would it be worthwhile to propose to the
>> > > > Defcore working group that future compliance requirements include
>> > > > the absence of extensions to officially covered APIs?
>> > >
>> > > I don't understand what you're getting at, Jeremy. Could you
>> > > elaborate?
>> > >
>> > > What do extensions have to do with future compliance requirements?
>> >
>> > Defcore's focus is on establishing interoperability standards for
>> > OpenStack deployments, to ease the end-user experience. Right now
>> > its model depends on a whitelist of API features. As discussed many
>> > times before and brought up again in this thread, when providers or
>> > distributors "augment" OpenStack APIs to add their own special
>> > features without implementing them upstream, this necessarily
>> > creates interoperability issues.
>>
>> Defcore's focus is on determining what "is OpenStack", w.r.t. what is
>> brandable as OpenStack. It's focus is not on establishing interoperability
>> standards.
>>
>
> I am not sure how you got to that conclusion, yes the defcore process has
> been very confusing and I am still not really sure what it was, but some
> part of it it *is* about interoperability/
>
> Although our wiki does get out of date very easily, I think this still holds
> true:
>
> DefCore sets base requirements by defining 1) capabilities, 2) code and 3)
> must-pass tests for all OpenStack products. This definition uses community
> resources and involvement to drive interoperability by creating the minimum
> standards for products labeled "OpenStack."
>
> https://wiki.openstack.org/wiki/Governance/DefCoreCommittee

Related to this topic, Nova v2.1 API defines core APIs by itself[1],
but I feel now it is better to remove the definition from Nova.
On current implementation, the boot of nova-api fails if the above
core APIs are not loaded.
but that behavior seems conflict to Defcore process, and it would be
nice to concentrate on Defcore to define what are core APIs.

Thanks
Ken Ohmichi
---
[1]: 
https://github.com/openstack/nova/blob/master/nova/api/openstack/__init__.py#L66

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Rochelle Grober]

Jeremy Stanley on March 24, 2015 07:28 wrote:
On 2015-03-24 10:10:07 -0400 (-0400), Jay Pipes wrote:
[...]
> I'm really not a fan of the Defcore effort. This should come as no
> surprise to anyone. I've been quite blunt about my disdain for the
> focus on identifying which API things are mandatory and which are
> optional, in order to say some vendor's product "is OpenStack".
[...]

Understandable--and I'm certainly no fan of bureaucracy and
politicking--just pointing out that in the carrots-and-sticks bin we
have trademarks as a potential carrot to encourage people running
OpenStack commercially to further the goals of our community. If
there is broad community support for the idea that standard
OpenStack APIs should not be arbitrarily extended downstream because
it hurts interoperability, then this might be one way to improve the
situation. If there are other means to achieve this then they too
should be considered, obviously.

[Rockyg] Tl;dr.  Propose it to DefCore.  They might agree.  But they might have 
action items for developers to make it actionable on DefCore's part.

Hey, it sounds like a reasonable request to me, and it's a large operator 
making the request for DefCore to recognize that extensions break 
interoperability.  Plus, Defcore specifically states that vendor specific code 
within OpenStack is not considered for inclusion in the Defcore 
capabilities/test/designated code sets.

So then the questions become: 

* Are there any currently included capabilities that assume extensions work?
* Are there any likely candidates for future Defcore sets that assume 
extensions work?
* Will extensions be deprecated before capabilities requiring extensions are in 
the candidate pool?
* Once extensions are deprecated, or it is known that nothing in Defcore sets 
include the ability to use extensions, are there tests which will validate that 
extensions are not part of the cloud under test?

Something to think about, and if you are serious about limiting the negative 
effects of extensions, it is certainly worth proposing their elimination and/or 
restrictions on them to Defcore.

And an FYI, APIs and tests are not chosen by DefCore because they are 
meaningful, but because they are measurable.  DefCore uses User Surveys (and 
who knows what else) to determine adoption rates of OpenStack "capabilities" 
that are then converted to APIs and tests, etc.  If you have a implementable 
ideas on how better to measure individual clouds/cloud products against the 
user communities' utilization, they'd love to hear from you.  No, really. 
DefCore wants to hear of better ways to ensure that clouds with the OpenStack 
trademark mean that user implementations on those clouds are portable across 
compliant vendors within the scope of what those vendors advertise. 

--rocky

-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Jeremy Stanley]

On 2015-03-24 10:10:07 -0400 (-0400), Jay Pipes wrote:
[...]
> I'm really not a fan of the Defcore effort. This should come as no
> surprise to anyone. I've been quite blunt about my disdain for the
> focus on identifying which API things are mandatory and which are
> optional, in order to say some vendor's product "is OpenStack".
[...]

Understandable--and I'm certainly no fan of bureaucracy and
politicking--just pointing out that in the carrots-and-sticks bin we
have trademarks as a potential carrot to encourage people running
OpenStack commercially to further the goals of our community. If
there is broad community support for the idea that standard
OpenStack APIs should not be arbitrarily extended downstream because
it hurts interoperability, then this might be one way to improve the
situation. If there are other means to achieve this then they too
should be considered, obviously.
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Joe Gordon]

On Tue, Mar 24, 2015 at 7:10 AM, Jay Pipes  wrote:

> On Tue, Mar 24, 2015 at 01:04:46PM +, Jeremy Stanley wrote:
> > On 2015-03-23 21:31:30 -0400 (-0400), Jay Pipes wrote:
> > > On Mon, Mar 23, 2015 at 09:35:50PM +, Jeremy Stanley wrote:
> > > > On 2015-03-23 15:15:18 -0400 (-0400), Jay Pipes wrote:
> > > > [...]
> > > > > I don't want it suppressed. I want the use of API extensions and
> the
> > > > > extension framework(s) to be completely dropped for all future
> > > > > API-affecting work.
> > > > [...]
> > > >
> > > > Perhaps controversial, but would it be worthwhile to propose to the
> > > > Defcore working group that future compliance requirements include
> > > > the absence of extensions to officially covered APIs?
> > >
> > > I don't understand what you're getting at, Jeremy. Could you elaborate?
> > >
> > > What do extensions have to do with future compliance requirements?
> >
> > Defcore's focus is on establishing interoperability standards for
> > OpenStack deployments, to ease the end-user experience. Right now
> > its model depends on a whitelist of API features. As discussed many
> > times before and brought up again in this thread, when providers or
> > distributors "augment" OpenStack APIs to add their own special
> > features without implementing them upstream, this necessarily
> > creates interoperability issues.
>
> Defcore's focus is on determining what "is OpenStack", w.r.t. what is
> brandable as OpenStack. It's focus is not on establishing interoperability
> standards.
>
>
I am not sure how you got to that conclusion, yes the defcore process has
been very confusing and I am still not really sure what it was, but some
part of it it *is* about interoperability/

Although our wiki does get out of date very easily, I think this still
holds true:

*DefCore sets base requirements by defining 1) capabilities, 2) code and 3)
must-pass tests for all OpenStack products. This definition uses community
resources and involvement to drive interoperability by creating the minimum
standards for products labeled "OpenStack."*

*https://wiki.openstack.org/wiki/Governance/DefCoreCommittee
*


Here is another source:

"DefCore has a single goal expressed from two sides: 1) defining the “what
is OpenStack” brand for Vendors and 2) driving interoperability between
OpenStack installations. "

http://robhirschfeld.com/2015/02/26/openstack-defcore-accelerates-simplifies-with-clear-and-timely-guidelines-feedback/


> > What I'm suggesting is that Defcore should not just specify which
> > API features need to be supported, but also specify that nonstandard
> > API features must not be added to the APIs its covering.
>
> We're perilously close to re-igniting the "is OpenStack a set of APIs,
> or is OpenStack a set of implementations of those APIs" debate. A debate
> I'm happy to have, of course :)
>
> I'm really not a fan of the Defcore effort. This should come as no
> surprise to anyone. I've been quite blunt about my disdain for the focus
> on identifying which API things are mandatory and which are optional, in
> order to say some vendor's product "is OpenStack".
>
> That said, I'm not going to get in its way. If you think that Defcore
> should amend its compliancy guidelines to include the above, fine by me.
>
> Best,
> -jay
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Jay Pipes]

On Tue, Mar 24, 2015 at 01:04:46PM +, Jeremy Stanley wrote:
> On 2015-03-23 21:31:30 -0400 (-0400), Jay Pipes wrote:
> > On Mon, Mar 23, 2015 at 09:35:50PM +, Jeremy Stanley wrote:
> > > On 2015-03-23 15:15:18 -0400 (-0400), Jay Pipes wrote:
> > > [...]
> > > > I don't want it suppressed. I want the use of API extensions and the
> > > > extension framework(s) to be completely dropped for all future
> > > > API-affecting work.
> > > [...]
> > > 
> > > Perhaps controversial, but would it be worthwhile to propose to the
> > > Defcore working group that future compliance requirements include
> > > the absence of extensions to officially covered APIs?
> > 
> > I don't understand what you're getting at, Jeremy. Could you elaborate?
> > 
> > What do extensions have to do with future compliance requirements?
> 
> Defcore's focus is on establishing interoperability standards for
> OpenStack deployments, to ease the end-user experience. Right now
> its model depends on a whitelist of API features. As discussed many
> times before and brought up again in this thread, when providers or
> distributors "augment" OpenStack APIs to add their own special
> features without implementing them upstream, this necessarily
> creates interoperability issues.

Defcore's focus is on determining what "is OpenStack", w.r.t. what is
brandable as OpenStack. It's focus is not on establishing interoperability
standards.

> What I'm suggesting is that Defcore should not just specify which
> API features need to be supported, but also specify that nonstandard
> API features must not be added to the APIs its covering.

We're perilously close to re-igniting the "is OpenStack a set of APIs,
or is OpenStack a set of implementations of those APIs" debate. A debate
I'm happy to have, of course :)

I'm really not a fan of the Defcore effort. This should come as no
surprise to anyone. I've been quite blunt about my disdain for the focus
on identifying which API things are mandatory and which are optional, in
order to say some vendor's product "is OpenStack".

That said, I'm not going to get in its way. If you think that Defcore
should amend its compliancy guidelines to include the above, fine by me.

Best,
-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Jeremy Stanley]

On 2015-03-23 21:31:30 -0400 (-0400), Jay Pipes wrote:
> On Mon, Mar 23, 2015 at 09:35:50PM +, Jeremy Stanley wrote:
> > On 2015-03-23 15:15:18 -0400 (-0400), Jay Pipes wrote:
> > [...]
> > > I don't want it suppressed. I want the use of API extensions and the
> > > extension framework(s) to be completely dropped for all future
> > > API-affecting work.
> > [...]
> > 
> > Perhaps controversial, but would it be worthwhile to propose to the
> > Defcore working group that future compliance requirements include
> > the absence of extensions to officially covered APIs?
> 
> I don't understand what you're getting at, Jeremy. Could you elaborate?
> 
> What do extensions have to do with future compliance requirements?

Defcore's focus is on establishing interoperability standards for
OpenStack deployments, to ease the end-user experience. Right now
its model depends on a whitelist of API features. As discussed many
times before and brought up again in this thread, when providers or
distributors "augment" OpenStack APIs to add their own special
features without implementing them upstream, this necessarily
creates interoperability issues.

What I'm suggesting is that Defcore should not just specify which
API features need to be supported, but also specify that nonstandard
API features must not be added to the APIs its covering.
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Christopher Yeoh]

On Tue, Mar 24, 2015 at 5:45 AM, Jay Pipes  wrote:

> On Tue, Mar 10, 2015 at 09:52:14PM +1030, Christopher Yeoh wrote:
> > On Mon, 09 Mar 2015 16:14:21 -0400
> > Sean Dague  wrote:
> >
> > > On 03/09/2015 03:37 PM, Jay Pipes wrote:
> > > > On 03/08/2015 08:10 AM, Alex Xu wrote:
> > > >> Thanks for Jay point this out! If we have agreement on this and
> > > >> document it, that will be great for guiding developer how to add
> > > >> new API.
> > > >>
> > > >> I know we didn't want extension for API. But I think we still
> > > >> need modularity. I don't think we should put everything in a single
> > > >> file, that file will become huge in the future and hard to
> > > >> maintenance.
> > > >
> > > > I don't think everything should be in a single file either. In fact,
> > > > I've never advocated for that.
> > > >
> > > >> We can make the 'extension' not configurable. Replace 'extension'
> > > >> with another name, deprecate the extension info api int he
> > > >> future But that is not mean we should put everything in a file.
> > > >
> > > > I didn't say that in my email. I'm not sure where you got the
> > > > impression I want to put everything in one file?
> > > >
> > > >> For modularity, we need define what should be in a separated
> > > >> module(it is extension now.) There are three cases:
> > > >>
> > > >> 1. Add new resource
> > > >>  This is totally worth to put in a separated module.
> > > >
> > > > Agreed.
> > > >
> > > >> 2. Add new sub-resource
> > > >>  like server-tags, I prefer to put in a separated module, I
> > > >> don't think put another 100 lines code in the servers.py is good
> > > >> choice.
> > > >
> > > > Agreed, which is exactly what I said in my email:
> > > >
> > > > "Similarly, new microversion API functionality should live in a
> > > > module, as a top-level (or subcollection) Controller in
> > > > /nova/api/openstack/compute/, and should not be in the
> > > > /nova/api/openstack/compute/plugins/ directory. Why? Because it's
> > > > not a plugin."
> > > >
> >
> > Ok so I'm getting confused about what we're disagreeing about then.
>
> I wasn't disagreeing with you. I was disagreeing with Alex Xu :) Perhaps
> your mail client was confusing you?
>
> > However, the first microversion change
> > https://review.openstack.org/#/c/140313/32 is one case where we didn't
> > need to create a new extension, relying only on microversions to add a
> > new parameter to the response, whereas server tags does add a new
> > conroller (os-server-tags) which is non trivia so I think it does. Do
> > we disagree about that?
>
> The server tags patch does add more functionality to the API, for sure.
> But, as I've written about in this thread, there's no reason at all that
> it needed to use the extension framework. It could better have been
> written as a simple Controller object in a new file in
> /nova/api/openstack/compute/server_tags.py, with no use of the
> extension mechanism whatsoever.
>
> > btw in a situation now where (I think) we are saying we are not going
> > to do any work for 3rd parties to add their own API plugins and have a
> > well planned API we don't need the prefixes as we don't need the prefix
> > on parameter names as there won't be name clashes without us realising
> > during testing. And we certainly don't need the os- prefix is the
> > plugin alias, but we do neeed them unique across the API I believe
> > because of the way we store information about them.
>
> Right. There won't be any more name clashes if we don't allow API
> extensions any more. Which is what I'm asking for.
>
> > > >> 3. extend attributes and methods for a existed resource
> > > >> like add new attributes for servers, we can choice one of
> > > >> existed module to put it in. Just like this patch
> > > >> https://review.openstack.org/#/c/155853/
> > > >> But for servers-tags, it's sub-resource, we can put it in
> > > >> its-own module.
> > > >
> > > > Agreed, and that's what I put in my email.
> > > >
> > > >> If we didn't want to support extension right now, we can begin
> > > >> from not show servers-tags in extension info API first. That means
> > > >> extension info is freeze now. We deprecated the extension info api
> > > >> in later version.
> >
> > It can be surpressed from /extensions by adding it to the suppress list
> > in the extensions code. Probably a good idea to stop v2.1+microversion
> > code REST API users not accidentally relying on it.
>
> I don't want it suppressed. I want the use of API extensions and the
> extension framework(s) to be completely dropped for all future
> API-affecting work.
>
> > > > I don't understand what you're saying here. Could you elaborate?
> > > > What I am asking for is for new functionality (like the server-tags
> > > > subcollection resource), just add a new module called
> > > > /nova/api/openstack/compute/server_tags.py, create a Controller
> > > > object in that file with the new server tags resource, and don't
> > > > use any of the 

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Jay Pipes]

On Mon, Mar 23, 2015 at 09:35:50PM +, Jeremy Stanley wrote:
> On 2015-03-23 15:15:18 -0400 (-0400), Jay Pipes wrote:
> [...]
> > I don't want it suppressed. I want the use of API extensions and the
> > extension framework(s) to be completely dropped for all future
> > API-affecting work.
> [...]
> 
> Perhaps controversial, but would it be worthwhile to propose to the
> Defcore working group that future compliance requirements include
> the absence of extensions to officially covered APIs?

I don't understand what you're getting at, Jeremy. Could you elaborate?

What do extensions have to do with future compliance requirements?

Best,
-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Joe Gordon]

On Mon, Mar 23, 2015 at 2:35 PM, Jeremy Stanley  wrote:

> On 2015-03-23 15:15:18 -0400 (-0400), Jay Pipes wrote:
> [...]
> > I don't want it suppressed. I want the use of API extensions and the
> > extension framework(s) to be completely dropped for all future
> > API-affecting work.
> [...]
>
> Perhaps controversial, but would it be worthwhile to propose to the
> Defcore working group that future compliance requirements include
> the absence of extensions to officially covered APIs?
>

++


> --
> Jeremy Stanley
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Jeremy Stanley]

On 2015-03-23 15:15:18 -0400 (-0400), Jay Pipes wrote:
[...]
> I don't want it suppressed. I want the use of API extensions and the
> extension framework(s) to be completely dropped for all future
> API-affecting work.
[...]

Perhaps controversial, but would it be worthwhile to propose to the
Defcore working group that future compliance requirements include
the absence of extensions to officially covered APIs?
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Jay Pipes]

On Tue, Mar 10, 2015 at 09:52:14PM +1030, Christopher Yeoh wrote:
> On Mon, 09 Mar 2015 16:14:21 -0400
> Sean Dague  wrote:
> 
> > On 03/09/2015 03:37 PM, Jay Pipes wrote:
> > > On 03/08/2015 08:10 AM, Alex Xu wrote:
> > >> Thanks for Jay point this out! If we have agreement on this and
> > >> document it, that will be great for guiding developer how to add
> > >> new API.
> > >>
> > >> I know we didn't want extension for API. But I think we still
> > >> need modularity. I don't think we should put everything in a single
> > >> file, that file will become huge in the future and hard to
> > >> maintenance.
> > > 
> > > I don't think everything should be in a single file either. In fact,
> > > I've never advocated for that.
> > > 
> > >> We can make the 'extension' not configurable. Replace 'extension'
> > >> with another name, deprecate the extension info api int he
> > >> future But that is not mean we should put everything in a file.
> > > 
> > > I didn't say that in my email. I'm not sure where you got the
> > > impression I want to put everything in one file?
> > > 
> > >> For modularity, we need define what should be in a separated
> > >> module(it is extension now.) There are three cases:
> > >>
> > >> 1. Add new resource
> > >>  This is totally worth to put in a separated module.
> > > 
> > > Agreed.
> > > 
> > >> 2. Add new sub-resource
> > >>  like server-tags, I prefer to put in a separated module, I
> > >> don't think put another 100 lines code in the servers.py is good
> > >> choice.
> > > 
> > > Agreed, which is exactly what I said in my email:
> > > 
> > > "Similarly, new microversion API functionality should live in a
> > > module, as a top-level (or subcollection) Controller in
> > > /nova/api/openstack/compute/, and should not be in the
> > > /nova/api/openstack/compute/plugins/ directory. Why? Because it's
> > > not a plugin."
> > > 
> 
> Ok so I'm getting confused about what we're disagreeing about then.

I wasn't disagreeing with you. I was disagreeing with Alex Xu :) Perhaps
your mail client was confusing you?

> However, the first microversion change
> https://review.openstack.org/#/c/140313/32 is one case where we didn't
> need to create a new extension, relying only on microversions to add a
> new parameter to the response, whereas server tags does add a new
> conroller (os-server-tags) which is non trivia so I think it does. Do
> we disagree about that?

The server tags patch does add more functionality to the API, for sure.
But, as I've written about in this thread, there's no reason at all that
it needed to use the extension framework. It could better have been
written as a simple Controller object in a new file in
/nova/api/openstack/compute/server_tags.py, with no use of the
extension mechanism whatsoever.

> btw in a situation now where (I think) we are saying we are not going
> to do any work for 3rd parties to add their own API plugins and have a
> well planned API we don't need the prefixes as we don't need the prefix
> on parameter names as there won't be name clashes without us realising
> during testing. And we certainly don't need the os- prefix is the
> plugin alias, but we do neeed them unique across the API I believe
> because of the way we store information about them.

Right. There won't be any more name clashes if we don't allow API
extensions any more. Which is what I'm asking for.
 
> > >> 3. extend attributes and methods for a existed resource
> > >> like add new attributes for servers, we can choice one of
> > >> existed module to put it in. Just like this patch
> > >> https://review.openstack.org/#/c/155853/
> > >> But for servers-tags, it's sub-resource, we can put it in
> > >> its-own module.
> > > 
> > > Agreed, and that's what I put in my email.
> > > 
> > >> If we didn't want to support extension right now, we can begin
> > >> from not show servers-tags in extension info API first. That means
> > >> extension info is freeze now. We deprecated the extension info api
> > >> in later version.
> 
> It can be surpressed from /extensions by adding it to the suppress list
> in the extensions code. Probably a good idea to stop v2.1+microversion
> code REST API users not accidentally relying on it.

I don't want it suppressed. I want the use of API extensions and the
extension framework(s) to be completely dropped for all future
API-affecting work.

> > > I don't understand what you're saying here. Could you elaborate?
> > > What I am asking for is for new functionality (like the server-tags
> > > subcollection resource), just add a new module called
> > > /nova/api/openstack/compute/server_tags.py, create a Controller
> > > object in that file with the new server tags resource, and don't
> > > use any of the API extensions framework whatsoever.
> > > 
> > > In addition to that, for the changes to the main GET
> > > /servers/{server_id} resource, use microversions to decorate the
> > > /nova/api/openstack/compute/servers.py.Controller.show() me

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Jay Pipes]

Sorry for the delay in responding all. Comments inline.

On Mon, Mar 09, 2015 at 03:04:59PM -0700, melanie witt wrote:
> On Mar 9, 2015, at 13:14, Sean Dague  wrote:
> > So possibly another way to think about this is our prior signaling of
> > what was supported by Nova was signaled by the extension list. Our code
> > was refactored into a way that supported optional loading by that unit.
> > 
> > As we're making things less optional it probably makes sense to evolve
> > the API code tree to look more like our REST resource tree. Yes, that
> > means servers.py ends up being big, but it is less confusing that all
> > servers related code is in that file vs all over a bunch of other files.
> > 
> > So I'd agree that in this case server tags probably should just be in
> > servers.py. I also think long term we should do some "plugin collapse"
> > for stuff that's all really just features on one resource tree so that
> > the local filesystem code structure looks a bit more like the REST url tree.
> 
> I think this makes a lot of sense. When I read the question, "why is
> server tags being added as an extension" the answer that comes to mind
> first is, "because the extension framework is there and that's how
> things have been done so far."
> 
> I think the original thinking on extensions was, make everything
> optional so users can enable/disable as they please, operators can
> disable any feature by removing the extension. Another benefit is the
> ability for anyone to add a (non-useful to the community at-large)
> feature without having to patch in several places.
> 
> I used to be for extensions for the aforementioned benefits, but now I
> tend to think it's too flexible and complex. It's so flexible that you
> can easily get yourself into a situation where your deployment can't
> work with other useful tools/libraries/etc which expect a certain
> contract from the Nova API. It doesn't make sense to let the API we
> provide be so arbitrary. It's certainly not friendly to API users.

Right. This is the number one reason API extensions need to go bye bye.

> We still have the ability to disable or limit features based on policy
> -- I don't think we need to do it via extensions.

Agreed.

> The only problem that seems to be left is, how can we allow people to
> add un-upstreamable features to the API in their internal deployments?
> I know the ideal answer is "don't do that" but the reality is some
> things will never be agreed upon upstream and I do see value in the
> extension framework for that. I don't think anything in-tree should be
> implemented as extensions, though.

IMO, all of the above functionality belongs as an entirely different
endpoint and REST API.

If it's not in the Nova /nova/api/ directory, it's not the Nova API.

As I've written a number of times before, just because achieving some
modicum of consensus about some new API resource or feature is hard,
does not mean that we should spend our time enabling vendors and others
to circumvent the API writing process by using API extensions.

It's hard work coming to consensus about a new API feature. But it's
necessary work.

Best,
-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Andrew Laski]

On 03/09/2015 06:04 PM, melanie witt wrote:

On Mar 9, 2015, at 13:14, Sean Dague  wrote:


So possibly another way to think about this is our prior signaling of
what was supported by Nova was signaled by the extension list. Our code
was refactored into a way that supported optional loading by that unit.

As we're making things less optional it probably makes sense to evolve
the API code tree to look more like our REST resource tree. Yes, that
means servers.py ends up being big, but it is less confusing that all
servers related code is in that file vs all over a bunch of other files.

So I'd agree that in this case server tags probably should just be in
servers.py. I also think long term we should do some "plugin collapse"
for stuff that's all really just features on one resource tree so that
the local filesystem code structure looks a bit more like the REST url tree.

I think this makes a lot of sense. When I read the question, "why is server tags being added 
as an extension" the answer that comes to mind first is, "because the extension framework 
is there and that's how things have been done so far."

I think the original thinking on extensions was, make everything optional so 
users can enable/disable as they please, operators can disable any feature by 
removing the extension. Another benefit is the ability for anyone to add a 
(non-useful to the community at-large) feature without having to patch in 
several places.

I used to be for extensions for the aforementioned benefits, but now I tend to 
think it's too flexible and complex. It's so flexible that you can easily get 
yourself into a situation where your deployment can't work with other useful 
tools/libraries/etc which expect a certain contract from the Nova API. It 
doesn't make sense to let the API we provide be so arbitrary. It's certainly 
not friendly to API users.

We still have the ability to disable or limit features based on policy -- I 
don't think we need to do it via extensions.

The only problem that seems to be left is, how can we allow people to add un-upstreamable 
features to the API in their internal deployments? I know the ideal answer is "don't 
do that" but the reality is some things will never be agreed upon upstream and I do 
see value in the extension framework for that. I don't think anything in-tree should be 
implemented as extensions, though.


At the moment this is provided for by an experimental flag in the 
response headers: 
https://git.openstack.org/cgit/openstack/nova-specs/tree/specs/kilo/approved/api-microversions.rst#n182 
.  It is intended to be used for transitioning from the current state of 
extensions to a place where optional API extensions aren't allowed, but 
that discussion can continue if there's a case for allowing some 
optional components for deployers.  I'm in favor of having a mechanism 
for adding features to an deployment as long as it's exposed in a way 
that makes it clear that it's separate from the standard API, e.g. an 
entirely separate tree, not just resource prefixes.




melanie (melwitt)






__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Christopher Yeoh]

On Mon, 09 Mar 2015 16:14:21 -0400
Sean Dague  wrote:

> On 03/09/2015 03:37 PM, Jay Pipes wrote:
> > On 03/08/2015 08:10 AM, Alex Xu wrote:
> >> Thanks for Jay point this out! If we have agreement on this and
> >> document it, that will be great for guiding developer how to add
> >> new API.
> >>
> >> I know we didn't want extension for API. But I think we still
> >> need modularity. I don't think we should put everything in a single
> >> file, that file will become huge in the future and hard to
> >> maintenance.
> > 
> > I don't think everything should be in a single file either. In fact,
> > I've never advocated for that.
> > 
> >> We can make the 'extension' not configurable. Replace 'extension'
> >> with another name, deprecate the extension info api int he
> >> future But that is not mean we should put everything in a file.
> > 
> > I didn't say that in my email. I'm not sure where you got the
> > impression I want to put everything in one file?
> > 
> >> For modularity, we need define what should be in a separated
> >> module(it is extension now.) There are three cases:
> >>
> >> 1. Add new resource
> >>  This is totally worth to put in a separated module.
> > 
> > Agreed.
> > 
> >> 2. Add new sub-resource
> >>  like server-tags, I prefer to put in a separated module, I
> >> don't think put another 100 lines code in the servers.py is good
> >> choice.
> > 
> > Agreed, which is exactly what I said in my email:
> > 
> > "Similarly, new microversion API functionality should live in a
> > module, as a top-level (or subcollection) Controller in
> > /nova/api/openstack/compute/, and should not be in the
> > /nova/api/openstack/compute/plugins/ directory. Why? Because it's
> > not a plugin."
> > 

Ok so I'm getting confused about what we're disagreeing about then.

However, the first microversion change
https://review.openstack.org/#/c/140313/32 is one case where we didn't
need to create a new extension, relying only on microversions to add a
new parameter to the response, whereas server tags does add a new
conroller (os-server-tags) which is non trivia so I think it does. Do
we disagree about that?

btw in a situation now where (I think) we are saying we are not going
to do any work for 3rd parties to add their own API plugins and have a
well planned API we don't need the prefixes as we don't need the prefix
on parameter names as there won't be name clashes without us realising
during testing. And we certainly don't need the os- prefix is the
plugin alias, but we do neeed them unique across the API I believe
because of the way we store information about them.

> >> 3. extend attributes and methods for a existed resource
> >> like add new attributes for servers, we can choice one of
> >> existed module to put it in. Just like this patch
> >> https://review.openstack.org/#/c/155853/
> >> But for servers-tags, it's sub-resource, we can put it in
> >> its-own module.
> > 
> > Agreed, and that's what I put in my email.
> > 
> >> If we didn't want to support extension right now, we can begin
> >> from not show servers-tags in extension info API first. That means
> >> extension info is freeze now. We deprecated the extension info api
> >> in later version.


It can be surpressed from /extensions by adding it to the suppress list
in the extensions code. Probably a good idea to stop v2.1+microversion
code REST API users not accidentally relying on it.
> > 
> > I don't understand what you're saying here. Could you elaborate?
> > What I am asking for is for new functionality (like the server-tags
> > subcollection resource), just add a new module called
> > /nova/api/openstack/compute/server_tags.py, create a Controller
> > object in that file with the new server tags resource, and don't
> > use any of the API extensions framework whatsoever.
> > 
> > In addition to that, for the changes to the main GET
> > /servers/{server_id} resource, use microversions to decorate the
> > /nova/api/openstack/compute/servers.py.Controller.show() method for
> > 2.4 and add a "tags" key to the dict (note: NOT a
> > "os-server-tags:tags" key) returned by GET /servers/{server_id}. No
> > use of API extensions needed.
>


So I that's doabe but I think if we compared to the two wsgi.extends
is cleaner and less code and we have to have the separate module file
anyway for the controller. Can discuss this more later

Incidentally as has been mentioned before we need new names for the
API and where the files need to live needs cleaning up. For example v3
and v2.1 needs to be worked out of the directory paths.This clenaup not
on involves but directly  affects the all the related unit and
functional tests. Nor should we have contrib or should v2 live directly
in in nova/api/openstack/compute. But we also need a name for v2.1
microversions and should spend some time on that (does api modules work
for anyone?)

I think this dicussion indicates we should start with a bit of planning
first rather than just jump in and sta

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Alex Xu]

2015-03-10 3:37 GMT+08:00 Jay Pipes :

> On 03/08/2015 08:10 AM, Alex Xu wrote:
>
>> Thanks for Jay point this out! If we have agreement on this and document
>> it, that will be great for guiding developer how to add new API.
>>
>> I know we didn't want extension for API. But I think we still
>> need modularity. I don't think we should put everything in a single
>> file, that file will become huge in the future and hard to maintenance.
>>
>
> I don't think everything should be in a single file either. In fact, I've
> never advocated for that.
>
>  We can make the 'extension' not configurable. Replace 'extension' with
>> another name, deprecate the extension info api int he future But
>> that is not mean we should put everything in a file.
>>
>
> I didn't say that in my email. I'm not sure where you got the impression I
> want to put everything in one file?
>
>  For modularity, we need define what should be in a separated module(it
>> is extension now.) There are three cases:
>>
>> 1. Add new resource
>>  This is totally worth to put in a separated module.
>>
>
> Agreed.
>
>  2. Add new sub-resource
>>  like server-tags, I prefer to put in a separated module, I don't
>> think put another 100 lines code in the servers.py is good choice.
>>
>
> Agreed, which is exactly what I said in my email:
>
> "Similarly, new microversion API functionality should live in a
> module, as a top-level (or subcollection) Controller in
> /nova/api/openstack/compute/, and should not be in the
> /nova/api/openstack/compute/plugins/ directory. Why? Because it's
> not a plugin."
>
>  3. extend attributes and methods for a existed resource
>> like add new attributes for servers, we can choice one of existed
>> module to put it in. Just like this patch
>> https://review.openstack.org/#/c/155853/
>> But for servers-tags, it's sub-resource, we can put it in its-own
>> module.
>>
>
> Agreed, and that's what I put in my email.
>
>  If we didn't want to support extension right now, we can begin from not
>> show servers-tags in extension info API first. That means extension info
>> is freeze now. We deprecated the extension info api in later version.
>>
>
> I don't understand what you're saying here. Could you elaborate? What I am
> asking for is for new functionality (like the server-tags subcollection
> resource), just add a new module called 
> /nova/api/openstack/compute/server_tags.py,
> create a Controller object in that file with the new server tags resource,
> and don't use any of the API extensions framework whatsoever.
>
>
Ok, I think I see you now. Forgive me for any misunderstood.

Actually I mean we can freeze and depreciated the '/extensions' API at
here. We can freeze the '/extensions' API now, didn't show server-tags
extension in that API.

Follow the discussion, I think there is two things related to 'extension'.
One is '/extensions' API, another one is plugin framework. And I added
devref about that https://review.openstack.org/162912

For the modularity, I put in separated patch
https://review.openstack.org/162913

Those two things I think we can discussion them separately. (Emm.This
is kind of evidence about I misunderstood you)



> In addition to that, for the changes to the main GET /servers/{server_id}
> resource, use microversions to decorate the 
> /nova/api/openstack/compute/servers.py.Controller.show()
> method for 2.4 and add a "tags" key to the dict (note: NOT a
> "os-server-tags:tags" key) returned by GET /servers/{server_id}. No use of
> API extensions needed.
>
> Best,
> -jay
>
>  Thanks
>> Alex
>>
>> 2015-03-08 8:31 GMT+08:00 Jay Pipes > >:
>>
>> Hi Stackers,
>>
>> Now that microversions have been introduced to the Nova API (meaning
>> we can now have novaclient request, say, version 2.3 of the Nova API
>> using the special X-OpenStack-Nova-API-Version HTTP header), is
>> there any good reason to require API extensions at all for *new*
>> functionality.
>>
>> Sergey Nikitin is currently in the process of code review for the
>> final patch that adds server instance tagging to the Nova API:
>>
>> https://review.openstack.org/#__/c/128940
>> 
>>
>> Unfortunately, for some reason I really don't understand, Sergey is
>> being required to create an API extension called "os-server-tags" in
>> order to add the server tag functionality to the API. The patch
>> implements the 2.4 Nova API microversion, though, as you can see
>> from this part of the patch:
>>
>> https://review.openstack.org/#__/c/128940/43/nova/api/__
>> openstack/compute/plugins/v3/__server_tags.py
>> > openstack/compute/plugins/v3/server_tags.py>
>>
>> What is the point of creating a new "plugin"/API extension for this
>> new functionality? Why can't we just modify the
>> nova/api/openstack/compute/__server.py Controller.sho

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Alex Xu]

I have done the first version, follow this discussion I separated them into
two patches:

1. The discussion about eliminated extension:
https://review.openstack.org/162912

2. The discussion about modularity:
https://review.openstack.org/162913

After begin the writefound I loss some confidence for my English and
view. So please feel free to comment and update the doc to add your thought
and your view, and add yourself as co-author.(If you feel hopeless, you
also can start new one)

Thanks
Alex

2015-03-09 23:37 GMT+08:00 Alex Xu :

> ok, no problem, will take a look it tomorrow.
>
> 2015-03-09 20:18 GMT+08:00 Christopher Yeoh :
>
>>
>>
>> On Mon, Mar 9, 2015 at 10:08 PM, John Garbutt 
>> wrote:
>>
>>> +1
>>>
>>> Please could you submit a dev ref for this?
>>>
>>> We can argue on the review, a bit like this one:
>>>
>>> https://github.com/openstack/nova/blob/master/doc/source/devref/policy_enforcement.rst
>>>
>>> I think it'd also be a good idea to add a testcase (use test_plugins
>> directory where you can
>> define your own controller which is never plubished) for each example so
>> they don't get out of date
>>
>> Regards,
>>
>> Chris
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Ken'ichi Ohmichi]

2015-03-09 20:36 GMT+09:00 Christopher Yeoh :
>
>
> On Mon, Mar 9, 2015 at 9:35 PM, Sean Dague  wrote:
>>
>> On 03/07/2015 07:31 PM, Jay Pipes wrote:
>> > Hi Stackers,
>> >
>> > Now that microversions have been introduced to the Nova API (meaning we
>> > can now have novaclient request, say, version 2.3 of the Nova API using
>> > the special X-OpenStack-Nova-API-Version HTTP header), is there any good
>> > reason to require API extensions at all for *new* functionality.
>> >
>> > Sergey Nikitin is currently in the process of code review for the final
>> > patch that adds server instance tagging to the Nova API:
>> >
>> > https://review.openstack.org/#/c/128940
>> >
>> > Unfortunately, for some reason I really don't understand, Sergey is
>> > being required to create an API extension called "os-server-tags" in
>> > order to add the server tag functionality to the API. The patch
>> > implements the 2.4 Nova API microversion, though, as you can see from
>> > this part of the patch:
>> >
>> >
>> > https://review.openstack.org/#/c/128940/43/nova/api/openstack/compute/plugins/v3/server_tags.py
>> >
>> >
>> > What is the point of creating a new "plugin"/API extension for this new
>> > functionality? Why can't we just modify the
>> > nova/api/openstack/compute/server.py Controller.show() method and
>> > decorate it with a 2.4 microversion that adds a "tags" attribute to the
>> > returned server dictionary?
>> >
>> >
>> > https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/servers.py#L369
>> >
>> >
>> > Because we're using an API extension for this new server tags
>> > functionality, we are instead having the extension "extend" the server
>> > dictionary with an "os-server-tags:tags" key containing the list of
>> > string tags.
>> >
>> > This is ugly and pointless. We don't need to use API extensions any more
>> > for this stuff.
>> >
>> > A client knows that server tags are supported by the 2.4 API
>> > microversion. If the client requests the 2.4+ API, then we should just
>> > include the "tags" attribute in the server dictionary.
>> >
>> > Similarly, new microversion API functionality should live in a module,
>> > as a top-level (or subcollection) Controller in
>> > /nova/api/openstack/compute/, and should not be in the
>> > /nova/api/openstack/compute/plugins/ directory. Why? Because it's not a
>> > plugin.
>> >
>> > Why are we continuing to use these awkward, messy, and cumbersome API
>> > extensions?
>> >
>> > Please, I am begging the Nova core team. Let us stop this madness. No
>> > more API extensions.
>>
>> Agreed, the current extensions list exists to explain the base v2
>> functionality. I think we should consider that frozen and deprecated as
>> of v2.1 as we have a better way to express features.
>>
>> -Sean
>>
>
>
> So I think we can a microversion ASAP to remove support for /extensions.
> Obviously we'll need to keep the actual code
> to support v2.1 for quite a while though.

big +1 for removing /extensions.
Now /extensions API provides available futures on each cloud services,
but provided information is difficult to use.
In addition, there is a ugly trick[1] in the implementation for
providing v2 full compatible information.

> I think we still want some fields in the controller like we do because we
> want to automate JSON-HOME/Schema stuff (maybe not sure)

yeah, JSON-Home is one of standard ways for providing this kind of information.
The nova-spec is https://review.openstack.org/#/c/130715/
I hope it will be implement in Liberty.

Thanks
Ken Ohmichi

---
[1]: 
https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/plugins/v3/extension_info.py#L29

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Ken'ichi Ohmichi]

Hi,

2015-03-09 20:38 GMT+09:00 John Garbutt :
> On 8 March 2015 at 12:10, Alex Xu  wrote:
>> Thanks for Jay point this out! If we have agreement on this and document it,
>> that will be great for guiding developer how to add new API.
>
> +1
>
> Please could you submit a dev ref for this?
>
> We can argue on the review, a bit like this one:
> https://github.com/openstack/nova/blob/master/doc/source/devref/policy_enforcement.rst
>
>> For modularity, we need define what should be in a separated module(it is
>> extension now.) There are three cases:
>>
>> 1. Add new resource
>> This is totally worth to put in a separated module.
>
> +1

yeah, +1 for simple/readable code.

>> 2. Add new sub-resource
>> like server-tags, I prefer to put in a separated module, I don't think
>> put another 100 lines code in the servers.py is good choice.
>
> -1
>
> I hate the idea of show instance extension code for version 2.4 living
> separately to the rest of the instance show logic, when it really
> doesn't have to.
>
> It feels too heavyweight in its current form.
>
> Maybe we need a more modular way of expressing the extension within
> the same file?

That would depend on how size of a file.
I'd like to avoid a single huge file for reviewing a patch on firefox.
Sometimes, loading time is long due to the size(especially nova/compute/api.py).

>>> What is the point of creating a new "plugin"/API extension for this new
>>> functionality? Why can't we just modify the
>>> nova/api/openstack/compute/server.py Controller.show() method and decorate
>>> it with a 2.4 microversion that adds a "tags" attribute to the returned
>>> server dictionary?
>>>
>>> Similarly, new microversion API functionality should live in a module, as
>>> a top-level (or subcollection) Controller in /nova/api/openstack/compute/,
>>> and should not be in the /nova/api/openstack/compute/plugins/ directory.
>>> Why? Because it's not a plugin.
>
> Everything is a "plugin" in v3, no more distinction between core vs
> plugin. It needs renaming really.

+1, but we need to remove current v2(not v2.1) code before doing that.
To reduce cost for maintaining both v2 and v2.1 code, we will use v2.1
code only in long term and v2 code will be removed.
Now v2 code lives under /nova/api/openstack/compute and
/nova/api/openstack/compute/contrib, and v2.1 code does under
/nova/api/openstack/compute/plugins/v3.
So current directory structure would be easy to remove v2 code in
long term.

>>> Why are we continuing to use these awkward, messy, and cumbersome API
>>> extensions?
>
> We certainly should never be forced to add an extension to advertise
> new functionality anymore.
>
> Its a big reason why I want to see the API micro-versions succeed.

+1
I also don't want to add dummy extensions just for advertising a new
functionalities more.
When I saw the code first time, I was confused due to them.

Thanks
Ken Ohmichi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[melanie witt]

On Mar 9, 2015, at 13:14, Sean Dague  wrote:

> So possibly another way to think about this is our prior signaling of
> what was supported by Nova was signaled by the extension list. Our code
> was refactored into a way that supported optional loading by that unit.
> 
> As we're making things less optional it probably makes sense to evolve
> the API code tree to look more like our REST resource tree. Yes, that
> means servers.py ends up being big, but it is less confusing that all
> servers related code is in that file vs all over a bunch of other files.
> 
> So I'd agree that in this case server tags probably should just be in
> servers.py. I also think long term we should do some "plugin collapse"
> for stuff that's all really just features on one resource tree so that
> the local filesystem code structure looks a bit more like the REST url tree.

I think this makes a lot of sense. When I read the question, "why is server 
tags being added as an extension" the answer that comes to mind first is, 
"because the extension framework is there and that's how things have been done 
so far."

I think the original thinking on extensions was, make everything optional so 
users can enable/disable as they please, operators can disable any feature by 
removing the extension. Another benefit is the ability for anyone to add a 
(non-useful to the community at-large) feature without having to patch in 
several places.

I used to be for extensions for the aforementioned benefits, but now I tend to 
think it's too flexible and complex. It's so flexible that you can easily get 
yourself into a situation where your deployment can't work with other useful 
tools/libraries/etc which expect a certain contract from the Nova API. It 
doesn't make sense to let the API we provide be so arbitrary. It's certainly 
not friendly to API users. 

We still have the ability to disable or limit features based on policy -- I 
don't think we need to do it via extensions.

The only problem that seems to be left is, how can we allow people to add 
un-upstreamable features to the API in their internal deployments? I know the 
ideal answer is "don't do that" but the reality is some things will never be 
agreed upon upstream and I do see value in the extension framework for that. I 
don't think anything in-tree should be implemented as extensions, though.

melanie (melwitt)






signature.asc
Description: Message signed with OpenPGP using GPGMail
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Sean Dague]

On 03/09/2015 03:37 PM, Jay Pipes wrote:
> On 03/08/2015 08:10 AM, Alex Xu wrote:
>> Thanks for Jay point this out! If we have agreement on this and document
>> it, that will be great for guiding developer how to add new API.
>>
>> I know we didn't want extension for API. But I think we still
>> need modularity. I don't think we should put everything in a single
>> file, that file will become huge in the future and hard to maintenance.
> 
> I don't think everything should be in a single file either. In fact,
> I've never advocated for that.
> 
>> We can make the 'extension' not configurable. Replace 'extension' with
>> another name, deprecate the extension info api int he future But
>> that is not mean we should put everything in a file.
> 
> I didn't say that in my email. I'm not sure where you got the impression
> I want to put everything in one file?
> 
>> For modularity, we need define what should be in a separated module(it
>> is extension now.) There are three cases:
>>
>> 1. Add new resource
>>  This is totally worth to put in a separated module.
> 
> Agreed.
> 
>> 2. Add new sub-resource
>>  like server-tags, I prefer to put in a separated module, I don't
>> think put another 100 lines code in the servers.py is good choice.
> 
> Agreed, which is exactly what I said in my email:
> 
> "Similarly, new microversion API functionality should live in a
> module, as a top-level (or subcollection) Controller in
> /nova/api/openstack/compute/, and should not be in the
> /nova/api/openstack/compute/plugins/ directory. Why? Because it's
> not a plugin."
> 
>> 3. extend attributes and methods for a existed resource
>> like add new attributes for servers, we can choice one of existed
>> module to put it in. Just like this patch
>> https://review.openstack.org/#/c/155853/
>> But for servers-tags, it's sub-resource, we can put it in its-own
>> module.
> 
> Agreed, and that's what I put in my email.
> 
>> If we didn't want to support extension right now, we can begin from not
>> show servers-tags in extension info API first. That means extension info
>> is freeze now. We deprecated the extension info api in later version.
> 
> I don't understand what you're saying here. Could you elaborate? What I
> am asking for is for new functionality (like the server-tags
> subcollection resource), just add a new module called
> /nova/api/openstack/compute/server_tags.py, create a Controller object
> in that file with the new server tags resource, and don't use any of the
> API extensions framework whatsoever.
> 
> In addition to that, for the changes to the main GET
> /servers/{server_id} resource, use microversions to decorate the
> /nova/api/openstack/compute/servers.py.Controller.show() method for 2.4
> and add a "tags" key to the dict (note: NOT a "os-server-tags:tags" key)
> returned by GET /servers/{server_id}. No use of API extensions needed.

So possibly another way to think about this is our prior signaling of
what was supported by Nova was signaled by the extension list. Our code
was refactored into a way that supported optional loading by that unit.

As we're making things less optional it probably makes sense to evolve
the API code tree to look more like our REST resource tree. Yes, that
means servers.py ends up being big, but it is less confusing that all
servers related code is in that file vs all over a bunch of other files.

So I'd agree that in this case server tags probably should just be in
servers.py. I also think long term we should do some "plugin collapse"
for stuff that's all really just features on one resource tree so that
the local filesystem code structure looks a bit more like the REST url tree.

-Sean

-- 
Sean Dague
http://dague.net



signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Jay Pipes]

On 03/09/2015 07:32 AM, Christopher Yeoh wrote:


So the first thing I think we want to distinguish between plugins
being a REST API user or operator concept and it being a tool
developers use as a framework to support the Nova REST API. As I've
mentioned before I've no problem with the feature set of the API
being fixed (per microversion) across all Nova deployments. Get back
to me when we have consensus on that and its trivial to implement and
we'll no longer have the concept of core andextension/plugin.


Why are we continuing to add API extensions for new stuff?


But plugin like implementations using Stevedore as a tool for developers
to keep good modularity has proven to be very useful to keep complexity
level lower and interactions between modules much clearer.


Uhm, I beg to differ. The "plugin" implementation using Stevedore has 
made things way more complicated than they need to be.


It would be much simpler to have a single directory call 
/nova/api/openstack/compute/ with modules representing each resource 
collection controller.


> servers.py is

an example of this where in v2 I think we have/had the most complex
method and even with all the fix up work which has been done on it it is
still very complicated to understand.


It doesn't *need* to be more complicated. The microversions framework 
decorators allow us to encapsulate the differences between microversions 
for a particular method.


Best,
-jay


On Sun, Mar 8, 2015 at 11:01 AM, Jay Pipes mailto:jaypi...@gmail.com>> wrote:

Hi Stackers,

Now that microversions have been introduced to the Nova API (meaning
we can now have novaclient request, say, version 2.3 of the Nova API
using the special X-OpenStack-Nova-API-Version HTTP header), is
there any good reason to require API extensions at all for *new*
functionality.

Sergey Nikitin is currently in the process of code review for the
final patch that adds server instance tagging to the Nova API:

https://review.openstack.org/#__/c/128940


Unfortunately, for some reason I really don't understand, Sergey is
being required to create an API extension called "os-server-tags" in
order to add the server tag functionality to the API. The patch
implements the 2.4 Nova API microversion, though, as you can see
from this part of the patch:


https://review.openstack.org/#__/c/128940/43/nova/api/__openstack/compute/plugins/v3/__server_tags.py



What is the point of creating a new "plugin"/API extension for this
new functionality? Why can't we just modify the
nova/api/openstack/compute/__server.py Controller.show() method and
decorate it with a 2.4 microversion that adds a "tags" attribute to
the returned server dictionary?


Actually I think it does more than just add extra reponse information:
- it adds extra tags parameter to show
   - it doesn't add it to index, but it probably should add the response
information to detail to be consistent with the rest of the API
- It adds a new resource /servers/server_id/tags
- with create, delete and delete all supported. I don't think that
these belong in servers.py


https://github.com/openstack/__nova/blob/master/nova/api/__openstack/compute/servers.py#__L369



Because we're using an API extension for this new server tags
functionality, we are instead having the extension "extend" the
server dictionary with an "os-server-tags:tags" key containing the
list of string tags.

This is ugly and pointless. We don't need to use API extensions any
more for this stuff.


So we had a prefix rule originally in V2 to allow for extensions and
guarantee no name clashes. I'd be happy removing this requirement, even
removing old ones as long as we have consensus.

A client knows that server tags are supported by the 2.4 API
microversion. If the client requests the 2.4+ API, then we should
just include the "tags" attribute in the server dictionary.

Similarly, new microversion API functionality should live in a
module, as a top-level (or subcollection) Controller in
/nova/api/openstack/compute/, and should not be in the
/nova/api/openstack/compute/__plugins/ directory. Why? Because it's
not a plugin.

So I don't see how that changes whether we're using plugins (from a user
point of view) or not. The good news for you is that
there is fixing the shambles of a directory structure for the api is on
the list of things to do, it just wasn't a high prioirty things for us
in Kilo,
get v2.1 and microversions out. For example, we have v3 in the directory
path as well for historical reasons and we also have a contrib directory
in compute and none of those are really "contrib" now either.  Now the
nova/

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Jay Pipes]

On 03/08/2015 08:10 AM, Alex Xu wrote:

Thanks for Jay point this out! If we have agreement on this and document
it, that will be great for guiding developer how to add new API.

I know we didn't want extension for API. But I think we still
need modularity. I don't think we should put everything in a single
file, that file will become huge in the future and hard to maintenance.


I don't think everything should be in a single file either. In fact, 
I've never advocated for that.



We can make the 'extension' not configurable. Replace 'extension' with
another name, deprecate the extension info api int he future But
that is not mean we should put everything in a file.


I didn't say that in my email. I'm not sure where you got the impression 
I want to put everything in one file?



For modularity, we need define what should be in a separated module(it
is extension now.) There are three cases:

1. Add new resource
 This is totally worth to put in a separated module.


Agreed.


2. Add new sub-resource
 like server-tags, I prefer to put in a separated module, I don't
think put another 100 lines code in the servers.py is good choice.


Agreed, which is exactly what I said in my email:

"Similarly, new microversion API functionality should live in a
module, as a top-level (or subcollection) Controller in
/nova/api/openstack/compute/, and should not be in the
/nova/api/openstack/compute/plugins/ directory. Why? Because it's
not a plugin."


3. extend attributes and methods for a existed resource
like add new attributes for servers, we can choice one of existed
module to put it in. Just like this patch
https://review.openstack.org/#/c/155853/
But for servers-tags, it's sub-resource, we can put it in its-own
module.


Agreed, and that's what I put in my email.


If we didn't want to support extension right now, we can begin from not
show servers-tags in extension info API first. That means extension info
is freeze now. We deprecated the extension info api in later version.


I don't understand what you're saying here. Could you elaborate? What I 
am asking for is for new functionality (like the server-tags 
subcollection resource), just add a new module called 
/nova/api/openstack/compute/server_tags.py, create a Controller object 
in that file with the new server tags resource, and don't use any of the 
API extensions framework whatsoever.


In addition to that, for the changes to the main GET 
/servers/{server_id} resource, use microversions to decorate the 
/nova/api/openstack/compute/servers.py.Controller.show() method for 2.4 
and add a "tags" key to the dict (note: NOT a "os-server-tags:tags" key) 
returned by GET /servers/{server_id}. No use of API extensions needed.


Best,
-jay


Thanks
Alex

2015-03-08 8:31 GMT+08:00 Jay Pipes mailto:jaypi...@gmail.com>>:

Hi Stackers,

Now that microversions have been introduced to the Nova API (meaning
we can now have novaclient request, say, version 2.3 of the Nova API
using the special X-OpenStack-Nova-API-Version HTTP header), is
there any good reason to require API extensions at all for *new*
functionality.

Sergey Nikitin is currently in the process of code review for the
final patch that adds server instance tagging to the Nova API:

https://review.openstack.org/#__/c/128940


Unfortunately, for some reason I really don't understand, Sergey is
being required to create an API extension called "os-server-tags" in
order to add the server tag functionality to the API. The patch
implements the 2.4 Nova API microversion, though, as you can see
from this part of the patch:


https://review.openstack.org/#__/c/128940/43/nova/api/__openstack/compute/plugins/v3/__server_tags.py



What is the point of creating a new "plugin"/API extension for this
new functionality? Why can't we just modify the
nova/api/openstack/compute/__server.py Controller.show() method and
decorate it with a 2.4 microversion that adds a "tags" attribute to
the returned server dictionary?


https://github.com/openstack/__nova/blob/master/nova/api/__openstack/compute/servers.py#__L369



Because we're using an API extension for this new server tags
functionality, we are instead having the extension "extend" the
server dictionary with an "os-server-tags:tags" key containing the
list of string tags.

This is ugly and pointless. We don't need to use API extensions any
more for this stuff.

A client knows that server tags are supported by the 2.4 API
microversion. If the client requests the 2.4+ API, then we should
just include the "tags" attribute in the server dictionary.

Similarly, new microversion API functionality should l

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Alex Xu]

ok, no problem, will take a look it tomorrow.

2015-03-09 20:18 GMT+08:00 Christopher Yeoh :

>
>
> On Mon, Mar 9, 2015 at 10:08 PM, John Garbutt 
> wrote:
>
>> +1
>>
>> Please could you submit a dev ref for this?
>>
>> We can argue on the review, a bit like this one:
>>
>> https://github.com/openstack/nova/blob/master/doc/source/devref/policy_enforcement.rst
>>
>> I think it'd also be a good idea to add a testcase (use test_plugins
> directory where you can
> define your own controller which is never plubished) for each example so
> they don't get out of date
>
> Regards,
>
> Chris
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Attila Fazekas]

- Original Message -
> From: "Christopher Yeoh" 
> To: "OpenStack Development Mailing List (not for usage questions)" 
> 
> Sent: Monday, March 9, 2015 1:04:15 PM
> Subject: Re: [openstack-dev] [nova][api] Microversions. And why do we need 
> API extensions for new API functionality?
> 
> 
> 
> On Mon, Mar 9, 2015 at 10:08 PM, John Garbutt < j...@johngarbutt.com > wrote:
> 
> 
> Hi,
> 
> I think I agree with Jay here, but let me explain...
> 
> On 8 March 2015 at 12:10, Alex Xu < sou...@gmail.com > wrote:
> > Thanks for Jay point this out! If we have agreement on this and document
> > it,
> > that will be great for guiding developer how to add new API.
> 
> +1
> 
> Please could you submit a dev ref for this?
> 
> We can argue on the review, a bit like this one:
> https://github.com/openstack/nova/blob/master/doc/source/devref/policy_enforcement.rst
> 
> > For modularity, we need define what should be in a separated module(it is
> > extension now.) There are three cases:
> > 
> > 1. Add new resource
> > This is totally worth to put in a separated module.
> 
> +1
> 
> > 2. Add new sub-resource
> > like server-tags, I prefer to put in a separated module, I don't think
> > put another 100 lines code in the servers.py is good choice.
> 
> -1
> 
> I hate the idea of show instance extension code for version 2.4 living
> separately to the rest of the instance show logic, when it really
> doesn't have to.
> 
> It feels too heavyweight in its current form.
> 
> 
> If the only thing server-tags did was to add a parameter then we wouldn't
> need a new extension,
> but its not, it adds another resource with associated actions
> 
> 
> Maybe we need a more modular way of expressing the extension within
> the same file?
> 
> 
> I think servers.py is simply to big. Its much harder to read and debug than
> any other plugin just because of its size - or
> maybe I just need a 50" monitor :) I'd rather ensure functionality common
> server-tags and the API is kept together rather than
> spread through servers.py
> 
No, it isn't.
It is bellow 2k line. I usually use low level tools even for python related 
debugging.
For ex.: strace, gdb..
With the extension I get lot of files which may be involved may be not.
This causes me additional headache, because more difficult to see which file
is involved. After an strace I usually know what is the mistake, I just need to 
find
it in the code.
I do not like when I had to open more than 3 files, after I see what went wrong.
I some cases I use gdb, just to get python stack traces just before the first 
incorrect
step is detected, in other cases git grep is sufficient.

Actually for me the extensions increases the required monitor number,
some cases I also need to use more complicated approaches.
I tied lot of python profiler tool as well, but there is no single all cases 
win version,
extra custom hack is required in many cases to get something close what I want.

> 
> > 3. extend attributes and methods for a existed resource
> > like add new attributes for servers, we can choice one of existed module
> > to put it in. Just like this patch https://review.openstack.org/#/c/155853/
> 
> +1
> 
> I wish it was easier to read, but I hope thats fixable long term.
> 
> > 2015-03-08 8:31 GMT+08:00 Jay Pipes < jaypi...@gmail.com >:
> >> Now that microversions have been introduced to the Nova API (meaning we
> >> can now have novaclient request, say, version 2.3 of the Nova API using
> >> the
> >> special X-OpenStack-Nova-API-Version HTTP header), is there any good
> >> reason
> >> to require API extensions at all for *new* functionality.
> 
> As above, a new "resource" probably should get a new "plugins/v3" module
> right?
> 
> It feels (at worst) borderline in the os-server-tags case, due to the
> extra actions.
> 
> >> What is the point of creating a new "plugin"/API extension for this new
> >> functionality? Why can't we just modify the
> >> nova/api/openstack/compute/server.py Controller.show() method and decorate
> >> it with a 2.4 microversion that adds a "tags" attribute to the returned
> >> server dictionary?
> >> 
> >> Similarly, new microversion API functionality should live in a module, as
> >> a top-level (or subcollection) Controller in /nova/api/openstack/compute/,
> >> and should not be in the /nova/api/openstack/compute/plugins/ directory.
> >> Why? Because it's not a plugin.
> 
> Everything is a "plugin" in v3, no

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Christopher Yeoh]

On Mon, Mar 9, 2015 at 10:08 PM, John Garbutt  wrote:

> +1
>
> Please could you submit a dev ref for this?
>
> We can argue on the review, a bit like this one:
>
> https://github.com/openstack/nova/blob/master/doc/source/devref/policy_enforcement.rst
>
> I think it'd also be a good idea to add a testcase (use test_plugins
directory where you can
define your own controller which is never plubished) for each example so
they don't get out of date

Regards,

Chris
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Christopher Yeoh]

On Mon, Mar 9, 2015 at 10:08 PM, John Garbutt  wrote:

> Hi,
>
> I think I agree with Jay here, but let me explain...
>
> On 8 March 2015 at 12:10, Alex Xu  wrote:
> > Thanks for Jay point this out! If we have agreement on this and document
> it,
> > that will be great for guiding developer how to add new API.
>
> +1
>
> Please could you submit a dev ref for this?
>
> We can argue on the review, a bit like this one:
>
> https://github.com/openstack/nova/blob/master/doc/source/devref/policy_enforcement.rst
>
> > For modularity, we need define what should be in a separated module(it is
> > extension now.) There are three cases:
> >
> > 1. Add new resource
> > This is totally worth to put in a separated module.
>
> +1
>
> > 2. Add new sub-resource
> > like server-tags, I prefer to put in a separated module, I don't
> think
> > put another 100 lines code in the servers.py is good choice.
>
> -1
>
> I hate the idea of show instance extension code for version 2.4 living
> separately to the rest of the instance show logic, when it really
> doesn't have to.
>
> It feels too heavyweight in its current form.
>
>
If the only thing server-tags did was to add a parameter then we wouldn't
need a new extension,
but its not, it adds another resource with associated actions


> Maybe we need a more modular way of expressing the extension within
> the same file?
>
>
I think servers.py is simply to big. Its much harder to read and debug than
any other plugin just because of its size - or
maybe I just need a 50" monitor :) I'd rather ensure functionality common
server-tags and the API is kept together rather than
spread through servers.py



> > 3. extend attributes and methods for a existed resource
> >like add new attributes for servers, we can choice one of existed
> module
> > to put it in. Just like this patch
> https://review.openstack.org/#/c/155853/
>
> +1
>
> I wish it was easier to read, but I hope thats fixable long term.
>
> > 2015-03-08 8:31 GMT+08:00 Jay Pipes :
> >> Now that microversions have been introduced to the Nova API (meaning we
> >> can now have novaclient request, say, version 2.3 of the Nova API using
> the
> >> special X-OpenStack-Nova-API-Version HTTP header), is there any good
> reason
> >> to require API extensions at all for *new* functionality.
>
> As above, a new "resource" probably should get a new "plugins/v3" module
> right?
>
> It feels (at worst) borderline in the os-server-tags case, due to the
> extra actions.
>
> >> What is the point of creating a new "plugin"/API extension for this new
> >> functionality? Why can't we just modify the
> >> nova/api/openstack/compute/server.py Controller.show() method and
> decorate
> >> it with a 2.4 microversion that adds a "tags" attribute to the returned
> >> server dictionary?
> >>
> >> Similarly, new microversion API functionality should live in a module,
> as
> >> a top-level (or subcollection) Controller in
> /nova/api/openstack/compute/,
> >> and should not be in the /nova/api/openstack/compute/plugins/ directory.
> >> Why? Because it's not a plugin.
>
> Everything is a "plugin" in v3, no more distinction between core vs
> plugin. It needs renaming really.
>
> It should look just like servers, I guess, which is a top level item:
>
> https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/plugins/v3/servers.py
>
> >> Why are we continuing to use these awkward, messy, and cumbersome API
> >> extensions?
>
> We certainly should never be forced to add an extension to advertise
> new functionality anymore.
>
> Its a big reason why I want to see the API micro-versions succeed.
>

Yep, there is I think no reason except to support /extensions for now and I
don't really think its worth having
two entry points, one for modules which will appear in /extensions and one
for modules which won't appear
in /extensioins. The overhead is low. We should warn v2.1+ users to ignore
/extensions unless they are legacy v2 api users
and they should remove their use of it anyway as soon as they get off v2.1.
They key to dumping it all is
when people tell us v2.1 really is behaving just like v2 so we can remove
the old v2 code and then later have a microversion that
doesn't support /extensions. I hope all the json-home stuff is in by then
:-)

>
> >> Please, I am begging the Nova core team. Let us stop this madness. No
> more
> >> API extensions.
>
> Lets try get something agreed in devref, so we are ready to go when
> Liberty opens.
>
> It would be nice to look at ways to fold back the existing extensions
> into the main code. I know there are v2.0 compatibility issues there,
> but I think/hope thats mostly cosmetic at this point.
>
>
Yea we already did a lot of that in v3 and had to separate  some of them
out again for v2.1 (argh!). Others we have just faked (eg you load module
"X" you get module "Y" for free which doesn't really exist anymore - but
only for those that we were very sure that the extension only existed to
notify users that s

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[John Garbutt]

Hi,

I think I agree with Jay here, but let me explain...

On 8 March 2015 at 12:10, Alex Xu  wrote:
> Thanks for Jay point this out! If we have agreement on this and document it,
> that will be great for guiding developer how to add new API.

+1

Please could you submit a dev ref for this?

We can argue on the review, a bit like this one:
https://github.com/openstack/nova/blob/master/doc/source/devref/policy_enforcement.rst

> For modularity, we need define what should be in a separated module(it is
> extension now.) There are three cases:
>
> 1. Add new resource
> This is totally worth to put in a separated module.

+1

> 2. Add new sub-resource
> like server-tags, I prefer to put in a separated module, I don't think
> put another 100 lines code in the servers.py is good choice.

-1

I hate the idea of show instance extension code for version 2.4 living
separately to the rest of the instance show logic, when it really
doesn't have to.

It feels too heavyweight in its current form.

Maybe we need a more modular way of expressing the extension within
the same file?

> 3. extend attributes and methods for a existed resource
>like add new attributes for servers, we can choice one of existed module
> to put it in. Just like this patch https://review.openstack.org/#/c/155853/

+1

I wish it was easier to read, but I hope thats fixable long term.

> 2015-03-08 8:31 GMT+08:00 Jay Pipes :
>> Now that microversions have been introduced to the Nova API (meaning we
>> can now have novaclient request, say, version 2.3 of the Nova API using the
>> special X-OpenStack-Nova-API-Version HTTP header), is there any good reason
>> to require API extensions at all for *new* functionality.

As above, a new "resource" probably should get a new "plugins/v3" module right?

It feels (at worst) borderline in the os-server-tags case, due to the
extra actions.

>> What is the point of creating a new "plugin"/API extension for this new
>> functionality? Why can't we just modify the
>> nova/api/openstack/compute/server.py Controller.show() method and decorate
>> it with a 2.4 microversion that adds a "tags" attribute to the returned
>> server dictionary?
>>
>> Similarly, new microversion API functionality should live in a module, as
>> a top-level (or subcollection) Controller in /nova/api/openstack/compute/,
>> and should not be in the /nova/api/openstack/compute/plugins/ directory.
>> Why? Because it's not a plugin.

Everything is a "plugin" in v3, no more distinction between core vs
plugin. It needs renaming really.

It should look just like servers, I guess, which is a top level item:
https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/plugins/v3/servers.py

>> Why are we continuing to use these awkward, messy, and cumbersome API
>> extensions?

We certainly should never be forced to add an extension to advertise
new functionality anymore.

Its a big reason why I want to see the API micro-versions succeed.

>> Please, I am begging the Nova core team. Let us stop this madness. No more
>> API extensions.

Lets try get something agreed in devref, so we are ready to go when
Liberty opens.

It would be nice to look at ways to fold back the existing extensions
into the main code. I know there are v2.0 compatibility issues there,
but I think/hope thats mostly cosmetic at this point.

Thanks,
John

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Christopher Yeoh]

On Mon, Mar 9, 2015 at 9:35 PM, Sean Dague  wrote:

> On 03/07/2015 07:31 PM, Jay Pipes wrote:
> > Hi Stackers,
> >
> > Now that microversions have been introduced to the Nova API (meaning we
> > can now have novaclient request, say, version 2.3 of the Nova API using
> > the special X-OpenStack-Nova-API-Version HTTP header), is there any good
> > reason to require API extensions at all for *new* functionality.
> >
> > Sergey Nikitin is currently in the process of code review for the final
> > patch that adds server instance tagging to the Nova API:
> >
> > https://review.openstack.org/#/c/128940
> >
> > Unfortunately, for some reason I really don't understand, Sergey is
> > being required to create an API extension called "os-server-tags" in
> > order to add the server tag functionality to the API. The patch
> > implements the 2.4 Nova API microversion, though, as you can see from
> > this part of the patch:
> >
> >
> https://review.openstack.org/#/c/128940/43/nova/api/openstack/compute/plugins/v3/server_tags.py
> >
> >
> > What is the point of creating a new "plugin"/API extension for this new
> > functionality? Why can't we just modify the
> > nova/api/openstack/compute/server.py Controller.show() method and
> > decorate it with a 2.4 microversion that adds a "tags" attribute to the
> > returned server dictionary?
> >
> >
> https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/servers.py#L369
> >
> >
> > Because we're using an API extension for this new server tags
> > functionality, we are instead having the extension "extend" the server
> > dictionary with an "os-server-tags:tags" key containing the list of
> > string tags.
> >
> > This is ugly and pointless. We don't need to use API extensions any more
> > for this stuff.
> >
> > A client knows that server tags are supported by the 2.4 API
> > microversion. If the client requests the 2.4+ API, then we should just
> > include the "tags" attribute in the server dictionary.
> >
> > Similarly, new microversion API functionality should live in a module,
> > as a top-level (or subcollection) Controller in
> > /nova/api/openstack/compute/, and should not be in the
> > /nova/api/openstack/compute/plugins/ directory. Why? Because it's not a
> > plugin.
> >
> > Why are we continuing to use these awkward, messy, and cumbersome API
> > extensions?
> >
> > Please, I am begging the Nova core team. Let us stop this madness. No
> > more API extensions.
>
> Agreed, the current extensions list exists to explain the base v2
> functionality. I think we should consider that frozen and deprecated as
> of v2.1 as we have a better way to express features.
>
> -Sean
>
>

So I think we can a microversion ASAP to remove support for /extensions.
Obviously we'll need to keep the actual code
to support v2.1 for quite a while though.

I think we still want some fields in the controller like we do because we
want to automate JSON-HOME/Schema stuff (maybe not sure)


> --
> Sean Dague
> http://dague.net
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Christopher Yeoh]

Hi,

Apologies for the slow reply, long weekend because of a public holiday over
here. I'm probably going to end up repeating part of what
Alex has mentioned as well.

So the first thing I think we want to distinguish between plugins being a
REST API user or operator concept and it being
 a tool developers use as a framework to support the Nova REST API. As I've
mentioned before I've no problem with the feature set of the
API being fixed (per microversion) across all Nova deployments. Get back to
me when we have consensus on that and its trivial to
implement and we'll no longer have the concept of core and extension/plugin.

But plugin like implementations using Stevedore as a tool for developers to
keep good modularity has proven to be very useful to keep complexity level
lower and interactions between modules much clearer. servers.py is an
example of this where in v2 I think we have/had the most complex method and
even with all the fix up work which has been done on it it is still very
complicated to understand.


On Sun, Mar 8, 2015 at 11:01 AM, Jay Pipes  wrote:

> Hi Stackers,
>
> Now that microversions have been introduced to the Nova API (meaning we
> can now have novaclient request, say, version 2.3 of the Nova API using the
> special X-OpenStack-Nova-API-Version HTTP header), is there any good reason
> to require API extensions at all for *new* functionality.
>
> Sergey Nikitin is currently in the process of code review for the final
> patch that adds server instance tagging to the Nova API:
>
> https://review.openstack.org/#/c/128940
>
> Unfortunately, for some reason I really don't understand, Sergey is being
> required to create an API extension called "os-server-tags" in order to add
> the server tag functionality to the API. The patch implements the 2.4 Nova
> API microversion, though, as you can see from this part of the patch:
>
> https://review.openstack.org/#/c/128940/43/nova/api/
> openstack/compute/plugins/v3/server_tags.py
>
> What is the point of creating a new "plugin"/API extension for this new
> functionality? Why can't we just modify the 
> nova/api/openstack/compute/server.py
> Controller.show() method and decorate it with a 2.4 microversion that adds
> a "tags" attribute to the returned server dictionary?
>
>
Actually I think it does more than just add extra reponse information:
- it adds extra tags parameter to show
  - it doesn't add it to index, but it probably should add the response
information to detail to be consistent with the rest of the API
- It adds a new resource /servers/server_id/tags
   - with create, delete and delete all supported. I don't think that these
belong in servers.py



> https://github.com/openstack/nova/blob/master/nova/api/
> openstack/compute/servers.py#L369
>
> Because we're using an API extension for this new server tags
> functionality, we are instead having the extension "extend" the server
> dictionary with an "os-server-tags:tags" key containing the list of string
> tags.
>
> This is ugly and pointless. We don't need to use API extensions any more
> for this stuff.
>
>
So we had a prefix rule originally in V2 to allow for extensions and
guarantee no name clashes. I'd be happy removing this requirement, even
removing old ones as long as we have consensus.


> A client knows that server tags are supported by the 2.4 API microversion.
> If the client requests the 2.4+ API, then we should just include the "tags"
> attribute in the server dictionary.
>
> Similarly, new microversion API functionality should live in a module, as
> a top-level (or subcollection) Controller in /nova/api/openstack/compute/,
> and should not be in the /nova/api/openstack/compute/plugins/ directory.
> Why? Because it's not a plugin.
>
> So I don't see how that changes whether we're using plugins (from a user
point of view) or not. The good news for you is that
there is fixing the shambles of a directory structure for the api is on the
list of things to do, it just wasn't a high prioirty things for us in Kilo,
get v2.1 and microversions out. For example, we have v3 in the directory
path as well for historical reasons and we also have a contrib directory in
compute and none of those are really "contrib" now either.  Now the
nova/api/openstack/compute/ directory where you want to put all the v2
microversions code is currently full of v2 core code already.  It just
makes more sense to me to wait unti the old v2 core code can be removed
because the v2.1 api is considered equivalent and then move the v2.1
microversions code into its final place , rather than a shuffle now to move
the old v2 code (along with all the changes need to the unittests) and then
just have to delete it again not much longer.





> Why are we continuing to use these awkward, messy, and cumbersome API
> extensions?
>
> Please, I am begging the Nova core team. Let us stop this madness. No more
> API extensions.
>
>
It is still not clear to me exactly what you mean by use of an extension.
None of us ours 

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Sean Dague]

On 03/07/2015 07:31 PM, Jay Pipes wrote:
> Hi Stackers,
> 
> Now that microversions have been introduced to the Nova API (meaning we
> can now have novaclient request, say, version 2.3 of the Nova API using
> the special X-OpenStack-Nova-API-Version HTTP header), is there any good
> reason to require API extensions at all for *new* functionality.
> 
> Sergey Nikitin is currently in the process of code review for the final
> patch that adds server instance tagging to the Nova API:
> 
> https://review.openstack.org/#/c/128940
> 
> Unfortunately, for some reason I really don't understand, Sergey is
> being required to create an API extension called "os-server-tags" in
> order to add the server tag functionality to the API. The patch
> implements the 2.4 Nova API microversion, though, as you can see from
> this part of the patch:
> 
> https://review.openstack.org/#/c/128940/43/nova/api/openstack/compute/plugins/v3/server_tags.py
> 
> 
> What is the point of creating a new "plugin"/API extension for this new
> functionality? Why can't we just modify the
> nova/api/openstack/compute/server.py Controller.show() method and
> decorate it with a 2.4 microversion that adds a "tags" attribute to the
> returned server dictionary?
> 
> https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/servers.py#L369
> 
> 
> Because we're using an API extension for this new server tags
> functionality, we are instead having the extension "extend" the server
> dictionary with an "os-server-tags:tags" key containing the list of
> string tags.
> 
> This is ugly and pointless. We don't need to use API extensions any more
> for this stuff.
> 
> A client knows that server tags are supported by the 2.4 API
> microversion. If the client requests the 2.4+ API, then we should just
> include the "tags" attribute in the server dictionary.
> 
> Similarly, new microversion API functionality should live in a module,
> as a top-level (or subcollection) Controller in
> /nova/api/openstack/compute/, and should not be in the
> /nova/api/openstack/compute/plugins/ directory. Why? Because it's not a
> plugin.
> 
> Why are we continuing to use these awkward, messy, and cumbersome API
> extensions?
> 
> Please, I am begging the Nova core team. Let us stop this madness. No
> more API extensions.

Agreed, the current extensions list exists to explain the base v2
functionality. I think we should consider that frozen and deprecated as
of v2.1 as we have a better way to express features.

-Sean

-- 
Sean Dague
http://dague.net



signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Attila Fazekas]

I agree with Jay.

The extension layer is also expensive in CPU usage,
and it also makes more difficult to troubleshoot issues.


- Original Message -
> From: "Jay Pipes" 
> To: "OpenStack Development Mailing List" , 
> "Sergey Nikitin"
> 
> Sent: Sunday, March 8, 2015 1:31:34 AM
> Subject: [openstack-dev] [nova][api] Microversions. And why do we need API 
> extensions for new API functionality?
> 
> Hi Stackers,
> 
> Now that microversions have been introduced to the Nova API (meaning we
> can now have novaclient request, say, version 2.3 of the Nova API using
> the special X-OpenStack-Nova-API-Version HTTP header), is there any good
> reason to require API extensions at all for *new* functionality.
> 
> Sergey Nikitin is currently in the process of code review for the final
> patch that adds server instance tagging to the Nova API:
> 
> https://review.openstack.org/#/c/128940
> 
> Unfortunately, for some reason I really don't understand, Sergey is
> being required to create an API extension called "os-server-tags" in
> order to add the server tag functionality to the API. The patch
> implements the 2.4 Nova API microversion, though, as you can see from
> this part of the patch:
> 
> https://review.openstack.org/#/c/128940/43/nova/api/openstack/compute/plugins/v3/server_tags.py
> 
> What is the point of creating a new "plugin"/API extension for this new
> functionality? Why can't we just modify the
> nova/api/openstack/compute/server.py Controller.show() method and
> decorate it with a 2.4 microversion that adds a "tags" attribute to the
> returned server dictionary?
> 
> https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/servers.py#L369
> 
> Because we're using an API extension for this new server tags
> functionality, we are instead having the extension "extend" the server
> dictionary with an "os-server-tags:tags" key containing the list of
> string tags.
> 
> This is ugly and pointless. We don't need to use API extensions any more
> for this stuff.
> 
> A client knows that server tags are supported by the 2.4 API
> microversion. If the client requests the 2.4+ API, then we should just
> include the "tags" attribute in the server dictionary.
> 
> Similarly, new microversion API functionality should live in a module,
> as a top-level (or subcollection) Controller in
> /nova/api/openstack/compute/, and should not be in the
> /nova/api/openstack/compute/plugins/ directory. Why? Because it's not a
> plugin.
> 
> Why are we continuing to use these awkward, messy, and cumbersome API
> extensions?
> 
> Please, I am begging the Nova core team. Let us stop this madness. No
> more API extensions.
> 
> Best,
> -jay
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Alex Xu]

Thanks for Jay point this out! If we have agreement on this and document
it, that will be great for guiding developer how to add new API.

I know we didn't want extension for API. But I think we still
need modularity. I don't think we should put everything in a single file,
that file will become huge in the future and hard to maintenance. We can
make the 'extension' not configurable. Replace 'extension' with another
name, deprecate the extension info api int he future But that is not
mean we should put everything in a file.

For modularity, we need define what should be in a separated module(it is
extension now.) There are three cases:

1. Add new resource
This is totally worth to put in a separated module.
2. Add new sub-resource
like server-tags, I prefer to put in a separated module, I don't think
put another 100 lines code in the servers.py is good choice.
3. extend attributes and methods for a existed resource
   like add new attributes for servers, we can choice one of existed module
to put it in. Just like this patch https://review.openstack.org/#/c/155853/
   But for servers-tags, it's sub-resource, we can put it in its-own module.

If we didn't want to support extension right now, we can begin from not
show servers-tags in extension info API first. That means extension info is
freeze now. We deprecated the extension info api in later version.

Thanks
Alex

2015-03-08 8:31 GMT+08:00 Jay Pipes :

> Hi Stackers,
>
> Now that microversions have been introduced to the Nova API (meaning we
> can now have novaclient request, say, version 2.3 of the Nova API using the
> special X-OpenStack-Nova-API-Version HTTP header), is there any good reason
> to require API extensions at all for *new* functionality.
>
> Sergey Nikitin is currently in the process of code review for the final
> patch that adds server instance tagging to the Nova API:
>
> https://review.openstack.org/#/c/128940
>
> Unfortunately, for some reason I really don't understand, Sergey is being
> required to create an API extension called "os-server-tags" in order to add
> the server tag functionality to the API. The patch implements the 2.4 Nova
> API microversion, though, as you can see from this part of the patch:
>
> https://review.openstack.org/#/c/128940/43/nova/api/
> openstack/compute/plugins/v3/server_tags.py
>
> What is the point of creating a new "plugin"/API extension for this new
> functionality? Why can't we just modify the 
> nova/api/openstack/compute/server.py
> Controller.show() method and decorate it with a 2.4 microversion that adds
> a "tags" attribute to the returned server dictionary?
>
> https://github.com/openstack/nova/blob/master/nova/api/
> openstack/compute/servers.py#L369
>
> Because we're using an API extension for this new server tags
> functionality, we are instead having the extension "extend" the server
> dictionary with an "os-server-tags:tags" key containing the list of string
> tags.
>
> This is ugly and pointless. We don't need to use API extensions any more
> for this stuff.
>
> A client knows that server tags are supported by the 2.4 API microversion.
> If the client requests the 2.4+ API, then we should just include the "tags"
> attribute in the server dictionary.
>
> Similarly, new microversion API functionality should live in a module, as
> a top-level (or subcollection) Controller in /nova/api/openstack/compute/,
> and should not be in the /nova/api/openstack/compute/plugins/ directory.
> Why? Because it's not a plugin.
>
> Why are we continuing to use these awkward, messy, and cumbersome API
> extensions?
>
> Please, I am begging the Nova core team. Let us stop this madness. No more
> API extensions.
>
> Best,
> -jay
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova][api] Microversions. And why do we need API extensions for new API functionality?

[Jay Pipes]

Hi Stackers,

Now that microversions have been introduced to the Nova API (meaning we 
can now have novaclient request, say, version 2.3 of the Nova API using 
the special X-OpenStack-Nova-API-Version HTTP header), is there any good 
reason to require API extensions at all for *new* functionality.


Sergey Nikitin is currently in the process of code review for the final 
patch that adds server instance tagging to the Nova API:


https://review.openstack.org/#/c/128940

Unfortunately, for some reason I really don't understand, Sergey is 
being required to create an API extension called "os-server-tags" in 
order to add the server tag functionality to the API. The patch 
implements the 2.4 Nova API microversion, though, as you can see from 
this part of the patch:


https://review.openstack.org/#/c/128940/43/nova/api/openstack/compute/plugins/v3/server_tags.py

What is the point of creating a new "plugin"/API extension for this new 
functionality? Why can't we just modify the 
nova/api/openstack/compute/server.py Controller.show() method and 
decorate it with a 2.4 microversion that adds a "tags" attribute to the 
returned server dictionary?


https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/servers.py#L369

Because we're using an API extension for this new server tags 
functionality, we are instead having the extension "extend" the server 
dictionary with an "os-server-tags:tags" key containing the list of 
string tags.


This is ugly and pointless. We don't need to use API extensions any more 
for this stuff.


A client knows that server tags are supported by the 2.4 API 
microversion. If the client requests the 2.4+ API, then we should just 
include the "tags" attribute in the server dictionary.


Similarly, new microversion API functionality should live in a module, 
as a top-level (or subcollection) Controller in 
/nova/api/openstack/compute/, and should not be in the 
/nova/api/openstack/compute/plugins/ directory. Why? Because it's not a 
plugin.


Why are we continuing to use these awkward, messy, and cumbersome API 
extensions?


Please, I am begging the Nova core team. Let us stop this madness. No 
more API extensions.


Best,
-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev