Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On 6 March 2014 13:18, Christopher Yeoh wrote: > On Thu, Mar 6, 2014 at 10:24 PM, John Garbutt wrote: >> >> On 5 March 2014 03:44, Christopher Yeoh wrote: >> > But this plan is certainly something I'm happy to support. >> >> One extra thing we need to consider, how to deal with new APIs while >> we go through this transition? >> >> I don't really have any answers to hand, but I want v3 to get released >> ASAP, and I want people to easily add API features to Nova. If the >> proxy is ready early, maybe we could have people implement only v3 >> extensions, then optionally add v2 extension that just uses wraps the >> v2 proxy + v3 extensions. >> > > So pretty much any extension which is added now (or those that have gone in > during Icehouse) should > offer an API which is exactly the same. There's no excuse for divergence so > what you suggest is most likely > quite doable. We might not even need a proxy in some cases to make it > available in the v2 namespace. > > >> >> > - I'm not sure how this affects how we approach the tasks work. Will >> > need to think about that more. >> >> +1 >> >> Its a thread of its own, but I think improving instance-actions, still >> leaving tasks for v3, might be the way forward. >> >> While we need to avoid feature creep, I still think if we add tasks >> into v3 in the right way, it could be what makes people move to v3. >> > > Yea we really need to flesh out what we want from tasks long term. > >> >> One more thing... I wonder if all new extensions should be considered >> "experimental" (or version 0) for at least one cycle. In theory, that >> should help us avoid some of the worst mistakes when adding new APIs. >> > > Yes, so this I think is a similar suggestion to being able to have > extensions first drop > into a holding area outside of Nova. Because the whole freeze deadline rush > is a recipe for making > compromises around the API that we don't want to live with for the long term > but do so > because we want a feature to merge soon. I don't like the idea of encouraging these to be out of tree. I prefer the idea of getting better at controlled in-tree innovation. > But I think whatever approach we > take it needs > to come with the possibility that if its not fixed up in a reasonable time, > it may get removed. > So we don't end up with a large pool of experimental things. +1 I totally forgot about that, but I agree. Remove things that were experimental at the last major release at Feature Freeze, or something like that, could work. > As an aside, I think we need to improve the process we use for API related > features. Because a lot of the > problems that get picked up during code review could have been avoided if we > had a better review > earlier on that just focussed on the API design independent of > implementation and Nova internals. +1 We have more detailed blueprint reviews now, but I agree we need to get much better at this. I think the ideas around moving to gerrit for blueprint reviews are a good starting point. Seeing each others reviews will help us collectively improve in the same way as our code reviews today. John ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On 03/06/2014 08:18 AM, Christopher Yeoh wrote: > As an aside, I think we need to improve the process we use for API > related features. Because a lot of the > problems that get picked up during code review could have been avoided > if we had a better review > earlier on that just focussed on the API design independent of > implementation and Nova internals. Yes, I would absolutely like to get better about doing this as a part of our blueprint review process. -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On Thu, Mar 6, 2014 at 10:24 PM, John Garbutt wrote: > On 5 March 2014 03:44, Christopher Yeoh wrote: > > But this plan is certainly something I'm happy to support. > One extra thing we need to consider, how to deal with new APIs while > we go through this transition? > > I don't really have any answers to hand, but I want v3 to get released > ASAP, and I want people to easily add API features to Nova. If the > proxy is ready early, maybe we could have people implement only v3 > extensions, then optionally add v2 extension that just uses wraps the > v2 proxy + v3 extensions. > > So pretty much any extension which is added now (or those that have gone in during Icehouse) should offer an API which is exactly the same. There's no excuse for divergence so what you suggest is most likely quite doable. We might not even need a proxy in some cases to make it available in the v2 namespace. > > - I'm not sure how this affects how we approach the tasks work. Will > > need to think about that more. > > +1 > > Its a thread of its own, but I think improving instance-actions, still > leaving tasks for v3, might be the way forward. > > While we need to avoid feature creep, I still think if we add tasks > into v3 in the right way, it could be what makes people move to v3. > > Yea we really need to flesh out what we want from tasks long term. > One more thing... I wonder if all new extensions should be considered > "experimental" (or version 0) for at least one cycle. In theory, that > should help us avoid some of the worst mistakes when adding new APIs. > > Yes, so this I think is a similar suggestion to being able to have extensions first drop into a holding area outside of Nova. Because the whole freeze deadline rush is a recipe for making compromises around the API that we don't want to live with for the long term but do so because we want a feature to merge soon. But I think whatever approach we take it needs to come with the possibility that if its not fixed up in a reasonable time, it may get removed. So we don't end up with a large pool of experimental things. As an aside, I think we need to improve the process we use for API related features. Because a lot of the problems that get picked up during code review could have been avoided if we had a better review earlier on that just focussed on the API design independent of implementation and Nova internals. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On Thu, Mar 6, 2014 at 8:35 AM, Russell Bryant wrote: > On 03/05/2014 04:52 PM, Everett Toews wrote: > > On Mar 5, 2014, at 8:16 AM, Russell Bryant wrote: > > > >> I think SDK support is critical for the success of v3 long term. I > >> expect most people are using the APIs through one of the major SDKs, so > >> v3 won't take off until that happens. I think our top priority in Nova > >> to help ensure this happens is to provide top notch documentation on the > >> v3 API, as well as all of the differences between v2 and v3. > > > > Yes. Thank you. > > > > And the earlier we can see the first parts of this documentation, both > the differences between v2 and v3 and the final version, the better. If we > can give you feedback on early versions of the docs, the whole thing will > go much more smoothly. > > > > You can find us in #openstack-sdks on IRC. > > Sounds good. > > I know there is at least this wiki page started to document the changes, > but I believe there is still a lot missing. > > https://wiki.openstack.org/wiki/NovaAPIv2tov3 > > Yes, unfortunately we realised a little late in development just how important a document like this would be so its not as complete as I'd like. It doesn't for example address input validation changes. But it is something that definitely needs to be 100% complete and is getting better as the tempest testing side fills out. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On 5 March 2014 03:44, Christopher Yeoh wrote: > But this plan is certainly something I'm happy to support. +1 On 5 March 2014 03:44, Christopher Yeoh wrote: > So I think this a good compromise to keep things moving. Some aspects > that we'll need to consider: > > - We need more tempest coverage of Nova because it doesn't cover all of > the Nova API yet. We've been working on increasing this as part of > the V3 API work anyway (and V2 support is an easyish side effect). > But more people willing to write tempest tests are always welcome :-) +1 This seems key to making sure we do any "v2" compatibility right. > - I think in practice this will probably mean that V3 API is > realistically only a K rather than J thing - just in terms of allowing > a reasonable timeline to not only implement the v2 compat but get > feedback from deployers. +1 One extra thing we need to consider, how to deal with new APIs while we go through this transition? I don't really have any answers to hand, but I want v3 to get released ASAP, and I want people to easily add API features to Nova. If the proxy is ready early, maybe we could have people implement only v3 extensions, then optionally add v2 extension that just uses wraps the v2 proxy + v3 extensions. > - I'm not sure how this affects how we approach the tasks work. Will > need to think about that more. +1 Its a thread of its own, but I think improving instance-actions, still leaving tasks for v3, might be the way forward. While we need to avoid feature creep, I still think if we add tasks into v3 in the right way, it could be what makes people move to v3. One more thing... I wonder if all new extensions should be considered "experimental" (or version 0) for at least one cycle. In theory, that should help us avoid some of the worst mistakes when adding new APIs. John ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
> -Original Message-
> From: Jay Pipes [mailto:jaypi...@gmail.com]
> Sent: Wednesday, March 05, 2014 10:52 PM
> To: openstack-dev@lists.openstack.org
> Subject: Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
>
> On Wed, 2014-03-05 at 05:43 +, Kenichi Oomichi wrote:
> > > -Original Message-
> > > From: Dan Smith [mailto:d...@danplanet.com]
> > > Sent: Wednesday, March 05, 2014 9:09 AM
> > > To: OpenStack Development Mailing List (not for usage questions)
> > > Subject: Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
> > >
> > > > What I'd like to do next is work through a new proposal that includes
> > > > keeping both v2 and v3, but with a new added focus of minimizing the
> > > > cost. This should include a path away from the dual code bases and to
> > > > something like the "v2.1" proposal.
> > >
> > > I think that the most we can hope for is consensus on _something_. So,
> > > the thing that I'm hoping would mostly satisfy the largest number of
> > > people is:
> > >
> > > - Leaving v2 and v3 as they are today in the tree, and with v3 still
> > > marked experimental for the moment
> > > - We start on a v2 proxy to v3, with the first goal of fully
> > > implementing the v2 API on top of v3, as judged by tempest
> > > - We define the criteria for removing the current v2 code and marking
> > > the v3 code supported as:
> > > - The v2 proxy passes tempest
> > > - The v2 proxy has sign-off from some major deployers as something
> > >they would be comfortable using in place of the existing v2 code
> > > - The v2 proxy seems to us to be lower maintenance and otherwise
> > >preferable to either keeping both, breaking all our users, deleting
> > >v3 entirely, etc
> >
> > Thanks, Dan.
> > The above criteria is reasonable to me.
> >
> > Now Tempest does not check API responses in many cases.
> > For example, Tempest does not check what API attributes("flavor", "image",
> > etc.) should be included in the response body of "create a server" API.
> > So we need to improve Tempest coverage from this viewpoint for verifying
> > any backward incompatibility does not happen on v2.1 API.
> > We started this improvement for Tempest and have proposed some patches
> > for it now.
>
> Kenichi-san, you may also want to check out this ML post from David
> Kranz:
>
> http://lists.openstack.org/pipermail/openstack-dev/2014-March/028920.html
Hi Jay-san,
Thank you for pointing it out. That is a good point :-)
I will join in David's idea.
Thanks
Ken'ichi Ohmichi
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On 03/05/2014 04:52 PM, Everett Toews wrote: > On Mar 5, 2014, at 8:16 AM, Russell Bryant wrote: > >> I think SDK support is critical for the success of v3 long term. I >> expect most people are using the APIs through one of the major SDKs, so >> v3 won't take off until that happens. I think our top priority in Nova >> to help ensure this happens is to provide top notch documentation on the >> v3 API, as well as all of the differences between v2 and v3. > > Yes. Thank you. > > And the earlier we can see the first parts of this documentation, both the > differences between v2 and v3 and the final version, the better. If we can > give you feedback on early versions of the docs, the whole thing will go much > more smoothly. > > You can find us in #openstack-sdks on IRC. Sounds good. I know there is at least this wiki page started to document the changes, but I believe there is still a lot missing. https://wiki.openstack.org/wiki/NovaAPIv2tov3 -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On Mar 5, 2014, at 8:16 AM, Russell Bryant wrote: > I think SDK support is critical for the success of v3 long term. I > expect most people are using the APIs through one of the major SDKs, so > v3 won't take off until that happens. I think our top priority in Nova > to help ensure this happens is to provide top notch documentation on the > v3 API, as well as all of the differences between v2 and v3. Yes. Thank you. And the earlier we can see the first parts of this documentation, both the differences between v2 and v3 and the final version, the better. If we can give you feedback on early versions of the docs, the whole thing will go much more smoothly. You can find us in #openstack-sdks on IRC. Cheers, Everett ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On 03/05/2014 08:52 AM, Christopher Lefelhocz wrote: > I like this plan as it addresses my primary concern of getting deployments > comfortable with the transition. > > We don't mention SDKs in the plan. It seems like getting at least one SDK > to use v3 would provide us additional data in the transition. There is > clear risk associated with that, but it may help to work out any > additional issues that crop up. I think this plan is mostly about ensuring we don't drown in this transition. We're focusing on lowering our maintenance cost so that the transition to the new API can happen more naturally. I think SDK support is critical for the success of v3 long term. I expect most people are using the APIs through one of the major SDKs, so v3 won't take off until that happens. I think our top priority in Nova to help ensure this happens is to provide top notch documentation on the v3 API, as well as all of the differences between v2 and v3. -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On Wed, 2014-03-05 at 05:43 +, Kenichi Oomichi wrote:
> > -Original Message-
> > From: Dan Smith [mailto:d...@danplanet.com]
> > Sent: Wednesday, March 05, 2014 9:09 AM
> > To: OpenStack Development Mailing List (not for usage questions)
> > Subject: Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
> >
> > > What I'd like to do next is work through a new proposal that includes
> > > keeping both v2 and v3, but with a new added focus of minimizing the
> > > cost. This should include a path away from the dual code bases and to
> > > something like the "v2.1" proposal.
> >
> > I think that the most we can hope for is consensus on _something_. So,
> > the thing that I'm hoping would mostly satisfy the largest number of
> > people is:
> >
> > - Leaving v2 and v3 as they are today in the tree, and with v3 still
> > marked experimental for the moment
> > - We start on a v2 proxy to v3, with the first goal of fully
> > implementing the v2 API on top of v3, as judged by tempest
> > - We define the criteria for removing the current v2 code and marking
> > the v3 code supported as:
> > - The v2 proxy passes tempest
> > - The v2 proxy has sign-off from some major deployers as something
> >they would be comfortable using in place of the existing v2 code
> > - The v2 proxy seems to us to be lower maintenance and otherwise
> >preferable to either keeping both, breaking all our users, deleting
> >v3 entirely, etc
>
> Thanks, Dan.
> The above criteria is reasonable to me.
>
> Now Tempest does not check API responses in many cases.
> For example, Tempest does not check what API attributes("flavor", "image",
> etc.) should be included in the response body of "create a server" API.
> So we need to improve Tempest coverage from this viewpoint for verifying
> any backward incompatibility does not happen on v2.1 API.
> We started this improvement for Tempest and have proposed some patches
> for it now.
Kenichi-san, you may also want to check out this ML post from David
Kranz:
http://lists.openstack.org/pipermail/openstack-dev/2014-March/028920.html
Best,
-jay
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
I like this plan as it addresses my primary concern of getting deployments comfortable with the transition. We don't mention SDKs in the plan. It seems like getting at least one SDK to use v3 would provide us additional data in the transition. There is clear risk associated with that, but it may help to work out any additional issues that crop up. Christopher On 3/4/14 6:09 PM, "Dan Smith" wrote: >> What I'd like to do next is work through a new proposal that includes >> keeping both v2 and v3, but with a new added focus of minimizing the >> cost. This should include a path away from the dual code bases and to >> something like the "v2.1" proposal. > >I think that the most we can hope for is consensus on _something_. So, >the thing that I'm hoping would mostly satisfy the largest number of >people is: > >- Leaving v2 and v3 as they are today in the tree, and with v3 still > marked experimental for the moment >- We start on a v2 proxy to v3, with the first goal of fully > implementing the v2 API on top of v3, as judged by tempest >- We define the criteria for removing the current v2 code and marking > the v3 code supported as: > - The v2 proxy passes tempest > - The v2 proxy has sign-off from some major deployers as something > they would be comfortable using in place of the existing v2 code > - The v2 proxy seems to us to be lower maintenance and otherwise > preferable to either keeping both, breaking all our users, deleting > v3 entirely, etc >- We keep this until we either come up with a proxy that works, or > decide that it's not worth the cost, etc. > >I think the list of benefits here are: > >- Gives the v3 code a chance to address some of the things we have > identified as lacking in both trees >- Gives us a chance to determine if the proxy approach is reasonable or > a nightmare >- Gives a clear go/no-go line in the sand that we can ask deployers to > critique or approve > >It doesn't address all of my concerns, but at the risk of just having >the whole community split over this discussion, I think this is probably >(hopefully?) something we can all get behind. > >Thoughts? > >--Dan > >___ >OpenStack-dev mailing list >OpenStack-dev@lists.openstack.org >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On 03/04/2014 10:44 PM, Christopher Yeoh wrote: > On Tue, 04 Mar 2014 16:09:21 -0800 > Dan Smith wrote: > >>> What I'd like to do next is work through a new proposal that >>> includes keeping both v2 and v3, but with a new added focus of >>> minimizing the cost. This should include a path away from the dual >>> code bases and to something like the "v2.1" proposal. >> >> I think that the most we can hope for is consensus on _something_. So, >> the thing that I'm hoping would mostly satisfy the largest number of >> people is: >> >> - Leaving v2 and v3 as they are today in the tree, and with v3 still >> marked experimental for the moment >> - We start on a v2 proxy to v3, with the first goal of fully >> implementing the v2 API on top of v3, as judged by tempest >> - We define the criteria for removing the current v2 code and marking >> the v3 code supported as: >> - The v2 proxy passes tempest >> - The v2 proxy has sign-off from some major deployers as something >>they would be comfortable using in place of the existing v2 code >> - The v2 proxy seems to us to be lower maintenance and otherwise >>preferable to either keeping both, breaking all our users, deleting >>v3 entirely, etc >> - We keep this until we either come up with a proxy that works, or >> decide that it's not worth the cost, etc. >> >> I think the list of benefits here are: >> >> - Gives the v3 code a chance to address some of the things we have >> identified as lacking in both trees >> - Gives us a chance to determine if the proxy approach is reasonable >> or a nightmare >> - Gives a clear go/no-go line in the sand that we can ask deployers to >> critique or approve >> >> It doesn't address all of my concerns, but at the risk of just having >> the whole community split over this discussion, I think this is >> probably (hopefully?) something we can all get behind. >> >> Thoughts? I think this is a solid plan to move forward on, and gives the proxy idea a chance to prove itself. > So I think this a good compromise to keep things moving. Some aspects > that we'll need to consider: > > - We need more tempest coverage of Nova because it doesn't cover all of > the Nova API yet. We've been working on increasing this as part of > the V3 API work anyway (and V2 support is an easyish side effect). > But more people willing to write tempest tests are always welcome :-) 100% agreed. I *highly* encourage any groups that are CDing OpenStack to get engaged in Tempest, because that's our upstream mechanism for blocking breaking code. Enhancements on the Nova v2 testing will ensure that v2 surface does not change in ways that are important to people. > - I think in practice this will probably mean that V3 API is > realistically only a K rather than J thing - just in terms of allowing > a reasonable timeline to not only implement the v2 compat but get > feedback from deployers. > - I'm not sure how this affects how we approach the tasks work. Will > need to think about that more. > > But this plan is certainly something I'm happy to support. Agreed. If we are committing to this route, I would like to see both a POC on the proxy, and early findings at Atlanta, so we can figure out if this is crazy or sane, and really define the exit criteria. The point of giving the proxy idea a chance, is the assumption that it's actually a less overhead way to evolve our API, and still keep backwards compat. But if we don't have some good indications of that being true in Atlanta, then I don't want us fully committing to a long slog into the unknown here. So work is cut out for folks that want to head down this path. But I think the early data in Atlanta will be incredibly valuable in figuring out how we move forward. -Sean -- Sean Dague Samsung Research America s...@dague.net / sean.da...@samsung.com http://dague.net signature.asc Description: OpenPGP digital signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
> -Original Message-
> From: Dan Smith [mailto:d...@danplanet.com]
> Sent: Wednesday, March 05, 2014 9:09 AM
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
>
> > What I'd like to do next is work through a new proposal that includes
> > keeping both v2 and v3, but with a new added focus of minimizing the
> > cost. This should include a path away from the dual code bases and to
> > something like the "v2.1" proposal.
>
> I think that the most we can hope for is consensus on _something_. So,
> the thing that I'm hoping would mostly satisfy the largest number of
> people is:
>
> - Leaving v2 and v3 as they are today in the tree, and with v3 still
> marked experimental for the moment
> - We start on a v2 proxy to v3, with the first goal of fully
> implementing the v2 API on top of v3, as judged by tempest
> - We define the criteria for removing the current v2 code and marking
> the v3 code supported as:
> - The v2 proxy passes tempest
> - The v2 proxy has sign-off from some major deployers as something
>they would be comfortable using in place of the existing v2 code
> - The v2 proxy seems to us to be lower maintenance and otherwise
>preferable to either keeping both, breaking all our users, deleting
>v3 entirely, etc
Thanks, Dan.
The above criteria is reasonable to me.
Now Tempest does not check API responses in many cases.
For example, Tempest does not check what API attributes("flavor", "image",
etc.) should be included in the response body of "create a server" API.
So we need to improve Tempest coverage from this viewpoint for verifying
any backward incompatibility does not happen on v2.1 API.
We started this improvement for Tempest and have proposed some patches
for it now.
Thanks
Ken'ichi Ohmichi
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On Tue, 04 Mar 2014 16:09:21 -0800 Dan Smith wrote: > > What I'd like to do next is work through a new proposal that > > includes keeping both v2 and v3, but with a new added focus of > > minimizing the cost. This should include a path away from the dual > > code bases and to something like the "v2.1" proposal. > > I think that the most we can hope for is consensus on _something_. So, > the thing that I'm hoping would mostly satisfy the largest number of > people is: > > - Leaving v2 and v3 as they are today in the tree, and with v3 still > marked experimental for the moment > - We start on a v2 proxy to v3, with the first goal of fully > implementing the v2 API on top of v3, as judged by tempest > - We define the criteria for removing the current v2 code and marking > the v3 code supported as: > - The v2 proxy passes tempest > - The v2 proxy has sign-off from some major deployers as something >they would be comfortable using in place of the existing v2 code > - The v2 proxy seems to us to be lower maintenance and otherwise >preferable to either keeping both, breaking all our users, deleting >v3 entirely, etc > - We keep this until we either come up with a proxy that works, or > decide that it's not worth the cost, etc. > > I think the list of benefits here are: > > - Gives the v3 code a chance to address some of the things we have > identified as lacking in both trees > - Gives us a chance to determine if the proxy approach is reasonable > or a nightmare > - Gives a clear go/no-go line in the sand that we can ask deployers to > critique or approve > > It doesn't address all of my concerns, but at the risk of just having > the whole community split over this discussion, I think this is > probably (hopefully?) something we can all get behind. > > Thoughts? So I think this a good compromise to keep things moving. Some aspects that we'll need to consider: - We need more tempest coverage of Nova because it doesn't cover all of the Nova API yet. We've been working on increasing this as part of the V3 API work anyway (and V2 support is an easyish side effect). But more people willing to write tempest tests are always welcome :-) - I think in practice this will probably mean that V3 API is realistically only a K rather than J thing - just in terms of allowing a reasonable timeline to not only implement the v2 compat but get feedback from deployers. - I'm not sure how this affects how we approach the tasks work. Will need to think about that more. But this plan is certainly something I'm happy to support. Chris > > --Dan > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On Tue, 04 Mar 2014 12:10:15 -0500 Russell Bryant wrote: > > What I'd like to do next is work through a new proposal that includes > keeping both v2 and v3, but with a new added focus of minimizing the > cost. This should include a path away from the dual code bases and to > something like the "v2.1" proposal. That sounds good to me. I would be very interested in any feedback that people have around the concept of v2.1 that looks just like v2 but has strong input validation. So that would be the only backwards incompatible change and would only affect those who are currently misusing the API. Because we are not modifying the original V2 API code people could do side by side tests with real clients to see how badly in practice an individual bit of software is. And we'll have tempest tests to verify the correct input behaviour path for V2 remains the same. In the meantime we'll aim to get some more fleshed out POC code out for the decorator approach to implementing V2 on the V3 codebase that we can show at the design summit. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On Wed, Mar 5, 2014 at 11:09 AM, Dan Smith wrote: >> What I'd like to do next is work through a new proposal that includes >> keeping both v2 and v3, but with a new added focus of minimizing the >> cost. This should include a path away from the dual code bases and to >> something like the "v2.1" proposal. > > I think that the most we can hope for is consensus on _something_. So, > the thing that I'm hoping would mostly satisfy the largest number of > people is: > > - Leaving v2 and v3 as they are today in the tree, and with v3 still > marked experimental for the moment > - We start on a v2 proxy to v3, with the first goal of fully > implementing the v2 API on top of v3, as judged by tempest > - We define the criteria for removing the current v2 code and marking > the v3 code supported as: > - The v2 proxy passes tempest > - The v2 proxy has sign-off from some major deployers as something >they would be comfortable using in place of the existing v2 code > - The v2 proxy seems to us to be lower maintenance and otherwise >preferable to either keeping both, breaking all our users, deleting >v3 entirely, etc > - We keep this until we either come up with a proxy that works, or > decide that it's not worth the cost, etc. > > I think the list of benefits here are: > > - Gives the v3 code a chance to address some of the things we have > identified as lacking in both trees > - Gives us a chance to determine if the proxy approach is reasonable or > a nightmare > - Gives a clear go/no-go line in the sand that we can ask deployers to > critique or approve > > It doesn't address all of my concerns, but at the risk of just having > the whole community split over this discussion, I think this is probably > (hopefully?) something we can all get behind. > > Thoughts? I think this is a good plan. Michael -- Rackspace Australia ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On Tue, 4 Mar 2014 11:29:27 -0600 Anne Gentle wrote: > I still sense that the struggle with Compute v3 is the lack of > documentation for contributor developers but also especially end > users so that we could get feedback early and often. > > My original understanding, passed by word-of-mouth, was that the goal > for v3 was to define an expanded core that nearly all deployers could > confidently put into production to serve their users needs. So I think in practice the reverse has occurred and the core has got smaller. I think that's perhaps the nature of attempting to get consensus - its far easier to get agreement that something should be optional than get agreement that everyone should support it. I believe that we really need a debate around this because as others have mentioned it directly impacts interoperability between openstack deployments for users. But we should keep this debate separate from the v2/v3 one :-) Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On March 4, 2014 at 16:13:45, Dan Smith (d...@danplanet.com) wrote: > What I'd like to do next is work through a new proposal that includes > keeping both v2 and v3, but with a new added focus of minimizing the > cost. This should include a path away from the dual code bases and to > something like the "v2.1" proposal. I think that the most we can hope for is consensus on _something_. So, the thing that I'm hoping would mostly satisfy the largest number of people is: - Leaving v2 and v3 as they are today in the tree, and with v3 still marked experimental for the moment - We start on a v2 proxy to v3, with the first goal of fully implementing the v2 API on top of v3, as judged by tempest - We define the criteria for removing the current v2 code and marking the v3 code supported as: - The v2 proxy passes tempest - The v2 proxy has sign-off from some major deployers as something they would be comfortable using in place of the existing v2 code - The v2 proxy seems to us to be lower maintenance and otherwise preferable to either keeping both, breaking all our users, deleting v3 entirely, etc - We keep this until we either come up with a proxy that works, or decide that it's not worth the cost, etc. This seems reasonable. I think the list of benefits here are: - Gives the v3 code a chance to address some of the things we have identified as lacking in both trees - Gives us a chance to determine if the proxy approach is reasonable or a nightmare - Gives a clear go/no-go line in the sand that we can ask deployers to critique or approve +1 on this. As a deployer this is a good stance and I especially like the clear go/no-go line above the other “benefits” with the assumption we are keeping V2 as is (e.g. not planning on deprecating out sections/changing interfaces, or evolving the API to be more V3 like). It doesn't address all of my concerns, but at the risk of just having the whole community split over this discussion, I think this is probably (hopefully?) something we can all get behind. I agree this doesn’t solve all the concerns, but it’s a good middle ground to stand on. I obviously have a personal preference as a deployer/supporter of OpenStack environments. I have concerns over the V2 proxy, but as long as we are keeping V2 as is, this can move us towards a larger change to V3 and have the solid tempest coverage, I don't see a reason to say “this is a bad approach”. Cheers, Morgan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
> What I'd like to do next is work through a new proposal that includes > keeping both v2 and v3, but with a new added focus of minimizing the > cost. This should include a path away from the dual code bases and to > something like the "v2.1" proposal. I think that the most we can hope for is consensus on _something_. So, the thing that I'm hoping would mostly satisfy the largest number of people is: - Leaving v2 and v3 as they are today in the tree, and with v3 still marked experimental for the moment - We start on a v2 proxy to v3, with the first goal of fully implementing the v2 API on top of v3, as judged by tempest - We define the criteria for removing the current v2 code and marking the v3 code supported as: - The v2 proxy passes tempest - The v2 proxy has sign-off from some major deployers as something they would be comfortable using in place of the existing v2 code - The v2 proxy seems to us to be lower maintenance and otherwise preferable to either keeping both, breaking all our users, deleting v3 entirely, etc - We keep this until we either come up with a proxy that works, or decide that it's not worth the cost, etc. I think the list of benefits here are: - Gives the v3 code a chance to address some of the things we have identified as lacking in both trees - Gives us a chance to determine if the proxy approach is reasonable or a nightmare - Gives a clear go/no-go line in the sand that we can ask deployers to critique or approve It doesn't address all of my concerns, but at the risk of just having the whole community split over this discussion, I think this is probably (hopefully?) something we can all get behind. Thoughts? --Dan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On Tue, 4 Mar 2014 09:26:18 -0800 Vishvananda Ishaya wrote: > > On Mar 4, 2014, at 9:10 AM, Russell Bryant wrote: > > > > Thank you all for your participation on this topic. It has been > > quite controversial, but the API we expose to our users is a really > > big deal. I'm feeling more and more confident that we're coming > > through this with a much better understanding of the problem space > > overall, as well as a better plan going forward than we had a few > > weeks ago. > > Hey Russell, > > Thanks for bringing this to the mailing list and being open to > discussion and collaboration. Also, thanks to everyone who is > participating in the plan. Doing this kind of thing in the open is > difficult and it has lead to a ton of debate, but this is the right > way to do things. It says a lot about the strength of our community > that we are able to have conversations like this without devolving > into arguments and flame wars. +1 to this. Discussions like this are very difficult to have and it is a great feature of the community that everyone remains polite and civil with each other. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On Tue, 4 Mar 2014 13:14:01 + "Daniel P. Berrange" wrote: > On Tue, Mar 04, 2014 at 07:49:03AM -0500, Sean Dague wrote: > > So this thread is getting deep again, as I expect they all will, so > > I'm just going to top post and take the ire for doing so. > > > > I also want to summarize what I've seen in the threads so far: > > > > v2 needed forever - if I do a sentiment analysis here looking at the > > orgs people are coming from, most of this is currently coming from > > Rackspace folks (publicly). There might be many reasons for this, > > one of which is the fact that they've done a big transition in the > > near past (between *not openstack* and Openstack), and felt that > > pain. Understanding that pain is good. > > > > It is interesting that Phil actually brings up a completely > > different issue from the HP cloud side, which is the amount of > > complaints they are having to field about how terrible the v2 API > > is. HP has actually had an OpenStack cloud public longer than > > Rackspace. So this feedback shouldn't be lost. > > > > So I feel like while some deployers have expressed no interest in > > moving forward on API, others can't get there soon enough. > > > > Which makes me think a lot about point 4. As has already been > > suggested we could actually make v2 a proxy to v3. And just like > > with images and volumes, it becomes frozen in Juno, and people that > > want more features will move to the v3 API. Just like with other > > services. > > > This requires internal cleanups as well. However it wouldn't shut > > down future evolution of the interface. > > > > Nova really has 4 interfaces today > > * Nova v2 JSON > > * Nova v2 XML > > * Nova v3 JSON > > * EC2 > > > > I feel like if we had to lose one to decrease maintenance cost, > > Nova v2 XML is the one to lose. And if we did, v2 on v3 isn't the > > craziest thing in the world. It's not free, but neither is the > > backport. > > A proxy of v2 onto v3 is appealing, but do we think we have good > enough testing of v2 to ensure that any proxy impl is bug-for-bug > compatible with the original native v2 implementation ? Avoiding > breakage of client apps is to me the key reason for keeping v2 > around, so we'd want very high confidence that any proxy impl is > functionally identical with the orginal impl. So if 100% bug for bug compatibility is our top concern, then the last thing we want to do is to try to evolve the V2 API. Because we will end up breaking it accidentally. And it impacts what internal changes we can make because we've had problems with accidentally exposing internal implementation issues through the API. > If we want to proxy v2 onto v3, then by that same argument should we > be proxying EC2 onto v3 as well. ie Nova v3 JSON be the only > supported API, and every thing else just be a proxy, potentially > maintained out of tree from main nova codebase. So I think this is a good long term goal (whether the proxying code is in our out of tree is up for debate). There are apparently issues with the EC2 needing something that our API doesn't expose, but perhaps it could. It just needs people who are willing to look at doing that work. Just as a random data point, when the devs who have been looking at implementing the google compute engine asked about this sort of thing on the mailing list, we suggested that they look at layering their API on top of the Nova API. And if IIRC they said that would be possible. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On 03/04/2014 12:42 PM, Sean Dague wrote: > On 03/04/2014 12:26 PM, Vishvananda Ishaya wrote: >> >> On Mar 4, 2014, at 9:10 AM, Russell Bryant >> wrote: >>> >>> Thank you all for your participation on this topic. It has >>> been quite controversial, but the API we expose to our users is >>> a really big deal. I'm feeling more and more confident that >>> we're coming through this with a much better understanding of >>> the problem space overall, as well as a better plan going >>> forward than we had a few weeks ago. >> >> Hey Russell, >> >> Thanks for bringing this to the mailing list and being open to >> discussion and collaboration. Also, thanks to everyone who is >> participating in the plan. Doing this kind of thing in the open >> is difficult and it has lead to a ton of debate, but this is the >> right way to do things. It says a lot about the strength of our >> community that we are able to have conversations like this >> without devolving into arguments and flame wars. >> >> Vish > > +1, and definitely appreciate Russell's leadership through this > whole discussion. Thanks for the kind words. It really is a group effort. Even in the face of an incredibly controversial topic, we can't be afraid to ask hard questions. It takes a lot of maturity and focus to work through the answers toward some sort of consensus without it turning into, as Vish said, arguments and flame wars. Nova (and OpenStack overall) is made up of a pretty incredible group of people. -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On 03/04/2014 12:26 PM, Vishvananda Ishaya wrote: > > On Mar 4, 2014, at 9:10 AM, Russell Bryant wrote: >> >> Thank you all for your participation on this topic. It has been quite >> controversial, but the API we expose to our users is a really big deal. >> I'm feeling more and more confident that we're coming through this with >> a much better understanding of the problem space overall, as well as a >> better plan going forward than we had a few weeks ago. > > Hey Russell, > > Thanks for bringing this to the mailing list and being open to discussion > and collaboration. Also, thanks to everyone who is participating in the > plan. Doing this kind of thing in the open is difficult and it has lead to > a ton of debate, but this is the right way to do things. It says a lot > about the strength of our community that we are able to have conversations > like this without devolving into arguments and flame wars. > > Vish +1, and definitely appreciate Russell's leadership through this whole discussion. -Sean -- Sean Dague Samsung Research America s...@dague.net / sean.da...@samsung.com http://dague.net signature.asc Description: OpenPGP digital signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On Mar 4, 2014, at 9:10 AM, Russell Bryant wrote: > > Thank you all for your participation on this topic. It has been quite > controversial, but the API we expose to our users is a really big deal. > I'm feeling more and more confident that we're coming through this with > a much better understanding of the problem space overall, as well as a > better plan going forward than we had a few weeks ago. Hey Russell, Thanks for bringing this to the mailing list and being open to discussion and collaboration. Also, thanks to everyone who is participating in the plan. Doing this kind of thing in the open is difficult and it has lead to a ton of debate, but this is the right way to do things. It says a lot about the strength of our community that we are able to have conversations like this without devolving into arguments and flame wars. Vish signature.asc Description: Message signed with OpenPGP using GPGMail ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On Tue, Mar 4, 2014 at 11:10 AM, Russell Bryant wrote: > On 03/03/2014 12:32 PM, Russell Bryant wrote: > > There has been quite a bit of discussion about the future of the v3 API > > recently. > > :-) > > Since this proposal was posted, it is clear that there is not much > support for it, much less consensus. That's progress because it now > seems clear to me that the path proposed (keep only v2) isn't the right > answer. > > Let's reflect a bit on some of the other progress that I think has been > made: > > 1) Greater understanding and documentation of the v3 API effort > > It has led to a larger group of people taking a much closer look at what > has been done with the v3 API so far. That has widened the net for > feedback on what else should be done before we could call it done. > > Chris has put together an excellent page with the most comprehensive > overview of the v3 API effort that I've seen. I think this is very > helpful: > > http://ozlabs.org/~cyeoh/V3_API.html > > I still sense that the struggle with Compute v3 is the lack of documentation for contributor developers but also especially end users so that we could get feedback early and often. My original understanding, passed by word-of-mouth, was that the goal for v3 was to define an expanded core that nearly all deployers could confidently put into production to serve their users needs. Since there's no end-user-sympathetic documentation, we learned a bit too much about how it's made, that supposedly it's implemented with "all extensions" -- a revelation that I'd still prefer to be protected from. :) Or possibly I don't understand. But the thing is, as a user advocate I shouldn't need to know that. I should know what it does and what benefits it holds. I recently had to write a paragraph about v3 for the Operations Guide, and it was really difficult to write because of the conversational nature of the discussion. Worse still, it was difficult to tell a deployer where their voice could be best heard. I went with "respond on the user survey." I still sense we need to ensure we have data from users (deployers and end users) and that won't be available until May. > 2) Expansion on ideas to ease long term support of APIs > > Thinking through this has led to a lot of deep thought about what > changes we can make to support an API for a longer period of time. > These are all ideas that can be applied to v3: > > - minor-versions for the core API and what changes would be > considered acceptable under that scheme > > - how we can make significant changes that normally are not > backwards compatible optional so that clients can declare > support for them, easing the possible future need for another > major API revision. > > 3) New ideas to ease keeping both v2 and v3 > > There has been some excellent input from those that have been working on > the v3 API with some new ideas for how we can lessen the burden of > keeping both APIs long term. I'm personally especially interested in > the "v2.1" approach where v2 turns into code that transforms requests > and responses to/from v3 format. More on that here: > > http://ozlabs.org/~cyeoh/V3_API.html#v2_v3_dual_maintenance > > > What I'd like to do next is work through a new proposal that includes > keeping both v2 and v3, but with a new added focus of minimizing the > cost. This should include a path away from the dual code bases and to > something like the "v2.1" proposal. > I'd like to make a better API and I think details about this proposal helps us with that goal. I'd like the effort to continue but I'd like an additional focus during the Icehouse timeframe to write end user and SDK dev docs and to listen to the user survey respondents. Thanks Russell and Chris for the mega-efforts here. It matters and you're fighting the good fight. Anne > > Thank you all for your participation on this topic. It has been quite > controversial, but the API we expose to our users is a really big deal. > I'm feeling more and more confident that we're coming through this with > a much better understanding of the problem space overall, as well as a > better plan going forward than we had a few weeks ago. > > -- > Russell Bryant > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On 03/03/2014 12:32 PM, Russell Bryant wrote: > There has been quite a bit of discussion about the future of the v3 API > recently. :-) Since this proposal was posted, it is clear that there is not much support for it, much less consensus. That's progress because it now seems clear to me that the path proposed (keep only v2) isn't the right answer. Let's reflect a bit on some of the other progress that I think has been made: 1) Greater understanding and documentation of the v3 API effort It has led to a larger group of people taking a much closer look at what has been done with the v3 API so far. That has widened the net for feedback on what else should be done before we could call it done. Chris has put together an excellent page with the most comprehensive overview of the v3 API effort that I've seen. I think this is very helpful: http://ozlabs.org/~cyeoh/V3_API.html 2) Expansion on ideas to ease long term support of APIs Thinking through this has led to a lot of deep thought about what changes we can make to support an API for a longer period of time. These are all ideas that can be applied to v3: - minor-versions for the core API and what changes would be considered acceptable under that scheme - how we can make significant changes that normally are not backwards compatible optional so that clients can declare support for them, easing the possible future need for another major API revision. 3) New ideas to ease keeping both v2 and v3 There has been some excellent input from those that have been working on the v3 API with some new ideas for how we can lessen the burden of keeping both APIs long term. I'm personally especially interested in the "v2.1" approach where v2 turns into code that transforms requests and responses to/from v3 format. More on that here: http://ozlabs.org/~cyeoh/V3_API.html#v2_v3_dual_maintenance What I'd like to do next is work through a new proposal that includes keeping both v2 and v3, but with a new added focus of minimizing the cost. This should include a path away from the dual code bases and to something like the "v2.1" proposal. Thank you all for your participation on this topic. It has been quite controversial, but the API we expose to our users is a really big deal. I'm feeling more and more confident that we're coming through this with a much better understanding of the problem space overall, as well as a better plan going forward than we had a few weeks ago. -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On 03/03/2014 06:32 PM, Russell Bryant wrote: > There has been quite a bit of discussion about the future of the v3 API > recently. There has been growing support for the idea that we should > change course and focus on evolving the existing v2 API instead of > putting out a new major revision. This message is a more complete > presentation of that proposal that concludes that we can do what we > really need to do with only the v2 API. > > Keeping only the v2 API requires some confidence that we can stick with > it for years to come. We don't want to be revisiting this any time > soon. This message addresses a bunch of different questions about how > things would work if we only had v2. > > 1) What about tasks? > > In some cases, the proposed integration of tasks is backwards > compatible. A task ID will be added to a header. The biggest point of > debate was if and how we would change the response for creating a > server. For tasks in v2, we would not change the response by default. > The task ID would just be in a header. However, if and when the client > starts exposing version support information, we can provide an > alternative/preferred response based on tasks. > > For example: > >Accept: application/json;type=task > I feel that ability to expose tasks is the single most important thing we need to do from the API semantics standpoint, unless we redesign the API from scratch (see below). Just looking at how awkward and edge-case ridden the interaction between components that use each others APIs in OpenStack is, should be enough to convince anyone that this needs fixing. I am not sure if "tasks" will solve this, but the fact that we have tried to solve this in several different ways up until now, and the effort was being driven by large deployers, mostly tells me that this is an issue people need solved. >From that point of view - if we can do this with V2 we absolutely should. > 2) Versioning extensions > > One of the points being addressed in the v3 API was the ability to > version extensions. In v2, we have historically required new API > extensions, even for changes that are backwards compatible. We propose > the following: > > - Add a version number to v2 API extensions > - Allow backwards compatible changes to these API extensions, > accompanied by a version number increase > - Add the option to advertise an extension as deprecated, which can be > used for all those extensions created only to advertise the availability > of new input parameters > > 3) Core versioning > > Another pain point in API maintenance has been having to create API > extensions for every small addition to the core API. We propose that a > version number be exposed for the core API that exposes the revision of > the core API in use. With that in place, backwards compatible changes > such as adding a new property to a resource would be allowed when > accompanied by a version number increase. > > With versioning of the core and API extensions, we will be able to cut > down significantly on the number of changes that require an API > extension without sacrificing the ability of a client to discover > whether the addition is present or not. > The whole extensions vs. core discussion has been confusing me since the beginning, and I can't say it has changed much. After thinking about this for some time I've decided :) that I think Nova needs 2 APIs. Amazon EC2 was always meant to be exposed as a web service to it's users and having a REST API that exposes "resources" without actually going into details about what is happening is fine, and it's fine for people using Nova in a similar manner. It is clear from this that I think the Nova API borrows a lot from EC2. But I think nova would benefit from having a lower level API, just as a well designed software library would, that let's people build services on top of it, that might provide different stability guarantees, as it's customers would not be cloud applications, but rather other cloud infrastructure. I think that having things tired in this manner would answer a lot of questions about what is and isn't "core" and what we mean by "extensions". If I were to attempt a new API for Nova - I would start from the above. > 4) API Proxying > > We don't see proxying APIs as a problem. It is the cost we pay for > choosing to split apart projects after they are released. We don't > think it's fair to break users just because we have chosen to split > apart the backend implementation. > > Further, the APIs that are proxied are frozen while those in the other > projects are evolving. We believe that as more features are available > only via the native APIs in Cinder, Glance, and Neutron, users will > naturally migrate over to the native APIs. > > Over time, we can ensure clients are able to query the API without the > need to proxy by adding new formats or extensions that don't return data > that needed to be proxied. > Proxying is fine, and c
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
+1 on the v2 on v3 proxy idea. Since none of other suggestions lead to easy decision, we might pay more attention here, especially when there are surely stackers, including myself, willing to provide solid contribution to it. Further and detailed discussion on technical feasibility, work sizing, etc. does help. 发自139邮箱mail.10086.cn The following is the content of the forwarded email From：Sean Dague To：openstack-dev Date：2014-03-04 20:49:03 Subject：Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API So this thread is getting deep again, as I expect they all will, so I39m just going to top post and take the ire for doing so. I also want to summarize what I39ve seen in the threads so far: v2 needed forever - if I do a sentiment analysis here looking at the orgs people are coming from, most of this is currently coming from Rackspace folks (publicly). There might be many reasons for this, one of which is the fact that they39ve done a big transition in the near past (between *not openstack* and Openstack), and felt that pain. Understanding that pain is good. It is interesting that Phil actually brings up a completely different issue from the HP cloud side, which is the amount of complaints they are having to field about how terrible the v2 API is. HP has actually had an OpenStack cloud public longer than Rackspace. So this feedback shouldn39t be lost. So I feel like while some deployers have expressed no interest in moving forward on API, others can39t get there soon enough. Which makes me think a lot about point 4. As has already been suggested we could actually make v2 a proxy to v3. And just like with images and volumes, it becomes frozen in Juno, and people that want more features will move to the v3 API. Just like with other services. This requires internal cleanups as well. However it wouldn39t shut down future evolution of the interface. Nova really has 4 interfaces today * Nova v2 JSON * Nova v2 XML * Nova v3 JSON * EC2 I feel like if we had to lose one to decrease maintenance cost, Nova v2 XML is the one to lose. And if we did, v2 on v3 isn39t the craziest thing in the world. It39s not free, but neither is the backport. I also feel like the code duplication approach for v2 and v3 was taken because the Glance folks had a previously bad experience on common code with 2 APIs. However, their surface was small enough that the dual code trees were fine. Nova is a beast in this regard. I also feel like more folks are excited about working on the v2 on v3 approach, given that Kenichi is already prototyping on it. And it is important to have excitement in this work as well. The human factor, about the parts that people want to work on, is critical to the success of Nova as a project. This project has always been about coming up with the best ideas, regardless of where they came from. So I39d like to make sure people aren39t making up their mind early on this, and have retreated to the idea of winning this discussion. The outcome of this is important. Because it basically sets up the framework of how we work on improving the Nova interface... for a long time. I also feel like the 2 complaints that come up on v3 repeatedly, that it39s a bunch of copy and paste, and that it doesn39t implement anything new, are kind of disingenuous. Because those were explicit design constraints imposed and agreed on by the nova core team on the approach. And if those are the complaints on the approach, why weren39t those brought up in advance? At the end of the day, I39m on the fence. Because what I actually want is somewhat orthogonal to all of this. Which is a real discoverable interface, and substantially less optional Nova core. And I39ve yet to figure out which approach sets us up for doing that better. -Sean On 03/03/2014 12:32 PM, Russell Bryant wrote: > There has been quite a bit of discussion about the future of the v3 API > recently. There has been growing support for the idea that we should > change course and focus on evolving the existing v2 API instead of > putting out a new major revision. This message is a more complete > presentation of that proposal that concludes that we can do what we > really need to do with only the v2 API. > > Keeping only the v2 API requires some confidence that we can stick with > it for years to come. We don39t want to be revisiting this any time > soon. This message addresses a bunch of different questions about how > things would work if we only had v2. > > 1) What about tasks? > > In some cases, the proposed integration of tasks is backwards > compatible. A task ID will be added to a header. The biggest point of > debate was if and how we would change the response for creating a > server. For tasks in v2, we would not change the response by default. > The task ID would just be in a header. However, if and when the client > starts exposing version support information,
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On Mon, Mar 03, 2014 at 12:32:04PM -0500, Russell Bryant wrote: > 5) Capitalization and Naming Consistency > > Some of the changes in the v3 API included changes to capitalization and > naming for improved consistency. If we stick with v2 only, we will not > be able to make any of these changes. However, we believe that not > breaking any existing clients and not having to maintain a second API is > worth not making these changes, or supporting some indirection to > achieve this for newer clients if we decide it is important. While naming inconsistencies are not pretty, they are ultimately merely style issues. My view from working on API design in other projects like libvirt is that "cleaning up style issues" is never an acceptable justification for API breakage that would affect downstream consumers of an API. The top priority in a public API, is doing what's best for users of the API, since the pain suffered by users upon API breakage usually far outweighs the pain suffered by the people maintaining the API. So for me cleaning up style inconsistencies in v2 API is not a point that's relevant in making a decision to do a v3 API. It is merely a pleasant consequence if there are other compelling reasons that make a v3 API unavoidable. > 6) Response Code Consistency and Correctness > > The v2 API has many places where the response code returned for a given > operation is not strictly correct. For example a 200 is returned when a > 202 would be more appropriate. Correcting these issues should be > considered for improving the future use of the API, however there does > not seem to be any support for considering this a critical problem right > now. There are two approaches that can be taken to improve this in v2: > > Just fix them. Right now, we return some codes that imply we have dealt > with a request, when all we have done is queue it for processing (and > vice versa). In the future, we may change the backend in such a way that > a return code needs to change to continue to be accurate anyway. If we > just assume that return codes may change to properly reflect the action > that was taken, then we can correct existing errors and move on. > Accept them as wrong but not critically so. With this approach, we can > strive for correctness in the future without changing behavior of our > existing APIs. Nobody seems to complain about them right now, so > changing them seems to be little gain. If the client begins exposing a > version header (which we need for other things) then we could > alternately start returning accurate codes for those clients. > > The key point here is that we see a way forward with this in the v2 API > regardless of which path we choose. My experiance with libvirt is that error code reporting changes for APIs are something that's acceptable to do if you are adding detection of new error scenarios. The key thing is to document what applications can expect from error codes, so that even if new error codes are introduced, applications will still at least detect the scenario as an error, and not success. > 10) Conclusion and Proposal > > The v3 API effort has produced a lot of excellent work. However, the > majority opinion seems to be that we should avoid the cost of > maintaining two APIs if at all possible. We should apply what has been > learned to the existing API where we can and focus on making v2 > something that we can continue to maintain for years to come. > > We recognize and accept that it is a failure of Nova project leadership > that we did not come to this conclusion much sooner. We hope to have > learned from the experience to help avoiding a situation like this > happening again in the future. > > Please provide your input on this proposal, even if it is just agreement > with this as the way forward. What is the suggested plan of action & timeframe if this proposal is adopted ? Is there a sense of urgency to resolving this ? eg to avoid shipping v3 in Icehouse & thus avoid getting locked into supporting it ? If the latter would there be any value in keeping v3 code in the tree, but adding a code patch that forceably disables it (with no config param to turn it back on), such that we can wait until Atlanta to have a face- to-face discussion on its fate before taking the ultimate step of deleting all its code ? Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On 03/04/2014 08:14 AM, Daniel P. Berrange wrote: > On Tue, Mar 04, 2014 at 07:49:03AM -0500, Sean Dague wrote: >> So this thread is getting deep again, as I expect they all will, so I'm >> just going to top post and take the ire for doing so. >> >> I also want to summarize what I've seen in the threads so far: >> >> v2 needed forever - if I do a sentiment analysis here looking at the >> orgs people are coming from, most of this is currently coming from >> Rackspace folks (publicly). There might be many reasons for this, one of >> which is the fact that they've done a big transition in the near past >> (between *not openstack* and Openstack), and felt that pain. >> Understanding that pain is good. >> >> It is interesting that Phil actually brings up a completely different >> issue from the HP cloud side, which is the amount of complaints they are >> having to field about how terrible the v2 API is. HP has actually had an >> OpenStack cloud public longer than Rackspace. So this feedback shouldn't >> be lost. >> >> So I feel like while some deployers have expressed no interest in moving >> forward on API, others can't get there soon enough. >> >> Which makes me think a lot about point 4. As has already been suggested >> we could actually make v2 a proxy to v3. And just like with images and >> volumes, it becomes frozen in Juno, and people that want more features >> will move to the v3 API. Just like with other services. > >> This requires internal cleanups as well. However it wouldn't shut down >> future evolution of the interface. >> >> Nova really has 4 interfaces today >> * Nova v2 JSON >> * Nova v2 XML >> * Nova v3 JSON >> * EC2 >> >> I feel like if we had to lose one to decrease maintenance cost, Nova v2 >> XML is the one to lose. And if we did, v2 on v3 isn't the craziest thing >> in the world. It's not free, but neither is the backport. > > A proxy of v2 onto v3 is appealing, but do we think we have good enough > testing of v2 to ensure that any proxy impl is bug-for-bug compatible > with the original native v2 implementation ? Avoiding breakage of client > apps is to me the key reason for keeping v2 around, so we'd want very > high confidence that any proxy impl is functionally identical with the > orginal impl. So because of Nova objects, Icehouse v2 isn't bug-for-bug compatible with Havana v2 anyway. The API isn't isolated enough from the internals at this point to actually provide those guaruntees in v2. It doesn't seem unreasonable to me that v2 could be as good a guaruntee as the one we have. A focus on increased coverage of the v2 API in tempest would help. It would be great if the folks that are really interested in keeping stable v2 (and ensuring we lock down the behavior) would contribute there. > If we want to proxy v2 onto v3, then by that same argument should we > be proxying EC2 onto v3 as well. ie Nova v3 JSON be the only supported > API, and every thing else just be a proxy, potentially maintained out > of tree from main nova codebase. There are some fundamental issues around ids that make that problematic (and that's only one of the concerns I know about). EC2 as a proxy (out of tree) was discussed 3 cycles ago, and no one stepped up. And the result has been no one working on EC2. So I'm actually really concerned that if we don't reach a conclusion around Nova API that people are excited about working on, we see the same core collapse there as we saw on EC2. -Sean -- Sean Dague Samsung Research America s...@dague.net / sean.da...@samsung.com http://dague.net signature.asc Description: OpenPGP digital signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On Tue, Mar 04, 2014 at 07:49:03AM -0500, Sean Dague wrote: > So this thread is getting deep again, as I expect they all will, so I'm > just going to top post and take the ire for doing so. > > I also want to summarize what I've seen in the threads so far: > > v2 needed forever - if I do a sentiment analysis here looking at the > orgs people are coming from, most of this is currently coming from > Rackspace folks (publicly). There might be many reasons for this, one of > which is the fact that they've done a big transition in the near past > (between *not openstack* and Openstack), and felt that pain. > Understanding that pain is good. > > It is interesting that Phil actually brings up a completely different > issue from the HP cloud side, which is the amount of complaints they are > having to field about how terrible the v2 API is. HP has actually had an > OpenStack cloud public longer than Rackspace. So this feedback shouldn't > be lost. > > So I feel like while some deployers have expressed no interest in moving > forward on API, others can't get there soon enough. > > Which makes me think a lot about point 4. As has already been suggested > we could actually make v2 a proxy to v3. And just like with images and > volumes, it becomes frozen in Juno, and people that want more features > will move to the v3 API. Just like with other services. > This requires internal cleanups as well. However it wouldn't shut down > future evolution of the interface. > > Nova really has 4 interfaces today > * Nova v2 JSON > * Nova v2 XML > * Nova v3 JSON > * EC2 > > I feel like if we had to lose one to decrease maintenance cost, Nova v2 > XML is the one to lose. And if we did, v2 on v3 isn't the craziest thing > in the world. It's not free, but neither is the backport. A proxy of v2 onto v3 is appealing, but do we think we have good enough testing of v2 to ensure that any proxy impl is bug-for-bug compatible with the original native v2 implementation ? Avoiding breakage of client apps is to me the key reason for keeping v2 around, so we'd want very high confidence that any proxy impl is functionally identical with the orginal impl. If we want to proxy v2 onto v3, then by that same argument should we be proxying EC2 onto v3 as well. ie Nova v3 JSON be the only supported API, and every thing else just be a proxy, potentially maintained out of tree from main nova codebase. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On Tue, Mar 04, 2014 at 08:46:32AM +1100, Michael Still wrote: > I think its also pretty unfair on the people who put a lot of work > into the v3 API. We're seriously going to delete their code after they > put a year into it? > > To me OpenStack isn't just the users, its also the development > community. I think we do measurable harm to that development community > by doing this. We're teaching people that having a blessed plan that > was discussed at a summit is not enough to reassure their management > chain that they're not wasting time developing something. That worries > me a lot. I don't believe we should see this as a complete waste of time - it was really a learning experience. Certainly it would have been better if we had been able to not invest so much in v3, but the actual effort of doing the work clearly brought us to a level of understanding of the problemspace that we just didn't have when the work first started. I strongly disagree with your suggestion that we can never change our mind about a blueprint once it has been discussed at summit / approved. Certainly we should try and avoid it happening too often, but being able & willing to change our minds on ideas, when new information comes to light is a great benefit to the project over the long term. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
So this thread is getting deep again, as I expect they all will, so I'm just going to top post and take the ire for doing so. I also want to summarize what I've seen in the threads so far: v2 needed forever - if I do a sentiment analysis here looking at the orgs people are coming from, most of this is currently coming from Rackspace folks (publicly). There might be many reasons for this, one of which is the fact that they've done a big transition in the near past (between *not openstack* and Openstack), and felt that pain. Understanding that pain is good. It is interesting that Phil actually brings up a completely different issue from the HP cloud side, which is the amount of complaints they are having to field about how terrible the v2 API is. HP has actually had an OpenStack cloud public longer than Rackspace. So this feedback shouldn't be lost. So I feel like while some deployers have expressed no interest in moving forward on API, others can't get there soon enough. Which makes me think a lot about point 4. As has already been suggested we could actually make v2 a proxy to v3. And just like with images and volumes, it becomes frozen in Juno, and people that want more features will move to the v3 API. Just like with other services. This requires internal cleanups as well. However it wouldn't shut down future evolution of the interface. Nova really has 4 interfaces today * Nova v2 JSON * Nova v2 XML * Nova v3 JSON * EC2 I feel like if we had to lose one to decrease maintenance cost, Nova v2 XML is the one to lose. And if we did, v2 on v3 isn't the craziest thing in the world. It's not free, but neither is the backport. I also feel like the code duplication approach for v2 and v3 was taken because the Glance folks had a previously bad experience on common code with 2 APIs. However, their surface was small enough that the dual code trees were fine. Nova is a beast in this regard. I also feel like more folks are excited about working on the v2 on v3 approach, given that Kenichi is already prototyping on it. And it is important to have excitement in this work as well. The human factor, about the parts that people want to work on, is critical to the success of Nova as a project. This project has always been about coming up with the best ideas, regardless of where they came from. So I'd like to make sure people aren't making up their mind early on this, and have retreated to the idea of winning this discussion. The outcome of this is important. Because it basically sets up the framework of how we work on improving the Nova interface... for a long time. I also feel like the 2 complaints that come up on v3 repeatedly, that it's a bunch of copy and paste, and that it doesn't implement anything new, are kind of disingenuous. Because those were explicit design constraints imposed and agreed on by the nova core team on the approach. And if those are the complaints on the approach, why weren't those brought up in advance? At the end of the day, I'm on the fence. Because what I actually want is somewhat orthogonal to all of this. Which is a real discoverable interface, and substantially less optional Nova core. And I've yet to figure out which approach sets us up for doing that better. -Sean On 03/03/2014 12:32 PM, Russell Bryant wrote: > There has been quite a bit of discussion about the future of the v3 API > recently. There has been growing support for the idea that we should > change course and focus on evolving the existing v2 API instead of > putting out a new major revision. This message is a more complete > presentation of that proposal that concludes that we can do what we > really need to do with only the v2 API. > > Keeping only the v2 API requires some confidence that we can stick with > it for years to come. We don't want to be revisiting this any time > soon. This message addresses a bunch of different questions about how > things would work if we only had v2. > > 1) What about tasks? > > In some cases, the proposed integration of tasks is backwards > compatible. A task ID will be added to a header. The biggest point of > debate was if and how we would change the response for creating a > server. For tasks in v2, we would not change the response by default. > The task ID would just be in a header. However, if and when the client > starts exposing version support information, we can provide an > alternative/preferred response based on tasks. > > For example: > >Accept: application/json;type=task > > 2) Versioning extensions > > One of the points being addressed in the v3 API was the ability to > version extensions. In v2, we have historically required new API > extensions, even for changes that are backwards compatible. We propose > the following: > > - Add a version number to v2 API extensions > - Allow backwards compatible changes to these API extensions, > accompanied by a version number increase > - Add the option to advertise an extension as depreca
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On Mon, 03 Mar 2014 12:32:04 -0500 Russell Bryant wrote: > There has been quite a bit of discussion about the future of the v3 > API recently. There has been growing support for the idea that we > should change course and focus on evolving the existing v2 API > instead of putting out a new major revision. This message is a more > complete presentation of that proposal that concludes that we can do > what we really need to do with only the v2 API. > So one thing which I think is relevant to this discussion is that we are not comparing two theoretical paths. On one hand we have a v2 API related proposal which involves lot of backporting. I think there is a lack of appreciation in this proposal over how much effort would be involved to achieve this (and I don't think anyone is really putting their hand up to do the work required in the long term) as well as the risk involved in accidental changes to the V2 API. As a very rough guide of effort I think around 400+ patches have merged (not include copy/paste part 1 type patches) just in the Nova tree for the V3 API over Havana and Icehouse. This does not count the python novaclient or tempest support which would have to be implemented at some point to support an API changes which were backported. On the other hand we have the V3 API which with the exception of the nova-network support is almost complete and merged. And we have a much better idea of the effort as well as several people who want to and have the time to work on it as well as being willing to help reduce the dual maintenance overhead. To try to clarify what the V3 API would mean for users, developers and deployers I have put some information together here: http://ozlabs.org/~cyeoh/V3_API.html It also covers some approaches for reducing the dual maintenance issues in the short term and removing them in the long term. As well as a possible strategy for long term handling of significant API changes. Although having a V3 API does certainly involve some dual maintenance concerns about making internal Nova changes whilst supporting both the V2 and V3 API code. And so far in this discussion I don't think we have really managed to quantify what the dual maintenance overhead actually is (although the document above tries to). Also I don't think we should ignore the extra maintenance burdens discussed in in the linked document of attempting to evolve the V2 API and keep the old V2 API code base (which I think we may be able to separate conceptually from keeping support for the V2 API). Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On 03/03/2014 05:35 PM, Michael Still wrote: > On Tue, Mar 4, 2014 at 4:32 AM, Russell Bryant wrote: >> There has been quite a bit of discussion about the future of the v3 API >> recently. There has been growing support for the idea that we should >> change course and focus on evolving the existing v2 API instead of >> putting out a new major revision. This message is a more complete >> presentation of that proposal that concludes that we can do what we >> really need to do with only the v2 API. > > Ok, so process question. > > Who makes the final decision here? I think this is the most > contentious decision I can remember in nova, so I'm not sure how we > want to go about actually locking a decision in. Is that something the > PTL does after listening to everyone's thoughts? Or is there some sort > of vote? I don't have a strong opinion here, but I think we do need > some way to decide when this debate is over. My understanding is that it is ultimately a PTL call. However, I also believe that it is the job of the PTL to do as much as possible to drive a discussion to consensus, as opposed to acting as a dictator. I am absolutely *not* interested in making this decision alone. I think that would be very bad for the project. We'll take the time necessary to work through it and use a vote if absolutely necessary, but it won't be a PTL declaration. The intent behind this proposal is to drive the discussion forward. It's an attempt to lay out more concretely what the world would look like if we chose this route. If it's not good enough, why? Have an alternative concrete proposal to address concerns? Get it on the table. All of that is progress towards a resolution. I know this is quite the controversial subject given how much has been invested. I believe that at the end we'll be even more confident that we're headed in the best direction for the future of Nova. -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On Mar 3, 2014, at 1:25 PM, Vishvananda Ishaya wrote: > This seems like a reasonable and well thought out approach but It feels > like we are removing our ability to innovate. I know we are worried about > maintaining multiple APIs, but I’m still leaning towards putting the v3 > API out and just keeping v2 around for a long time. Yes, its a maintenance > burden, but if we aren’t adding a lot of features to v2, I don’t know if > it is really THAT bad. As an SDK developer this affects me and all other SDK developers directly. We need a guiding principle. To borrow a chestnut from Linux, “We do not break userspace!” Now DO NOT take this to mean that I’m against a v3 API. I truly understand the need to find a way forward with APIs to make them more consistent and open to innovation. Read on. You know what happens when you break a developer’s application? They start to examine their options w.r.t. where they deploy those applications. If OpenStack is going to deprecate any API I depend on and break my app, the time between now and that EOL is the perfect opportunity to research other platforms. And if they go somewhere else, there’s a good chance they’re not coming back. IMO v3 would be doable. But there would need to be commitment from the developers of OpenStack to support v2 until it can be proven that it’s no longer in use. “support”, “proven”, and “use” would all need to be defined but the point is that apps would not be broken, period. IMO v2 as proposed here would be doable. Obviously there should be no problem here. Some (probably not all) consequences to applications and SDKs of going the v3 route: 1. You need to accept that application developers (whether they’re using SDKs or not) might not switch to v3, ever. If there isn’t a compelling reason to upgrade, they likely won’t. It could be that v3 is just a bridge for OpenStack developers only to get to a v4 API that has enough compelling features for the rest of the world to upgrade. I want to draw an analogy to the whole Python 3 thing but, honestly, I’m not familiar enough with it and I think I’d be walking into a minefield around here. ;) 2. The copy/paste duplicate code that you'd have in OpenStack for v2 and v3? That’s probably going to be copy/paste duplicate code in the applications/SDKs too. More code. More bugs. More maintenance. Worse developer experience. 3. Any hope of applications/SDKs upgrading to v3 depends heavily on how well the delta is documented. It’s not enough to say that there is a v3 and have a handful of high-level bullets points saying what changed. We need an extremely fine level of detail about what has changed per parameter/method call/status code/whatever and what we should be using in v3 instead of what was in v2. Without this, upgrading is a non-starter. Some (probably not all) consequences to the SDKs of going the v2 route: 1. These likely all fall on the OpenStack services side. More difficult to innovate? Less consistent? etc. Personally, I’m pretty evenly split on which route to take. If the v3 route was taken and a commitment is made to supporting v2 indefinitely, I could get on board with that. If the v2 route was taken, it’s a no brainer. I trust that the OpenStack developers will make the call in the best interests of the application developers and operators that depend on the Nova API day in and day out. Guided by the principle that we do not break userspace. It’s a huge sign of the maturity and stability of OpenStack that this discussion is happening. Thank you, Everett ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On Tue, Mar 4, 2014 at 10:50 AM, Andrew Laski wrote: > Totally understand. But I would contend that it's because v3 is mostly > minor changes that we're even having this conversation. In all fairness, that's because that's what we asked them to do when we approved the blueprint. Michael -- Rackspace Australia ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
> -Original Message- > From: Russell Bryant [mailto:rbry...@redhat.com] > Sent: Tuesday, March 04, 2014 7:31 AM > To: openstack-dev@lists.openstack.org > Subject: Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API > > On 03/03/2014 04:25 PM, Vishvananda Ishaya wrote: > > This seems like a reasonable and well thought out approach but It > > feels like we are removing our ability to innovate. > > I think part of this exercise is trying to think through what freedom > we *would* have to innovate in v2 with a set of changes to code an > policy. That's what most of the proposal is about. If we still don't > have the freedom we need, that's a problem and is what we need to > figure out before this s final. > > Are there particular cases that you feel are missing in a v2 world > described by the proposal? > > > I’m worried that this is just delaying solving the inconsistency > > issues to some future date. > > It depends which consistency issues you're talking about. Many of > them I think we can fix today. Others the argument being made is that > they just aren't nearly important enough to fix yet unless they're > coming with a completely new API. > > Which inconsistencies are you concerned about in particular? What did > we miss in the proposal or which part specifically do you disagree with? I guess it is an issue about "Naming Consistency". I can see the proposal, but I feel now it is a little early decision because we have 2 months before the next summit and can investigate how to keep the maintenance cost low. First of all, I feel the big issue of v3 API is backward incompatibility changes for "Naming Consistency" since v2 API. That is very important point and we should maintain both APIs in long time, because users may use current API over 2 years at least. (from Joe and Thomas mails, thanks) As one possible solution, Thierry and Alex said v2.1 API. In v2.1 API, backward incompatibility issues related to "Naming Consistency" would not happen. API names(URLs, API attributes) would not be changed since current v2 API. Instead, the other features of v3(API extension customizability, strong input validation, code clean) are enabled. I created a prototype(https://review.openstack.org/#/c/77105/) for doing it. That translates a request of v2 format to one of v3 and passes it to v3 API method. After API method operation, it translates the response to v2 format again. That seems not so difficult, and we just need to contain the gap between v2 and v3 in Nova tree. If going forward with v2.1 API, I feel we will be able to remove current v2 API code because we will have backward compatible API. I think nobody believes current v2 API is better than v3 API. The main matter is the maintenance cost. So again, I feel it is better to clarify what is the concerned maintenance cost. After clarifying the cost, we can think how to reduce it. Possibly, that would be a latent issue of API development of not only v3 but also current v2. There is not any issue we can not solve :-) Thanks Ken'ichi Ohmichi ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On 03/03/14 at 06:17pm, Jay Pipes wrote: On Mon, 2014-03-03 at 17:56 -0500, Andrew Laski wrote: On 03/03/14 at 04:48pm, Jay Pipes wrote: >On Mon, 2014-03-03 at 15:24 -0500, Russell Bryant wrote: >> On 03/03/2014 02:59 PM, Jay Pipes wrote: >> > -1 from me. Sounds like a way to avoid badly needed change and >> > innovation in the API. When, for example, would we be able to propose a >> > patch that removed API extensions entirely? >> >> v3 didn't address this at all, anyway. I'm not even sure there's >> consensus on it. In any case, I think it's tangential and only relevant >> if we're starting with a new API. > >I guess I am saying that if the ostensibly "limited" changes and >standardization of Chris and others' V3 API work has now been decided is >too risky to use or of not enough value to users, then the chance that a >brand new API version seeing the light of day is pretty low. And that is >disappointing to me. Feel free to tell me I'm nuts, though. I'd happily >exchange my skepticism for some healthy optimism. This isn't about the v3 API changes being too risky or not of value to users. What has been most concerning about the v3 API is the amount of code it has introduced into the Nova tree and the amount of testing work that has been required. Doing this split once has been a pain but overall has been manageable up to this point, IMO. But at this point we don't have any reasonable idea on when v2 can be removed, so it's potentially with us for good. And while v3 improves the story around making API changes it doesn't address some of the things that caused us to look at v3 in the first place. So we're potentially setting ourselves up to eventually hold three or more API versions in tree. Before going down that road we came up with a proposal that allows us to move forward in much the same way as v3 but with a lower maintenance burden. What I would like to see is answers for how v3 can address some of the questions that are being asked of this proposal, such as removing API extensions entirely. Or how would we add entirely new concepts like tasks? Any changes that we make to V2 will prolong its life. I think you will agree with this, yes? My main point is a philosophical one: if we scrap the V3 effort and decide to instead make changes to V2, then by nature, I think we will delay the ability to make more significant changes to the API. I'd like to work on a brand new (v4?) Compute API that dramatically changes the way that compute operations are made. If we make changes to the V2 API and extend its life, I don't have as much incentive to work on a brand new API version, since I've already seen that even minor changes like those in the V3 API can't make it into mainline in 3 release cycles. Like I said, it's philosophical. Totally understand. But I would contend that it's because v3 is mostly minor changes that we're even having this conversation. Personally I love the changes that v3 provides, and want to see our API evolve and mature in the direction it's heading. But I also want it to have real value. Beyond a few renames and return code fixes what does it do that we can't do with v2? That's the question I'm still asking myself. > >> > The inconsistent naming, capitalization, numerous worthless or pointless >> > API extensions, ability to do similar or identical things in different >> > ways, and the incorrect result/response codes makes the Nova API look >> > immature and clownish. I'd love to see a competing proposal to this that >> > looks towards the future and a day when we can be proud of the Compute >> > API as a real differentiator vs. the EC2 API. As it stands, the v2 REST >> > API just makes OpenStack Compute look subpar at best, IMO. >> >> Much of what you discuss is addressed in the document. > >Well, it's discussed in the document... in as much to say "we won't >really change any of these things..." > >> I think the differentiation that you want comes from a new ground-up >> API, and not what's being discussed here (v3, or further evolution of v2). > >Yes, but my concern about this proposal is that it reinforces the status >quo and in so doing, slows down the pace of innovation at the API level. >And, a slow pace of API innovation in turn inhibits innovation at layers >of the code beneath the API. What innovation is being slowed down? This proposal was put together to compare against the current v3 effort and see if we can tease out these things. What can we do in v3 that we can't do in v2? See above. It's about the statement that this proposal makes to those developers who might have been considering contributing improvements to the Compute API. Best, -jay >> > Feel free to accuse me of wanting my cake and eating it, too. I guess >> > I'm just both hungry and impatient. >> >> Not that, exactly ... just ignoring some of the practicalities at hand, >> I think. > >Fair enough. :) > >Best, >-jay > > >___ >OpenS
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On Mon, 2014-03-03 at 17:56 -0500, Andrew Laski wrote: > On 03/03/14 at 04:48pm, Jay Pipes wrote: > >On Mon, 2014-03-03 at 15:24 -0500, Russell Bryant wrote: > >> On 03/03/2014 02:59 PM, Jay Pipes wrote: > >> > -1 from me. Sounds like a way to avoid badly needed change and > >> > innovation in the API. When, for example, would we be able to propose a > >> > patch that removed API extensions entirely? > >> > >> v3 didn't address this at all, anyway. I'm not even sure there's > >> consensus on it. In any case, I think it's tangential and only relevant > >> if we're starting with a new API. > > > >I guess I am saying that if the ostensibly "limited" changes and > >standardization of Chris and others' V3 API work has now been decided is > >too risky to use or of not enough value to users, then the chance that a > >brand new API version seeing the light of day is pretty low. And that is > >disappointing to me. Feel free to tell me I'm nuts, though. I'd happily > >exchange my skepticism for some healthy optimism. > > This isn't about the v3 API changes being too risky or not of value to > users. What has been most concerning about the v3 API is the amount of > code it has introduced into the Nova tree and the amount of testing work > that has been required. Doing this split once has been a pain but > overall has been manageable up to this point, IMO. But at this point we > don't have any reasonable idea on when v2 can be removed, so it's > potentially with us for good. And while v3 improves the story around > making API changes it doesn't address some of the things that caused us > to look at v3 in the first place. So we're potentially setting > ourselves up to eventually hold three or more API versions in tree. > Before going down that road we came up with a proposal that allows us to > move forward in much the same way as v3 but with a lower maintenance > burden. > > What I would like to see is answers for how v3 can address some of the > questions that are being asked of this proposal, such as removing API > extensions entirely. Or how would we add entirely new concepts like > tasks? Any changes that we make to V2 will prolong its life. I think you will agree with this, yes? My main point is a philosophical one: if we scrap the V3 effort and decide to instead make changes to V2, then by nature, I think we will delay the ability to make more significant changes to the API. I'd like to work on a brand new (v4?) Compute API that dramatically changes the way that compute operations are made. If we make changes to the V2 API and extend its life, I don't have as much incentive to work on a brand new API version, since I've already seen that even minor changes like those in the V3 API can't make it into mainline in 3 release cycles. Like I said, it's philosophical. > Personally I love the changes that v3 provides, and want to see our API > evolve and mature in the direction it's heading. But I also want it to > have real value. Beyond a few renames and return code fixes what does > it do that we can't do with v2? That's the question I'm still asking > myself. > > > > > >> > The inconsistent naming, capitalization, numerous worthless or pointless > >> > API extensions, ability to do similar or identical things in different > >> > ways, and the incorrect result/response codes makes the Nova API look > >> > immature and clownish. I'd love to see a competing proposal to this that > >> > looks towards the future and a day when we can be proud of the Compute > >> > API as a real differentiator vs. the EC2 API. As it stands, the v2 REST > >> > API just makes OpenStack Compute look subpar at best, IMO. > >> > >> Much of what you discuss is addressed in the document. > > > >Well, it's discussed in the document... in as much to say "we won't > >really change any of these things..." > > > >> I think the differentiation that you want comes from a new ground-up > >> API, and not what's being discussed here (v3, or further evolution of v2). > > > >Yes, but my concern about this proposal is that it reinforces the status > >quo and in so doing, slows down the pace of innovation at the API level. > >And, a slow pace of API innovation in turn inhibits innovation at layers > >of the code beneath the API. > > What innovation is being slowed down? This proposal was put together to > compare against the current v3 effort and see if we can tease out these > things. What can we do in v3 that we can't do in v2? See above. It's about the statement that this proposal makes to those developers who might have been considering contributing improvements to the Compute API. Best, -jay > >> > Feel free to accuse me of wanting my cake and eating it, too. I guess > >> > I'm just both hungry and impatient. > >> > >> Not that, exactly ... just ignoring some of the practicalities at hand, > >> I think. > > > >Fair enough. :) > > > >Best, > >-jay > > > > > >___ > >Op
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On 03/03/14 at 04:48pm, Jay Pipes wrote: On Mon, 2014-03-03 at 15:24 -0500, Russell Bryant wrote: On 03/03/2014 02:59 PM, Jay Pipes wrote: > -1 from me. Sounds like a way to avoid badly needed change and > innovation in the API. When, for example, would we be able to propose a > patch that removed API extensions entirely? v3 didn't address this at all, anyway. I'm not even sure there's consensus on it. In any case, I think it's tangential and only relevant if we're starting with a new API. I guess I am saying that if the ostensibly "limited" changes and standardization of Chris and others' V3 API work has now been decided is too risky to use or of not enough value to users, then the chance that a brand new API version seeing the light of day is pretty low. And that is disappointing to me. Feel free to tell me I'm nuts, though. I'd happily exchange my skepticism for some healthy optimism. This isn't about the v3 API changes being too risky or not of value to users. What has been most concerning about the v3 API is the amount of code it has introduced into the Nova tree and the amount of testing work that has been required. Doing this split once has been a pain but overall has been manageable up to this point, IMO. But at this point we don't have any reasonable idea on when v2 can be removed, so it's potentially with us for good. And while v3 improves the story around making API changes it doesn't address some of the things that caused us to look at v3 in the first place. So we're potentially setting ourselves up to eventually hold three or more API versions in tree. Before going down that road we came up with a proposal that allows us to move forward in much the same way as v3 but with a lower maintenance burden. What I would like to see is answers for how v3 can address some of the questions that are being asked of this proposal, such as removing API extensions entirely. Or how would we add entirely new concepts like tasks? Personally I love the changes that v3 provides, and want to see our API evolve and mature in the direction it's heading. But I also want it to have real value. Beyond a few renames and return code fixes what does it do that we can't do with v2? That's the question I'm still asking myself. > The inconsistent naming, capitalization, numerous worthless or pointless > API extensions, ability to do similar or identical things in different > ways, and the incorrect result/response codes makes the Nova API look > immature and clownish. I'd love to see a competing proposal to this that > looks towards the future and a day when we can be proud of the Compute > API as a real differentiator vs. the EC2 API. As it stands, the v2 REST > API just makes OpenStack Compute look subpar at best, IMO. Much of what you discuss is addressed in the document. Well, it's discussed in the document... in as much to say "we won't really change any of these things..." I think the differentiation that you want comes from a new ground-up API, and not what's being discussed here (v3, or further evolution of v2). Yes, but my concern about this proposal is that it reinforces the status quo and in so doing, slows down the pace of innovation at the API level. And, a slow pace of API innovation in turn inhibits innovation at layers of the code beneath the API. What innovation is being slowed down? This proposal was put together to compare against the current v3 effort and see if we can tease out these things. What can we do in v3 that we can't do in v2? > Feel free to accuse me of wanting my cake and eating it, too. I guess > I'm just both hungry and impatient. Not that, exactly ... just ignoring some of the practicalities at hand, I think. Fair enough. :) Best, -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On 3/3/2014 12:27 PM, Joe Gordon wrote: On Mon, Mar 3, 2014 at 9:32 AM, Russell Bryant wrote: There has been quite a bit of discussion about the future of the v3 API recently. There has been growing support for the idea that we should change course and focus on evolving the existing v2 API instead of putting out a new major revision. This message is a more complete presentation of that proposal that concludes that we can do what we really need to do with only the v2 API. Keeping only the v2 API requires some confidence that we can stick with it for years to come. We don't want to be revisiting this any time soon. This message addresses a bunch of different questions about how things would work if we only had v2. 1) What about tasks? In some cases, the proposed integration of tasks is backwards compatible. A task ID will be added to a header. The biggest point of debate was if and how we would change the response for creating a server. For tasks in v2, we would not change the response by default. The task ID would just be in a header. However, if and when the client starts exposing version support information, we can provide an alternative/preferred response based on tasks. For example: Accept: application/json;type=task The current https://wiki.openstack.org/wiki/APIChangeGuidelines says its OK add a new response header, so do we even need this? 2) Versioning extensions One of the points being addressed in the v3 API was the ability to version extensions. In v2, we have historically required new API extensions, even for changes that are backwards compatible. We propose the following: - Add a version number to v2 API extensions - Allow backwards compatible changes to these API extensions, accompanied by a version number increase - Add the option to advertise an extension as deprecated, which can be used for all those extensions created only to advertise the availability of new input parameters Can you elaborate on this last point. 3) Core versioning Another pain point in API maintenance has been having to create API extensions for every small addition to the core API. We propose that a version number be exposed for the core API that exposes the revision of the core API in use. With that in place, backwards compatible changes such as adding a new property to a resource would be allowed when accompanied by a version number increase. With versioning of the core and API extensions, we will be able to cut down significantly on the number of changes that require an API extension without sacrificing the ability of a client to discover whether the addition is present or not. ++, looks like we may need to update https://wiki.openstack.org/wiki/APIChangeGuidelines and make this clear to downstream users. 4) API Proxying We don't see proxying APIs as a problem. It is the cost we pay for choosing to split apart projects after they are released. We don't think it's fair to break users just because we have chosen to split apart the backend implementation. Further, the APIs that are proxied are frozen while those in the other projects are evolving. We believe that as more features are available only via the native APIs in Cinder, Glance, and Neutron, users will naturally migrate over to the native APIs. Over time, we can ensure clients are able to query the API without the need to proxy by adding new formats or extensions that don't return data that needed to be proxied. Can you expand on what this last paragraph means? While I agree in not breaking users. I assume this means we won't accept any new proxy APIs? 5) Capitalization and Naming Consistency Some of the changes in the v3 API included changes to capitalization and naming for improved consistency. If we stick with v2 only, we will not be able to make any of these changes. However, we believe that not breaking any existing clients and not having to maintain a second API is worth not making these changes, or supporting some indirection to achieve this for newer clients if we decide it is important. ++ 6) Response Code Consistency and Correctness The v2 API has many places where the response code returned for a given operation is not strictly correct. For example a 200 is returned when a 202 would be more appropriate. Correcting these issues should be considered for improving the future use of the API, however there does not seem to be any support for considering this a critical problem right now. There are two approaches that can be taken to improve this in v2: Just fix them. Right now, we return some codes that imply we have dealt with a request, when all we have done is queue it for processing (and vice versa). In the future, we may change the backend in such a way that a return code needs to change to continue to be accurate anyway. If we just assume that return codes may change to properly reflect the action that was taken, then we can correct existing errors and move on. Changing retu
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On Tue, Mar 4, 2014 at 4:32 AM, Russell Bryant wrote: > There has been quite a bit of discussion about the future of the v3 API > recently. There has been growing support for the idea that we should > change course and focus on evolving the existing v2 API instead of > putting out a new major revision. This message is a more complete > presentation of that proposal that concludes that we can do what we > really need to do with only the v2 API. Ok, so process question. Who makes the final decision here? I think this is the most contentious decision I can remember in nova, so I'm not sure how we want to go about actually locking a decision in. Is that something the PTL does after listening to everyone's thoughts? Or is there some sort of vote? I don't have a strong opinion here, but I think we do need some way to decide when this debate is over. Michael -- Rackspace Australia ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On 03/03/2014 04:25 PM, Vishvananda Ishaya wrote: > This seems like a reasonable and well thought out approach but It > feels like we are removing our ability to innovate. I think part of this exercise is trying to think through what freedom we *would* have to innovate in v2 with a set of changes to code an policy. That's what most of the proposal is about. If we still don't have the freedom we need, that's a problem and is what we need to figure out before this s final. Are there particular cases that you feel are missing in a v2 world described by the proposal? > I’m worried that this is just delaying solving the inconsistency > issues to some future date. It depends which consistency issues you're talking about. Many of them I think we can fix today. Others the argument being made is that they just aren't nearly important enough to fix yet unless they're coming with a completely new API. Which inconsistencies are you concerned about in particular? What did we miss in the proposal or which part specifically do you disagree with? -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On Mon, 2014-03-03 at 15:24 -0500, Russell Bryant wrote: > On 03/03/2014 02:59 PM, Jay Pipes wrote: > > -1 from me. Sounds like a way to avoid badly needed change and > > innovation in the API. When, for example, would we be able to propose a > > patch that removed API extensions entirely? > > v3 didn't address this at all, anyway. I'm not even sure there's > consensus on it. In any case, I think it's tangential and only relevant > if we're starting with a new API. I guess I am saying that if the ostensibly "limited" changes and standardization of Chris and others' V3 API work has now been decided is too risky to use or of not enough value to users, then the chance that a brand new API version seeing the light of day is pretty low. And that is disappointing to me. Feel free to tell me I'm nuts, though. I'd happily exchange my skepticism for some healthy optimism. > > The inconsistent naming, capitalization, numerous worthless or pointless > > API extensions, ability to do similar or identical things in different > > ways, and the incorrect result/response codes makes the Nova API look > > immature and clownish. I'd love to see a competing proposal to this that > > looks towards the future and a day when we can be proud of the Compute > > API as a real differentiator vs. the EC2 API. As it stands, the v2 REST > > API just makes OpenStack Compute look subpar at best, IMO. > > Much of what you discuss is addressed in the document. Well, it's discussed in the document... in as much to say "we won't really change any of these things..." > I think the differentiation that you want comes from a new ground-up > API, and not what's being discussed here (v3, or further evolution of v2). Yes, but my concern about this proposal is that it reinforces the status quo and in so doing, slows down the pace of innovation at the API level. And, a slow pace of API innovation in turn inhibits innovation at layers of the code beneath the API. > > Feel free to accuse me of wanting my cake and eating it, too. I guess > > I'm just both hungry and impatient. > > Not that, exactly ... just ignoring some of the practicalities at hand, > I think. Fair enough. :) Best, -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
I think its also pretty unfair on the people who put a lot of work into the v3 API. We're seriously going to delete their code after they put a year into it? To me OpenStack isn't just the users, its also the development community. I think we do measurable harm to that development community by doing this. We're teaching people that having a blessed plan that was discussed at a summit is not enough to reassure their management chain that they're not wasting time developing something. That worries me a lot. I'm yet to see a third party library (fog, jclouds, etc) express concern about a move to v3 if the deprecation cycle for v2 is nice and long. So why are we so worried? Michael On Tue, Mar 4, 2014 at 8:25 AM, Vishvananda Ishaya wrote: > This seems like a reasonable and well thought out approach but It feels > like we are removing our ability to innovate. I know we are worried about > maintaining multiple APIs, but I'm still leaning towards putting the v3 > API out and just keeping v2 around for a long time. Yes, its a maintenance > burden, but if we aren't adding a lot of features to v2, I don't know if > it is really THAT bad. > > I'm worried that this is just delaying solving the inconsistency issues to > some future date. > > Vish > > On Mar 3, 2014, at 9:32 AM, Russell Bryant wrote: > >> There has been quite a bit of discussion about the future of the v3 API >> recently. There has been growing support for the idea that we should >> change course and focus on evolving the existing v2 API instead of >> putting out a new major revision. This message is a more complete >> presentation of that proposal that concludes that we can do what we >> really need to do with only the v2 API. >> >> Keeping only the v2 API requires some confidence that we can stick with >> it for years to come. We don't want to be revisiting this any time >> soon. This message addresses a bunch of different questions about how >> things would work if we only had v2. >> >> 1) What about tasks? >> >> In some cases, the proposed integration of tasks is backwards >> compatible. A task ID will be added to a header. The biggest point of >> debate was if and how we would change the response for creating a >> server. For tasks in v2, we would not change the response by default. >> The task ID would just be in a header. However, if and when the client >> starts exposing version support information, we can provide an >> alternative/preferred response based on tasks. >> >> For example: >> >> Accept: application/json;type=task >> >> 2) Versioning extensions >> >> One of the points being addressed in the v3 API was the ability to >> version extensions. In v2, we have historically required new API >> extensions, even for changes that are backwards compatible. We propose >> the following: >> >> - Add a version number to v2 API extensions >> - Allow backwards compatible changes to these API extensions, >> accompanied by a version number increase >> - Add the option to advertise an extension as deprecated, which can be >> used for all those extensions created only to advertise the availability >> of new input parameters >> >> 3) Core versioning >> >> Another pain point in API maintenance has been having to create API >> extensions for every small addition to the core API. We propose that a >> version number be exposed for the core API that exposes the revision of >> the core API in use. With that in place, backwards compatible changes >> such as adding a new property to a resource would be allowed when >> accompanied by a version number increase. >> >> With versioning of the core and API extensions, we will be able to cut >> down significantly on the number of changes that require an API >> extension without sacrificing the ability of a client to discover >> whether the addition is present or not. >> >> 4) API Proxying >> >> We don't see proxying APIs as a problem. It is the cost we pay for >> choosing to split apart projects after they are released. We don't >> think it's fair to break users just because we have chosen to split >> apart the backend implementation. >> >> Further, the APIs that are proxied are frozen while those in the other >> projects are evolving. We believe that as more features are available >> only via the native APIs in Cinder, Glance, and Neutron, users will >> naturally migrate over to the native APIs. >> >> Over time, we can ensure clients are able to query the API without the >> need to proxy by adding new formats or extensions that don't return data >> that needed to be proxied. >> >> 5) Capitalization and Naming Consistency >> >> Some of the changes in the v3 API included changes to capitalization and >> naming for improved consistency. If we stick with v2 only, we will not >> be able to make any of these changes. However, we believe that not >> breaking any existing clients and not having to maintain a second API is >> worth not making these changes, or supporting some indirection to >> achieve thi
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On Mon, 03 Mar 2014 12:32:04 -0500 Russell Bryant wrote: > > The v3 API effort has produced a lot of excellent work. However, the > majority opinion seems to be that we should avoid the cost of > maintaining two APIs if at all possible. We should apply what has > been learned to the existing API where we can and focus on making v2 > something that we can continue to maintain for years to come. I'll respond to the technical points raised later, but I do want to point out now that I think this is a false dichotomy. That it is incorrect to say that the only way to avoid the cost of maintaining two APIs is to keep only the V2 API. Especially when the discussion then proceeds around being able to make backwards incompatible changes (deprecation) - which is a doing a major version bump whether we call it that or not and is keeping multiple versions around. > We recognize and accept that it is a failure of Nova project > leadership that we did not come to this conclusion much sooner. We > hope to have learned from the experience to help avoiding a situation > like this happening again in the future. The V3 API development work was encouraged and endorsed at both the Havana and Icehouse (and at Icehouse most of the V3 API patches that form the core of the API had already merged) and everyone had ample opportunity to comment or object then. But what I think disappoints me most is the way that this discussion has been approached. That the people who have done the most work on both the V3 and V2 API code in the last couple of cycles were not consulted first to see what could be done to address the concerns around dual maintenance. And instead an assertion was made that all of the work done for V3 would have to be dropped because of dual maintenance concerns without even trying to measure or explain what that dual maintenance overhead is. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
This seems like a reasonable and well thought out approach but It feels like we are removing our ability to innovate. I know we are worried about maintaining multiple APIs, but I’m still leaning towards putting the v3 API out and just keeping v2 around for a long time. Yes, its a maintenance burden, but if we aren’t adding a lot of features to v2, I don’t know if it is really THAT bad. I’m worried that this is just delaying solving the inconsistency issues to some future date. Vish On Mar 3, 2014, at 9:32 AM, Russell Bryant wrote: > There has been quite a bit of discussion about the future of the v3 API > recently. There has been growing support for the idea that we should > change course and focus on evolving the existing v2 API instead of > putting out a new major revision. This message is a more complete > presentation of that proposal that concludes that we can do what we > really need to do with only the v2 API. > > Keeping only the v2 API requires some confidence that we can stick with > it for years to come. We don't want to be revisiting this any time > soon. This message addresses a bunch of different questions about how > things would work if we only had v2. > > 1) What about tasks? > > In some cases, the proposed integration of tasks is backwards > compatible. A task ID will be added to a header. The biggest point of > debate was if and how we would change the response for creating a > server. For tasks in v2, we would not change the response by default. > The task ID would just be in a header. However, if and when the client > starts exposing version support information, we can provide an > alternative/preferred response based on tasks. > > For example: > > Accept: application/json;type=task > > 2) Versioning extensions > > One of the points being addressed in the v3 API was the ability to > version extensions. In v2, we have historically required new API > extensions, even for changes that are backwards compatible. We propose > the following: > > - Add a version number to v2 API extensions > - Allow backwards compatible changes to these API extensions, > accompanied by a version number increase > - Add the option to advertise an extension as deprecated, which can be > used for all those extensions created only to advertise the availability > of new input parameters > > 3) Core versioning > > Another pain point in API maintenance has been having to create API > extensions for every small addition to the core API. We propose that a > version number be exposed for the core API that exposes the revision of > the core API in use. With that in place, backwards compatible changes > such as adding a new property to a resource would be allowed when > accompanied by a version number increase. > > With versioning of the core and API extensions, we will be able to cut > down significantly on the number of changes that require an API > extension without sacrificing the ability of a client to discover > whether the addition is present or not. > > 4) API Proxying > > We don't see proxying APIs as a problem. It is the cost we pay for > choosing to split apart projects after they are released. We don't > think it's fair to break users just because we have chosen to split > apart the backend implementation. > > Further, the APIs that are proxied are frozen while those in the other > projects are evolving. We believe that as more features are available > only via the native APIs in Cinder, Glance, and Neutron, users will > naturally migrate over to the native APIs. > > Over time, we can ensure clients are able to query the API without the > need to proxy by adding new formats or extensions that don't return data > that needed to be proxied. > > 5) Capitalization and Naming Consistency > > Some of the changes in the v3 API included changes to capitalization and > naming for improved consistency. If we stick with v2 only, we will not > be able to make any of these changes. However, we believe that not > breaking any existing clients and not having to maintain a second API is > worth not making these changes, or supporting some indirection to > achieve this for newer clients if we decide it is important. > > 6) Response Code Consistency and Correctness > > The v2 API has many places where the response code returned for a given > operation is not strictly correct. For example a 200 is returned when a > 202 would be more appropriate. Correcting these issues should be > considered for improving the future use of the API, however there does > not seem to be any support for considering this a critical problem right > now. There are two approaches that can be taken to improve this in v2: > > Just fix them. Right now, we return some codes that imply we have dealt > with a request, when all we have done is queue it for processing (and > vice versa). In the future, we may change the backend in such a way that > a return code needs to change to continue to be accurate anyway. If w
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On 03/03/2014 02:59 PM, Jay Pipes wrote: > -1 from me. Sounds like a way to avoid badly needed change and > innovation in the API. When, for example, would we be able to propose a > patch that removed API extensions entirely? v3 didn't address this at all, anyway. I'm not even sure there's consensus on it. In any case, I think it's tangential and only relevant if we're starting with a new API. > The inconsistent naming, capitalization, numerous worthless or pointless > API extensions, ability to do similar or identical things in different > ways, and the incorrect result/response codes makes the Nova API look > immature and clownish. I'd love to see a competing proposal to this that > looks towards the future and a day when we can be proud of the Compute > API as a real differentiator vs. the EC2 API. As it stands, the v2 REST > API just makes OpenStack Compute look subpar at best, IMO. Much of what you discuss is addressed in the document. I think the differentiation that you want comes from a new ground-up API, and not what's being discussed here (v3, or further evolution of v2). > Feel free to accuse me of wanting my cake and eating it, too. I guess > I'm just both hungry and impatient. Not that, exactly ... just ignoring some of the practicalities at hand, I think. -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
-1 from me. Sounds like a way to avoid badly needed change and innovation in the API. When, for example, would we be able to propose a patch that removed API extensions entirely? The inconsistent naming, capitalization, numerous worthless or pointless API extensions, ability to do similar or identical things in different ways, and the incorrect result/response codes makes the Nova API look immature and clownish. I'd love to see a competing proposal to this that looks towards the future and a day when we can be proud of the Compute API as a real differentiator vs. the EC2 API. As it stands, the v2 REST API just makes OpenStack Compute look subpar at best, IMO. Feel free to accuse me of wanting my cake and eating it, too. I guess I'm just both hungry and impatient. Best, -jay On Mon, 2014-03-03 at 12:32 -0500, Russell Bryant wrote: > There has been quite a bit of discussion about the future of the v3 API > recently. There has been growing support for the idea that we should > change course and focus on evolving the existing v2 API instead of > putting out a new major revision. This message is a more complete > presentation of that proposal that concludes that we can do what we > really need to do with only the v2 API. > > Keeping only the v2 API requires some confidence that we can stick with > it for years to come. We don't want to be revisiting this any time > soon. This message addresses a bunch of different questions about how > things would work if we only had v2. > > 1) What about tasks? > > In some cases, the proposed integration of tasks is backwards > compatible. A task ID will be added to a header. The biggest point of > debate was if and how we would change the response for creating a > server. For tasks in v2, we would not change the response by default. > The task ID would just be in a header. However, if and when the client > starts exposing version support information, we can provide an > alternative/preferred response based on tasks. > > For example: > >Accept: application/json;type=task > > 2) Versioning extensions > > One of the points being addressed in the v3 API was the ability to > version extensions. In v2, we have historically required new API > extensions, even for changes that are backwards compatible. We propose > the following: > > - Add a version number to v2 API extensions > - Allow backwards compatible changes to these API extensions, > accompanied by a version number increase > - Add the option to advertise an extension as deprecated, which can be > used for all those extensions created only to advertise the availability > of new input parameters > > 3) Core versioning > > Another pain point in API maintenance has been having to create API > extensions for every small addition to the core API. We propose that a > version number be exposed for the core API that exposes the revision of > the core API in use. With that in place, backwards compatible changes > such as adding a new property to a resource would be allowed when > accompanied by a version number increase. > > With versioning of the core and API extensions, we will be able to cut > down significantly on the number of changes that require an API > extension without sacrificing the ability of a client to discover > whether the addition is present or not. > > 4) API Proxying > > We don't see proxying APIs as a problem. It is the cost we pay for > choosing to split apart projects after they are released. We don't > think it's fair to break users just because we have chosen to split > apart the backend implementation. > > Further, the APIs that are proxied are frozen while those in the other > projects are evolving. We believe that as more features are available > only via the native APIs in Cinder, Glance, and Neutron, users will > naturally migrate over to the native APIs. > > Over time, we can ensure clients are able to query the API without the > need to proxy by adding new formats or extensions that don't return data > that needed to be proxied. > > 5) Capitalization and Naming Consistency > > Some of the changes in the v3 API included changes to capitalization and > naming for improved consistency. If we stick with v2 only, we will not > be able to make any of these changes. However, we believe that not > breaking any existing clients and not having to maintain a second API is > worth not making these changes, or supporting some indirection to > achieve this for newer clients if we decide it is important. > > 6) Response Code Consistency and Correctness > > The v2 API has many places where the response code returned for a given > operation is not strictly correct. For example a 200 is returned when a > 202 would be more appropriate. Correcting these issues should be > considered for improving the future use of the API, however there does > not seem to be any support for considering this a critical problem right > now. There are two approaches that can be taken to improve
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On Mon, Mar 3, 2014 at 10:49 AM, Russell Bryant wrote: > On 03/03/2014 01:27 PM, Joe Gordon wrote: >> On Mon, Mar 3, 2014 at 9:32 AM, Russell Bryant wrote: >>> 1) What about tasks? >>> >>> In some cases, the proposed integration of tasks is backwards >>> compatible. A task ID will be added to a header. The biggest point of >>> debate was if and how we would change the response for creating a >>> server. For tasks in v2, we would not change the response by default. >>> The task ID would just be in a header. However, if and when the client >>> starts exposing version support information, we can provide an >>> alternative/preferred response based on tasks. >>> >>> For example: >>> >>>Accept: application/json;type=task >> >> The current https://wiki.openstack.org/wiki/APIChangeGuidelines says >> its OK add a new response header, so do we even need this? > > This is for the case that we want to actually change the response body. > >>> - Add the option to advertise an extension as deprecated, which can be >>> used for all those extensions created only to advertise the availability >>> of new input parameters >> >> Can you elaborate on this last point. > > We have previously required API extensions for adding some things like > input parameters or attributes on a resource. The addition of > versioning for extensions allow us to do this without adding extensions. > The point is just that it would be nice if we can mark extensions in > this category as deprecated and possibly removed since we can express > these things in terms of versions instead. That's what I thought just making sure. > >>> >>> 3) Core versioning >>> >>> Another pain point in API maintenance has been having to create API >>> extensions for every small addition to the core API. We propose that a >>> version number be exposed for the core API that exposes the revision of >>> the core API in use. With that in place, backwards compatible changes >>> such as adding a new property to a resource would be allowed when >>> accompanied by a version number increase. >>> >>> With versioning of the core and API extensions, we will be able to cut >>> down significantly on the number of changes that require an API >>> extension without sacrificing the ability of a client to discover >>> whether the addition is present or not. >> >> ++, looks like we may need to update >> https://wiki.openstack.org/wiki/APIChangeGuidelines and make this >> clear to downstream users. > > Right, just shooting for some agreement first. > >>> >>> 4) API Proxying >>> >>> We don't see proxying APIs as a problem. It is the cost we pay for >>> choosing to split apart projects after they are released. We don't >>> think it's fair to break users just because we have chosen to split >>> apart the backend implementation. >>> >>> Further, the APIs that are proxied are frozen while those in the other >>> projects are evolving. We believe that as more features are available >>> only via the native APIs in Cinder, Glance, and Neutron, users will >>> naturally migrate over to the native APIs. >>> >>> Over time, we can ensure clients are able to query the API without the >>> need to proxy by adding new formats or extensions that don't return data >>> that needed to be proxied. >> >> Can you expand on what this last paragraph means? >> >> While I agree in not breaking users. I assume this means we won't >> accept any new proxy APIs? > > If proxying is required to make an existing API continue to work, we > should accept it. Agreed, but we should accept a new API call that is just a proxy (unless its EC2 ...) > >>> 6) Response Code Consistency and Correctness >>> >>> The v2 API has many places where the response code returned for a given >>> operation is not strictly correct. For example a 200 is returned when a >>> 202 would be more appropriate. Correcting these issues should be >>> considered for improving the future use of the API, however there does >>> not seem to be any support for considering this a critical problem right >>> now. There are two approaches that can be taken to improve this in v2: >>> >>> Just fix them. Right now, we return some codes that imply we have dealt >>> with a request, when all we have done is queue it for processing (and >>> vice versa). In the future, we may change the backend in such a way that >>> a return code needs to change to continue to be accurate anyway. If we >>> just assume that return codes may change to properly reflect the action >>> that was taken, then we can correct existing errors and move on. >> >> Changing return codes always scares me, we risk breaking code that >> says if '==200.' Although having versioned backwards compatable APIs >> makes this a little better. > > See below ... > >>> Accept them as wrong but not critically so. With this approach, we can >>> strive for correctness in the future without changing behavior of our >>> existing APIs. Nobody seems to complain about them right now, so >>> changing them seems
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On 03/03/14 at 10:27am, Joe Gordon wrote: On Mon, Mar 3, 2014 at 9:32 AM, Russell Bryant wrote: There has been quite a bit of discussion about the future of the v3 API recently. There has been growing support for the idea that we should change course and focus on evolving the existing v2 API instead of putting out a new major revision. This message is a more complete presentation of that proposal that concludes that we can do what we really need to do with only the v2 API. Keeping only the v2 API requires some confidence that we can stick with it for years to come. We don't want to be revisiting this any time soon. This message addresses a bunch of different questions about how things would work if we only had v2. 1) What about tasks? In some cases, the proposed integration of tasks is backwards compatible. A task ID will be added to a header. The biggest point of debate was if and how we would change the response for creating a server. For tasks in v2, we would not change the response by default. The task ID would just be in a header. However, if and when the client starts exposing version support information, we can provide an alternative/preferred response based on tasks. For example: Accept: application/json;type=task The current https://wiki.openstack.org/wiki/APIChangeGuidelines says its OK add a new response header, so do we even need this? This would be needed in order to add more than the response header. Ideally the task(s) would be returned in the response body as well. But we can get started by using the header until we have a way for clients to indicate a request version. 2) Versioning extensions One of the points being addressed in the v3 API was the ability to version extensions. In v2, we have historically required new API extensions, even for changes that are backwards compatible. We propose the following: - Add a version number to v2 API extensions - Allow backwards compatible changes to these API extensions, accompanied by a version number increase - Add the option to advertise an extension as deprecated, which can be used for all those extensions created only to advertise the availability of new input parameters Can you elaborate on this last point. There are some extensions whose sole purpose is to enable accepting a new request parameter or adding a field to a response. Some of them are just feature flags for other extensions (user_quotas), and some modify the response themselves (flavor_swap). In both cases the functionality could be merged into another extension with a version bump and then they could be marked as deprecated. 3) Core versioning Another pain point in API maintenance has been having to create API extensions for every small addition to the core API. We propose that a version number be exposed for the core API that exposes the revision of the core API in use. With that in place, backwards compatible changes such as adding a new property to a resource would be allowed when accompanied by a version number increase. With versioning of the core and API extensions, we will be able to cut down significantly on the number of changes that require an API extension without sacrificing the ability of a client to discover whether the addition is present or not. ++, looks like we may need to update https://wiki.openstack.org/wiki/APIChangeGuidelines and make this clear to downstream users. 4) API Proxying We don't see proxying APIs as a problem. It is the cost we pay for choosing to split apart projects after they are released. We don't think it's fair to break users just because we have chosen to split apart the backend implementation. Further, the APIs that are proxied are frozen while those in the other projects are evolving. We believe that as more features are available only via the native APIs in Cinder, Glance, and Neutron, users will naturally migrate over to the native APIs. Over time, we can ensure clients are able to query the API without the need to proxy by adding new formats or extensions that don't return data that needed to be proxied. Can you expand on what this last paragraph means? While I agree in not breaking users. I assume this means we won't accept any new proxy APIs? 5) Capitalization and Naming Consistency Some of the changes in the v3 API included changes to capitalization and naming for improved consistency. If we stick with v2 only, we will not be able to make any of these changes. However, we believe that not breaking any existing clients and not having to maintain a second API is worth not making these changes, or supporting some indirection to achieve this for newer clients if we decide it is important. ++ 6) Response Code Consistency and Correctness The v2 API has many places where the response code returned for a given operation is not strictly correct. For example a 200 is returned when a 202 would be more appropriate. Correcting these issues should be consid
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On 03/03/2014 01:27 PM, Joe Gordon wrote: > On Mon, Mar 3, 2014 at 9:32 AM, Russell Bryant wrote: >> 1) What about tasks? >> >> In some cases, the proposed integration of tasks is backwards >> compatible. A task ID will be added to a header. The biggest point of >> debate was if and how we would change the response for creating a >> server. For tasks in v2, we would not change the response by default. >> The task ID would just be in a header. However, if and when the client >> starts exposing version support information, we can provide an >> alternative/preferred response based on tasks. >> >> For example: >> >>Accept: application/json;type=task > > The current https://wiki.openstack.org/wiki/APIChangeGuidelines says > its OK add a new response header, so do we even need this? This is for the case that we want to actually change the response body. >> - Add the option to advertise an extension as deprecated, which can be >> used for all those extensions created only to advertise the availability >> of new input parameters > > Can you elaborate on this last point. We have previously required API extensions for adding some things like input parameters or attributes on a resource. The addition of versioning for extensions allow us to do this without adding extensions. The point is just that it would be nice if we can mark extensions in this category as deprecated and possibly removed since we can express these things in terms of versions instead. >> >> 3) Core versioning >> >> Another pain point in API maintenance has been having to create API >> extensions for every small addition to the core API. We propose that a >> version number be exposed for the core API that exposes the revision of >> the core API in use. With that in place, backwards compatible changes >> such as adding a new property to a resource would be allowed when >> accompanied by a version number increase. >> >> With versioning of the core and API extensions, we will be able to cut >> down significantly on the number of changes that require an API >> extension without sacrificing the ability of a client to discover >> whether the addition is present or not. > > ++, looks like we may need to update > https://wiki.openstack.org/wiki/APIChangeGuidelines and make this > clear to downstream users. Right, just shooting for some agreement first. >> >> 4) API Proxying >> >> We don't see proxying APIs as a problem. It is the cost we pay for >> choosing to split apart projects after they are released. We don't >> think it's fair to break users just because we have chosen to split >> apart the backend implementation. >> >> Further, the APIs that are proxied are frozen while those in the other >> projects are evolving. We believe that as more features are available >> only via the native APIs in Cinder, Glance, and Neutron, users will >> naturally migrate over to the native APIs. >> >> Over time, we can ensure clients are able to query the API without the >> need to proxy by adding new formats or extensions that don't return data >> that needed to be proxied. > > Can you expand on what this last paragraph means? > > While I agree in not breaking users. I assume this means we won't > accept any new proxy APIs? If proxying is required to make an existing API continue to work, we should accept it. >> 6) Response Code Consistency and Correctness >> >> The v2 API has many places where the response code returned for a given >> operation is not strictly correct. For example a 200 is returned when a >> 202 would be more appropriate. Correcting these issues should be >> considered for improving the future use of the API, however there does >> not seem to be any support for considering this a critical problem right >> now. There are two approaches that can be taken to improve this in v2: >> >> Just fix them. Right now, we return some codes that imply we have dealt >> with a request, when all we have done is queue it for processing (and >> vice versa). In the future, we may change the backend in such a way that >> a return code needs to change to continue to be accurate anyway. If we >> just assume that return codes may change to properly reflect the action >> that was taken, then we can correct existing errors and move on. > > Changing return codes always scares me, we risk breaking code that > says if '==200.' Although having versioned backwards compatable APIs > makes this a little better. See below ... >> Accept them as wrong but not critically so. With this approach, we can >> strive for correctness in the future without changing behavior of our >> existing APIs. Nobody seems to complain about them right now, so >> changing them seems to be little gain. If the client begins exposing a >> version header (which we need for other things) then we could >> alternately start returning accurate codes for those clients. > > Wait what? client needs version headers? Can you expand Again see below ... > ++ to accepting them as wrong and
Re: [openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
On Mon, Mar 3, 2014 at 9:32 AM, Russell Bryant wrote: > There has been quite a bit of discussion about the future of the v3 API > recently. There has been growing support for the idea that we should > change course and focus on evolving the existing v2 API instead of > putting out a new major revision. This message is a more complete > presentation of that proposal that concludes that we can do what we > really need to do with only the v2 API. > > Keeping only the v2 API requires some confidence that we can stick with > it for years to come. We don't want to be revisiting this any time > soon. This message addresses a bunch of different questions about how > things would work if we only had v2. > > 1) What about tasks? > > In some cases, the proposed integration of tasks is backwards > compatible. A task ID will be added to a header. The biggest point of > debate was if and how we would change the response for creating a > server. For tasks in v2, we would not change the response by default. > The task ID would just be in a header. However, if and when the client > starts exposing version support information, we can provide an > alternative/preferred response based on tasks. > > For example: > >Accept: application/json;type=task The current https://wiki.openstack.org/wiki/APIChangeGuidelines says its OK add a new response header, so do we even need this? > > 2) Versioning extensions > > One of the points being addressed in the v3 API was the ability to > version extensions. In v2, we have historically required new API > extensions, even for changes that are backwards compatible. We propose > the following: > > - Add a version number to v2 API extensions > - Allow backwards compatible changes to these API extensions, > accompanied by a version number increase > - Add the option to advertise an extension as deprecated, which can be > used for all those extensions created only to advertise the availability > of new input parameters Can you elaborate on this last point. > > 3) Core versioning > > Another pain point in API maintenance has been having to create API > extensions for every small addition to the core API. We propose that a > version number be exposed for the core API that exposes the revision of > the core API in use. With that in place, backwards compatible changes > such as adding a new property to a resource would be allowed when > accompanied by a version number increase. > > With versioning of the core and API extensions, we will be able to cut > down significantly on the number of changes that require an API > extension without sacrificing the ability of a client to discover > whether the addition is present or not. ++, looks like we may need to update https://wiki.openstack.org/wiki/APIChangeGuidelines and make this clear to downstream users. > > 4) API Proxying > > We don't see proxying APIs as a problem. It is the cost we pay for > choosing to split apart projects after they are released. We don't > think it's fair to break users just because we have chosen to split > apart the backend implementation. > > Further, the APIs that are proxied are frozen while those in the other > projects are evolving. We believe that as more features are available > only via the native APIs in Cinder, Glance, and Neutron, users will > naturally migrate over to the native APIs. > > Over time, we can ensure clients are able to query the API without the > need to proxy by adding new formats or extensions that don't return data > that needed to be proxied. Can you expand on what this last paragraph means? While I agree in not breaking users. I assume this means we won't accept any new proxy APIs? > > 5) Capitalization and Naming Consistency > > Some of the changes in the v3 API included changes to capitalization and > naming for improved consistency. If we stick with v2 only, we will not > be able to make any of these changes. However, we believe that not > breaking any existing clients and not having to maintain a second API is > worth not making these changes, or supporting some indirection to > achieve this for newer clients if we decide it is important. > ++ > 6) Response Code Consistency and Correctness > > The v2 API has many places where the response code returned for a given > operation is not strictly correct. For example a 200 is returned when a > 202 would be more appropriate. Correcting these issues should be > considered for improving the future use of the API, however there does > not seem to be any support for considering this a critical problem right > now. There are two approaches that can be taken to improve this in v2: > > Just fix them. Right now, we return some codes that imply we have dealt > with a request, when all we have done is queue it for processing (and > vice versa). In the future, we may change the backend in such a way that > a return code needs to change to continue to be accurate anyway. If we > just assume that return codes may change to p
[openstack-dev] [Nova] Concrete Proposal for Keeping V2 API
There has been quite a bit of discussion about the future of the v3 API recently. There has been growing support for the idea that we should change course and focus on evolving the existing v2 API instead of putting out a new major revision. This message is a more complete presentation of that proposal that concludes that we can do what we really need to do with only the v2 API. Keeping only the v2 API requires some confidence that we can stick with it for years to come. We don't want to be revisiting this any time soon. This message addresses a bunch of different questions about how things would work if we only had v2. 1) What about tasks? In some cases, the proposed integration of tasks is backwards compatible. A task ID will be added to a header. The biggest point of debate was if and how we would change the response for creating a server. For tasks in v2, we would not change the response by default. The task ID would just be in a header. However, if and when the client starts exposing version support information, we can provide an alternative/preferred response based on tasks. For example: Accept: application/json;type=task 2) Versioning extensions One of the points being addressed in the v3 API was the ability to version extensions. In v2, we have historically required new API extensions, even for changes that are backwards compatible. We propose the following: - Add a version number to v2 API extensions - Allow backwards compatible changes to these API extensions, accompanied by a version number increase - Add the option to advertise an extension as deprecated, which can be used for all those extensions created only to advertise the availability of new input parameters 3) Core versioning Another pain point in API maintenance has been having to create API extensions for every small addition to the core API. We propose that a version number be exposed for the core API that exposes the revision of the core API in use. With that in place, backwards compatible changes such as adding a new property to a resource would be allowed when accompanied by a version number increase. With versioning of the core and API extensions, we will be able to cut down significantly on the number of changes that require an API extension without sacrificing the ability of a client to discover whether the addition is present or not. 4) API Proxying We don't see proxying APIs as a problem. It is the cost we pay for choosing to split apart projects after they are released. We don't think it's fair to break users just because we have chosen to split apart the backend implementation. Further, the APIs that are proxied are frozen while those in the other projects are evolving. We believe that as more features are available only via the native APIs in Cinder, Glance, and Neutron, users will naturally migrate over to the native APIs. Over time, we can ensure clients are able to query the API without the need to proxy by adding new formats or extensions that don't return data that needed to be proxied. 5) Capitalization and Naming Consistency Some of the changes in the v3 API included changes to capitalization and naming for improved consistency. If we stick with v2 only, we will not be able to make any of these changes. However, we believe that not breaking any existing clients and not having to maintain a second API is worth not making these changes, or supporting some indirection to achieve this for newer clients if we decide it is important. 6) Response Code Consistency and Correctness The v2 API has many places where the response code returned for a given operation is not strictly correct. For example a 200 is returned when a 202 would be more appropriate. Correcting these issues should be considered for improving the future use of the API, however there does not seem to be any support for considering this a critical problem right now. There are two approaches that can be taken to improve this in v2: Just fix them. Right now, we return some codes that imply we have dealt with a request, when all we have done is queue it for processing (and vice versa). In the future, we may change the backend in such a way that a return code needs to change to continue to be accurate anyway. If we just assume that return codes may change to properly reflect the action that was taken, then we can correct existing errors and move on. Accept them as wrong but not critically so. With this approach, we can strive for correctness in the future without changing behavior of our existing APIs. Nobody seems to complain about them right now, so changing them seems to be little gain. If the client begins exposing a version header (which we need for other things) then we could alternately start returning accurate codes for those clients. The key point here is that we see a way forward with this in the v2 API regardless of which path we choose. 7) Entrypoint based extensions The v3 effort included improvements to th
