Re: [openstack-dev] The Nova API in Kilo and Beyond
On 08/06/15 18:22, Jay Pipes wrote: On 06/05/2015 10:56 AM, Neil Jerram wrote: I guess that's why the GNU autoconf/configure system has always advised testing for particular wanted features, instead of looking at versions and then relying on carnal knowledge to know what those versions imply. I'm pretty sure you meant "tribal knowledge", not "carnal knowledge" :) -jay http://en.wikipedia.org/wiki/Carnal_knowledge Gosh, that article needs a restricted rating. :-) You're right that I didn't mean it in that sense! However, I _think_ the phrase is fairly commonly used, at least in the UK, also to mean something more like tribal or insider knowledge. I hope so, anyway. :-) Regards, Neil __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] The Nova API in Kilo and Beyond
On 06/08/2015 01:22 PM, Jay Pipes wrote: On 06/05/2015 10:56 AM, Neil Jerram wrote: I guess that's why the GNU autoconf/configure system has always advised testing for particular wanted features, instead of looking at versions and then relying on carnal knowledge to know what those versions imply. I'm pretty sure you meant "tribal knowledge", not "carnal knowledge" :) And I am just as sure he knew what he was saying and intentionally said "carnal" -jay http://en.wikipedia.org/wiki/Carnal_knowledge http://en.wikipedia.org/wiki/Tribal_knowledge __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] The Nova API in Kilo and Beyond
On 06/05/2015 10:56 AM, Neil Jerram wrote: I guess that's why the GNU autoconf/configure system has always advised testing for particular wanted features, instead of looking at versions and then relying on carnal knowledge to know what those versions imply. I'm pretty sure you meant "tribal knowledge", not "carnal knowledge" :) -jay http://en.wikipedia.org/wiki/Carnal_knowledge http://en.wikipedia.org/wiki/Tribal_knowledge __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] The Nova API in Kilo and Beyond
On 06/05/2015 11:03 AM, David Kranz wrote: > On 06/05/2015 07:32 AM, Sean Dague wrote: >> One of the things we realized at the summit was that we'd been working >> through a better future for the Nova API for the past 5 cycles, gotten >> somewhere quite useful, but had really done a poor job on communicating >> what was going on and why, and where things are headed next. >> >> I've written a bunch of English to explain it (which should be on the >> planet shortly as well) - >> https://dague.net/2015/06/05/the-nova-api-in-kilo-and-beyond-2/ (with >> lots of help from Ed Leaf, John Garbutt, and Matt Gillard on content and >> copy editing). >> >> Yes, this is one of those terrible mailing list posts that points people >> to read a thing not on the list (I appologize). But at 2700 words, I >> think you'll find it more comfortable to read not in email. >> >> Discussion is welcome here for any comments folks have. Some details >> were trimmed for the sake of it not being a 6000 word essay, and to make >> it accessible to people that don't have a ton of Nova internals >> knowledge. We'll do our best to field questions, all of which will be >> integrated into the eventual dev ref version of this. >> >> Thanks for your time, >> >> -Sean >> > Thanks, Sean. Great writeup. There are two issues I think might need > more clarification/amplification: > > 1. Does the microversion methodology, and the motivation for true > interoperability, imply that there needs to be a new version for every > bug fix that could be detected by users of an api? There was back and > forth about that in the review about the ip6 server list filter bug you > referenced. If so, this is a pretty strong constraint that will need > more guidance for reviewers about which kinds of changes need new > versions and which don't. Correct, Alex is working on this recommendation for the API WG. But, yes, user visible changes should be reflected in a version bump, so they user knows that feature A is supported at version X.Y. > 2. What is the policy for making incompatible changes, now that > versioning "allows" such changes to be made? If some one doesn't like > the name of one of the keys in a returned dict, and submits a change > with new microversion, how should that be evaluated? IIRC, this was an > issue that inspired some dislike about the original v3 work. That's a per project call. And it's about the tradeoffs in cost vs. benefit. Key renames are probably not worth it unless they are really confusing. For instance, the Nova team is currently considering (though in no ways has decided) renaming the "evacuate" action to "resurrect" because *so* much confusion has been created by the name evacuate (and so many incorrectly designed applications assuming it would do things it doesn't) that the pain of making people change might be worth it. Might. We'll see how it plays out. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] The Nova API in Kilo and Beyond
On 06/05/2015 10:56 AM, Neil Jerram wrote: > On 05/06/15 12:32, Sean Dague wrote: >> https://dague.net/2015/06/05/the-nova-api-in-kilo-and-beyond-2/ > > This is really informative and useful, thanks. > > A few comments / questions, with bits of your text in quotes: > > "Even figuring out what a cloud could do was pretty terrible. You could > approximate it by listing the extensions of the API, then having a bunch > of logic in your code to realize which extensions turned on or off > certain features, or added new data to payloads." > > I guess that's why the GNU autoconf/configure system has always advised > testing for particular wanted features, instead of looking at versions > and then relying on carnal knowledge to know what those versions imply. > Is that feature-testing-based approach impractical for OpenStack? It shouldn't be carnal knowledge. If we are talking about building an ecosystem we need to be really explicit about "this version gives you this contract". We have protocol versioning all through our internal RPC mechanisms, because if we didn't you'd need to write an order of magnitude more code to have an application work. You can always give your user a terrible contract, and make them do a ton of extra work on their side to figure out what's available. See... present day. But the firm belief is we should do better than that if we want to encourage an application ecosystem. > "Then she runs her code at against another cloud, which runs a version > of Nova that predates this change. She's now effectively gone back in > time. Her code now returns thousands of records instead of 1, and she's > terribly confused why. She also has no way to figure out if random cloud > Z is going to support this feature or not. So the only safe thing to do > is implement the filtering client side instead, which means the server > side filtering actually gained her very little. It's not something she > can ever determine will work ahead of time. It's an API that is > untrustworthy, so it's something that's best avoided." > > Except that she still has to do all this anyway - i.e. write the > client-side filtering, and figure out when to use it instead of > server-side - even if there was an API version change accompanying the > filtering feature. Doesn't she? Not if she's comfortable with a minimum supported version in her code. People abandon old systems all the time. No one is still writing IE6 supporting javascript code. The important thing is that a non trust worthy API, i.e. one that could return differently seemingly at random, is terrible. It makes developers pull out their hair and curse your name, and figure out if they can get off your platform entirely. Really stable contracts are a key feature in building an ecosystem above you. It's why your house has a concrete slab under it, not just a pile of sand. > The difference is just between making the switch based on a version > number, and making it based on detected feature support. > > "If you want features in the 2.3 microversion, ..." > > I especially appreciate this part, as I've been seeing all the chat > about microversions go past, and not really understanding it. > > FWIW, though - and maybe this is just me - when I hear "microversion", > I'm thinking of the "Z" in an "X.Y.Z" version number. (With X = major > and Y = minor.) So it's counterintuitive for me that "2.3" is a > microversion; it just sounds like a perfectly normal major/minor version > number. Are 2.0 and 2.1 microversions too? So, we used the word "microversion" in contrast to previous versioning in OpenStack which required a new endpoint. The reality of the implementation is we're working with a monotonically increasing counter. Y is going to increase forever. This *is not* semver. There is no semantic meaning / value judgement on each change. It is better thought about as a sequence number. > But this is just bikeshedding really, so feel free to ignore... > > "without building a giant autoconf-like system" > > Aha, so you probably did consider that option, then. :-) Just provide the thought exercise on API autoconf like system. In order to fully explore an API you are looking at basically running something like tempest against it. So you needed to run about 20K API calls, build and destroy 100 guests, volumes, objects, etc. That kind of run probably would take a couple of hours. When you are building against a public cloud, every API has a cost. Also, it a service. It can be upgraded at any time. Without something clear like a microversion declaration, you'd never know they cloud updated, worked differently, and now your application fails to work. Also, find me more than a handful of application developers that like writing autoconf tests. :) -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ..
Re: [openstack-dev] The Nova API in Kilo and Beyond
On 06/05/2015 10:34 AM, Chris Dent wrote: > On Fri, 5 Jun 2015, Sean Dague wrote: > >> Thanks for your time, > > Thanks for writing that up. > > I recognize that microversions exist and are as they are so I don't > want to derail, but my curiosity was piqued: > > Riddle me this: If Microversions are kind of like content-negotiation > (and we love content-negotiation) for APIs, why not just use content- > negotiation instead of a header? Instead of: > >X-OpenStack-Nova-API-Version: 2.114 > > do (media type made up and not suggesting it as canonical): > >Accept: application/openstack-nova-api-2.114+json > > or even > >Accept: application/vnd.openstack-nova-api+json; version=2.114 > > (and similar on the content-type header). There is precedent for > this sort of thing in, for example, the github api. > > (I'll not[1] write about "srsly, can we please stop giving Jackson the > Absent so much freaking power".) > > [1] whoops Realistically, this was in one of the early proposals. But one of the challenges with content negotiations is that if we actually did this as content negotiation then you also need to support quality levels like Accept: application/vnd.openstack-nova-api+json; q=0.8; version=2.114, application/vnd.openstack-nova-api+json; q=0.5; version=2.110, application/vnd.openstack-nova-api+json; q=0.2; version=2.100 The complexities in thinking through the programming for that on the client side were a little odd. Enough so, that with other concepts we were trying to get out there, this seemed like it was just going to confuse everyone quite a bit. Better to err on the explicit side. It also meant that the fall back case semantics were less clear. Because existing clients would just be requesting application/json. Which isn't application/vnd.openstack-nova-api+json at all. The Jackson the absent case is kind of important, because we were told by public cloud providers that turning off v2.0 is not an option for them in any forseable future. Major upgrades are hard, especially when you have a lot of people using an interface. See "python 3" for how to do major version bumps in a way that no one moves onto your new stuff in a timely manner. We're trying to explicitly avoid that by allowing people to move forward on their own time table, and actually not have to move everything at once. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] The Nova API in Kilo and Beyond
On 06/05/2015 07:32 AM, Sean Dague wrote: One of the things we realized at the summit was that we'd been working through a better future for the Nova API for the past 5 cycles, gotten somewhere quite useful, but had really done a poor job on communicating what was going on and why, and where things are headed next. I've written a bunch of English to explain it (which should be on the planet shortly as well) - https://dague.net/2015/06/05/the-nova-api-in-kilo-and-beyond-2/ (with lots of help from Ed Leaf, John Garbutt, and Matt Gillard on content and copy editing). Yes, this is one of those terrible mailing list posts that points people to read a thing not on the list (I appologize). But at 2700 words, I think you'll find it more comfortable to read not in email. Discussion is welcome here for any comments folks have. Some details were trimmed for the sake of it not being a 6000 word essay, and to make it accessible to people that don't have a ton of Nova internals knowledge. We'll do our best to field questions, all of which will be integrated into the eventual dev ref version of this. Thanks for your time, -Sean Thanks, Sean. Great writeup. There are two issues I think might need more clarification/amplification: 1. Does the microversion methodology, and the motivation for true interoperability, imply that there needs to be a new version for every bug fix that could be detected by users of an api? There was back and forth about that in the review about the ip6 server list filter bug you referenced. If so, this is a pretty strong constraint that will need more guidance for reviewers about which kinds of changes need new versions and which don't. 2. What is the policy for making incompatible changes, now that versioning "allows" such changes to be made? If some one doesn't like the name of one of the keys in a returned dict, and submits a change with new microversion, how should that be evaluated? IIRC, this was an issue that inspired some dislike about the original v3 work. -David __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] The Nova API in Kilo and Beyond
On 05/06/15 12:32, Sean Dague wrote: https://dague.net/2015/06/05/the-nova-api-in-kilo-and-beyond-2/ This is really informative and useful, thanks. A few comments / questions, with bits of your text in quotes: "Even figuring out what a cloud could do was pretty terrible. You could approximate it by listing the extensions of the API, then having a bunch of logic in your code to realize which extensions turned on or off certain features, or added new data to payloads." I guess that's why the GNU autoconf/configure system has always advised testing for particular wanted features, instead of looking at versions and then relying on carnal knowledge to know what those versions imply. Is that feature-testing-based approach impractical for OpenStack? "Then she runs her code at against another cloud, which runs a version of Nova that predates this change. She's now effectively gone back in time. Her code now returns thousands of records instead of 1, and she's terribly confused why. She also has no way to figure out if random cloud Z is going to support this feature or not. So the only safe thing to do is implement the filtering client side instead, which means the server side filtering actually gained her very little. It's not something she can ever determine will work ahead of time. It's an API that is untrustworthy, so it's something that's best avoided." Except that she still has to do all this anyway - i.e. write the client-side filtering, and figure out when to use it instead of server-side - even if there was an API version change accompanying the filtering feature. Doesn't she? The difference is just between making the switch based on a version number, and making it based on detected feature support. "If you want features in the 2.3 microversion, ..." I especially appreciate this part, as I've been seeing all the chat about microversions go past, and not really understanding it. FWIW, though - and maybe this is just me - when I hear "microversion", I'm thinking of the "Z" in an "X.Y.Z" version number. (With X = major and Y = minor.) So it's counterintuitive for me that "2.3" is a microversion; it just sounds like a perfectly normal major/minor version number. Are 2.0 and 2.1 microversions too? But this is just bikeshedding really, so feel free to ignore... "without building a giant autoconf-like system" Aha, so you probably did consider that option, then. :-) Many thanks, Neil __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] The Nova API in Kilo and Beyond
On Fri, 5 Jun 2015, Sean Dague wrote: Thanks for your time, Thanks for writing that up. I recognize that microversions exist and are as they are so I don't want to derail, but my curiosity was piqued: Riddle me this: If Microversions are kind of like content-negotiation (and we love content-negotiation) for APIs, why not just use content- negotiation instead of a header? Instead of: X-OpenStack-Nova-API-Version: 2.114 do (media type made up and not suggesting it as canonical): Accept: application/openstack-nova-api-2.114+json or even Accept: application/vnd.openstack-nova-api+json; version=2.114 (and similar on the content-type header). There is precedent for this sort of thing in, for example, the github api. (I'll not[1] write about "srsly, can we please stop giving Jackson the Absent so much freaking power".) [1] whoops -- Chris Dent tw:@anticdent freenode:cdent https://tank.peermore.com/tanks/cdent __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] The Nova API in Kilo and Beyond
One of the things we realized at the summit was that we'd been working through a better future for the Nova API for the past 5 cycles, gotten somewhere quite useful, but had really done a poor job on communicating what was going on and why, and where things are headed next. I've written a bunch of English to explain it (which should be on the planet shortly as well) - https://dague.net/2015/06/05/the-nova-api-in-kilo-and-beyond-2/ (with lots of help from Ed Leaf, John Garbutt, and Matt Gillard on content and copy editing). Yes, this is one of those terrible mailing list posts that points people to read a thing not on the list (I appologize). But at 2700 words, I think you'll find it more comfortable to read not in email. Discussion is welcome here for any comments folks have. Some details were trimmed for the sake of it not being a 6000 word essay, and to make it accessible to people that don't have a ton of Nova internals knowledge. We'll do our best to field questions, all of which will be integrated into the eventual dev ref version of this. Thanks for your time, -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
