Re: [openstack-dev] [heat] Client checking of server version
On 01/06/2016 12:05 PM, Zane Bitter wrote: On 05/01/16 16:37, Steven Hardy wrote: On Mon, Jan 04, 2016 at 03:53:07PM -0500, Jay Dobies wrote: I ran into an issue in a review about moving environment resolution from client to server [1]. It revolves around clients being able to access older versions of servers (that's a pretty simplistic description; see [2] for the spec). Before the holiday, Steve Hardy and I were talking about the complications involved. In my case, there's no good way to differentiate an older server from a legitimate error. Since the API isn't versioned to the extent that we can leverage that value, I was looking into using the template versions call. Something along the lines of: supported_versions = hc.template_versions.list() version_nums = [i.to_dict()['version'].split('.')[1] for i in supported_versions] mitaka_or_newer = [i for i in version_nums if i >= '2016-04-08'] Yes, I'm planning on cleaning that up before submitting it :) What I'm wondering is if I should make this into some sort of generalized utility method in the client, under the assumption that we'll need this sort of check in the future for the same backward compatibility requirements. So a few questions: 1. Does anyone strongly disagree to checking supported template versions as a way of determining the specifics of the server API. Ok, so some valid concerns have been raised over deriving things using the HOT version (although I do still wonder if the environment itself should be versioned, just like the templates, then we could rev the environment verion and say it supports a list, vs changing anything in the API, but that's probably a separate discussion). Taking a step back for a moment, the original discussion was around providing transparent access to the new interface via heatclient, but that isn't actually a hard requirement - the old interface works fine for many users, so we could just introduce a new interface (which would eventually become the default, after all non-EOL heat versions released support the new API argument): Currently we do: heat stack-create foo -f foo.yaml -e a.yaml -e b.yaml And this implies some client-side resolution of the multiple -e arguments. -e is short for "--environment-file", but we could introduce a new format, e.g "-E", short for "--environment-files": I agree with Zane, this looks like a usability (and backwards compat) nightmare. Not only do you have to get over everyone's muscle memory of typing `-e` (I've got it bad) but also all the scripts folks have that use heatclient. Then there's the docs between "If ... blah blah ... then use -E, otherwise use -e" will be a pretty fat stumbling block for folks that use different deploys of OpenStack (say, a Juno prod cloud and a Kilo staging cloud) if they want to use heat templates on both. heat stack-create foo -f foo.yaml -E a.yaml -E b.yaml This option would work the same way as the current interface, but it would pass the files unmodified for resolution inside heat (by using the new API format), and as it's opt-in, it's leaving all the current heatclient interfaces alone without any internal fallback logic? That would certainly work, but it sounds like a usability/support nightmare :( Is there a reason we wouldn't consider bumping the API version to 1.1 for this? We'll have to figure out how to do it some time. cheers, Zane. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Ryan Brown / Senior Software Engineer, Openstack / Red Hat, Inc. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat] Client checking of server version
On 01/06/2016 12:05 PM, Zane Bitter wrote: On 05/01/16 16:37, Steven Hardy wrote: On Mon, Jan 04, 2016 at 03:53:07PM -0500, Jay Dobies wrote: I ran into an issue in a review about moving environment resolution from client to server [1]. It revolves around clients being able to access older versions of servers (that's a pretty simplistic description; see [2] for the spec). Before the holiday, Steve Hardy and I were talking about the complications involved. In my case, there's no good way to differentiate an older server from a legitimate error. Since the API isn't versioned to the extent that we can leverage that value, I was looking into using the template versions call. Something along the lines of: supported_versions = hc.template_versions.list() version_nums = [i.to_dict()['version'].split('.')[1] for i in supported_versions] mitaka_or_newer = [i for i in version_nums if i >= '2016-04-08'] Yes, I'm planning on cleaning that up before submitting it :) What I'm wondering is if I should make this into some sort of generalized utility method in the client, under the assumption that we'll need this sort of check in the future for the same backward compatibility requirements. So a few questions: 1. Does anyone strongly disagree to checking supported template versions as a way of determining the specifics of the server API. Ok, so some valid concerns have been raised over deriving things using the HOT version (although I do still wonder if the environment itself should be versioned, just like the templates, then we could rev the environment verion and say it supports a list, vs changing anything in the API, but that's probably a separate discussion). Taking a step back for a moment, the original discussion was around providing transparent access to the new interface via heatclient, but that isn't actually a hard requirement - the old interface works fine for many users, so we could just introduce a new interface (which would eventually become the default, after all non-EOL heat versions released support the new API argument): Currently we do: heat stack-create foo -f foo.yaml -e a.yaml -e b.yaml And this implies some client-side resolution of the multiple -e arguments. -e is short for "--environment-file", but we could introduce a new format, e.g "-E", short for "--environment-files": heat stack-create foo -f foo.yaml -E a.yaml -E b.yaml This option would work the same way as the current interface, but it would pass the files unmodified for resolution inside heat (by using the new API format), and as it's opt-in, it's leaving all the current heatclient interfaces alone without any internal fallback logic? That would certainly work, but it sounds like a usability/support nightmare :( Is there a reason we wouldn't consider bumping the API version to 1.1 for this? We'll have to figure out how to do it some time. I started to look into the Nova specs on how they handle micro versions. I have a few other things on my plate I want to finish up this week, but I should be able to take a stab at a POC for it. cheers, Zane. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat] Client checking of server version
On 05/01/16 16:37, Steven Hardy wrote: On Mon, Jan 04, 2016 at 03:53:07PM -0500, Jay Dobies wrote: I ran into an issue in a review about moving environment resolution from client to server [1]. It revolves around clients being able to access older versions of servers (that's a pretty simplistic description; see [2] for the spec). Before the holiday, Steve Hardy and I were talking about the complications involved. In my case, there's no good way to differentiate an older server from a legitimate error. Since the API isn't versioned to the extent that we can leverage that value, I was looking into using the template versions call. Something along the lines of: supported_versions = hc.template_versions.list() version_nums = [i.to_dict()['version'].split('.')[1] for i in supported_versions] mitaka_or_newer = [i for i in version_nums if i >= '2016-04-08'] Yes, I'm planning on cleaning that up before submitting it :) What I'm wondering is if I should make this into some sort of generalized utility method in the client, under the assumption that we'll need this sort of check in the future for the same backward compatibility requirements. So a few questions: 1. Does anyone strongly disagree to checking supported template versions as a way of determining the specifics of the server API. Ok, so some valid concerns have been raised over deriving things using the HOT version (although I do still wonder if the environment itself should be versioned, just like the templates, then we could rev the environment verion and say it supports a list, vs changing anything in the API, but that's probably a separate discussion). Taking a step back for a moment, the original discussion was around providing transparent access to the new interface via heatclient, but that isn't actually a hard requirement - the old interface works fine for many users, so we could just introduce a new interface (which would eventually become the default, after all non-EOL heat versions released support the new API argument): Currently we do: heat stack-create foo -f foo.yaml -e a.yaml -e b.yaml And this implies some client-side resolution of the multiple -e arguments. -e is short for "--environment-file", but we could introduce a new format, e.g "-E", short for "--environment-files": heat stack-create foo -f foo.yaml -E a.yaml -E b.yaml This option would work the same way as the current interface, but it would pass the files unmodified for resolution inside heat (by using the new API format), and as it's opt-in, it's leaving all the current heatclient interfaces alone without any internal fallback logic? That would certainly work, but it sounds like a usability/support nightmare :( Is there a reason we wouldn't consider bumping the API version to 1.1 for this? We'll have to figure out how to do it some time. cheers, Zane. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat] Client checking of server version
On 06/01/16 08:53, Jay Dobies wrote: I ran into an issue in a review about moving environment resolution from client to server [1]. It revolves around clients being able to access older versions of servers (that's a pretty simplistic description; see [2] for the spec). Before the holiday, Steve Hardy and I were talking about the complications involved. In my case, there's no good way to differentiate an older server from a legitimate error. Hmmm, it's true that you'll likely just get a 400 error, but I'd hope that the error message is at least somewhat unique. Unfortunately, it's not, but I don't think it's due to a Heat problem so much as just the nature of the issue. Here's what's happening. Ah, OK, so we are silently ignoring invalid data in the request here: http://git.openstack.org/cgit/openstack/heat/tree/heat/api/openstack/v1/stacks.py#n359 That seems like a bug to me. We could fix that and backport it to at least a couple of releases. It would mean that the client would still be broken on any older release that didn't have the patch though. New Client: doesn't do client-side environment resolution before sending it to the server. Old Server: expects the environment to be fully populated and ignores the environment file(s) in the files dict. The result is the server spits back an error saying that, in my scenario, there is no type mapping for jdob::Resource1. The problem is, I get the exact same result for New Client + New Server + incomplete environment files. The reason I was looking for some sort of version checking is to avoid having logic that just says "Maybe it's because it's an old server, lemme resolve the environments and send the request again." It feels really wrong to trigger two create requests when it's the templates themselves that are wrong. Agree. Since the API isn't versioned to the extent that we can leverage that I mean... it totally is but so far we've chosen not to bump that version. And we mostly got away with it because we were only adding functionality. So far. value, I was looking into using the template versions call. Something along the lines of: supported_versions = hc.template_versions.list() version_nums = [i.to_dict()['version'].split('.')[1] for i in supported_versions] mitaka_or_newer = [i for i in version_nums if i >= '2016-04-08'] Yes, I'm planning on cleaning that up before submitting it :) What I'm wondering is if I should make this into some sort of generalized utility method in the client, under the assumption that we'll need this sort of check in the future for the same backward compatibility requirements. So a few questions: 1. Does anyone strongly disagree to checking supported template versions as a way of determining the specifics of the server API. Yes. Template versions are supposed to be pluggable, and are explicitly under control of the operator. We shouldn't be systematically inferring anything about the server version based on this; in general there's no causal relationship. 2. Does anything like this already exist that I can use? Not really; there's the "heat build-info" command, but that is also explicitly under the control of the operator (and is empty by default). 3. If not, any suggestions on where I should put it? I see a heat.common.utils module but I'm not sure if there is a convention against that module (or common in general) making live server calls. Thanks :D [1] https://review.openstack.org/#/c/239504/ [2] https://review.openstack.org/#/c/226157/ __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat] Client checking of server version
On 01/05/2016 04:37 PM, Steven Hardy wrote: On Mon, Jan 04, 2016 at 03:53:07PM -0500, Jay Dobies wrote: I ran into an issue in a review about moving environment resolution from client to server [1]. It revolves around clients being able to access older versions of servers (that's a pretty simplistic description; see [2] for the spec). Before the holiday, Steve Hardy and I were talking about the complications involved. In my case, there's no good way to differentiate an older server from a legitimate error. Since the API isn't versioned to the extent that we can leverage that value, I was looking into using the template versions call. Something along the lines of: supported_versions = hc.template_versions.list() version_nums = [i.to_dict()['version'].split('.')[1] for i in supported_versions] mitaka_or_newer = [i for i in version_nums if i >= '2016-04-08'] Yes, I'm planning on cleaning that up before submitting it :) What I'm wondering is if I should make this into some sort of generalized utility method in the client, under the assumption that we'll need this sort of check in the future for the same backward compatibility requirements. So a few questions: 1. Does anyone strongly disagree to checking supported template versions as a way of determining the specifics of the server API. Ok, so some valid concerns have been raised over deriving things using the HOT version (although I do still wonder if the environment itself should be versioned, just like the templates, then we could rev the environment verion and say it supports a list, vs changing anything in the API, but that's probably a separate discussion). Taking a step back for a moment, the original discussion was around providing transparent access to the new interface via heatclient, but that isn't actually a hard requirement - the old interface works fine for many users, so we could just introduce a new interface (which would eventually become the default, after all non-EOL heat versions released support the new API argument): Currently we do: heat stack-create foo -f foo.yaml -e a.yaml -e b.yaml And this implies some client-side resolution of the multiple -e arguments. -e is short for "--environment-file", but we could introduce a new format, e.g "-E", short for "--environment-files": heat stack-create foo -f foo.yaml -E a.yaml -E b.yaml This option would work the same way as the current interface, but it would pass the files unmodified for resolution inside heat (by using the new API format), and as it's opt-in, it's leaving all the current heatclient interfaces alone without any internal fallback logic? +1 My only concern is that the default isn't to exercise the "preferred" approach. However, perhaps I'm viewing things wrong with that as being preferred instead of just an alternate for non-heatclient. IIRC, the code is largely the same, just being called from two separate places (client v. server), so it's not an issue of duplication or the actual logic growing stale. And it shouldn't really be an issue of the server-side path accidentally breaking since there is CI around it. So maybe my concerns are overblown. It does feel weird to have to document something like that, trying to describe the differences between -e and -E, but I suppose if we mark -e as deprecated it should be understandable enough. This also has the benefit of letting this code land without having to do a major implementation of micro-versions, so that's a plus :) Steve __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat] Client checking of server version
I ran into an issue in a review about moving environment resolution from client to server [1]. It revolves around clients being able to access older versions of servers (that's a pretty simplistic description; see [2] for the spec). Before the holiday, Steve Hardy and I were talking about the complications involved. In my case, there's no good way to differentiate an older server from a legitimate error. Hmmm, it's true that you'll likely just get a 400 error, but I'd hope that the error message is at least somewhat unique. Unfortunately, it's not, but I don't think it's due to a Heat problem so much as just the nature of the issue. Here's what's happening. New Client: doesn't do client-side environment resolution before sending it to the server. Old Server: expects the environment to be fully populated and ignores the environment file(s) in the files dict. The result is the server spits back an error saying that, in my scenario, there is no type mapping for jdob::Resource1. The problem is, I get the exact same result for New Client + New Server + incomplete environment files. The reason I was looking for some sort of version checking is to avoid having logic that just says "Maybe it's because it's an old server, lemme resolve the environments and send the request again." It feels really wrong to trigger two create requests when it's the templates themselves that are wrong. Since the API isn't versioned to the extent that we can leverage that I mean... it totally is but so far we've chosen not to bump that version. And we mostly got away with it because we were only adding functionality. So far. value, I was looking into using the template versions call. Something along the lines of: supported_versions = hc.template_versions.list() version_nums = [i.to_dict()['version'].split('.')[1] for i in supported_versions] mitaka_or_newer = [i for i in version_nums if i >= '2016-04-08'] Yes, I'm planning on cleaning that up before submitting it :) What I'm wondering is if I should make this into some sort of generalized utility method in the client, under the assumption that we'll need this sort of check in the future for the same backward compatibility requirements. So a few questions: 1. Does anyone strongly disagree to checking supported template versions as a way of determining the specifics of the server API. Yes. Template versions are supposed to be pluggable, and are explicitly under control of the operator. We shouldn't be systematically inferring anything about the server version based on this; in general there's no causal relationship. 2. Does anything like this already exist that I can use? Not really; there's the "heat build-info" command, but that is also explicitly under the control of the operator (and is empty by default). 3. If not, any suggestions on where I should put it? I see a heat.common.utils module but I'm not sure if there is a convention against that module (or common in general) making live server calls. Thanks :D [1] https://review.openstack.org/#/c/239504/ [2] https://review.openstack.org/#/c/226157/ __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat] Client checking of server version
On Mon, Jan 04, 2016 at 03:53:07PM -0500, Jay Dobies wrote: > I ran into an issue in a review about moving environment resolution from > client to server [1]. It revolves around clients being able to access older > versions of servers (that's a pretty simplistic description; see [2] for the > spec). > > Before the holiday, Steve Hardy and I were talking about the complications > involved. In my case, there's no good way to differentiate an older server > from a legitimate error. > > Since the API isn't versioned to the extent that we can leverage that value, > I was looking into using the template versions call. Something along the > lines of: > > supported_versions = hc.template_versions.list() > version_nums = [i.to_dict()['version'].split('.')[1] for i in > supported_versions] > mitaka_or_newer = [i for i in version_nums if i >= '2016-04-08'] > > Yes, I'm planning on cleaning that up before submitting it :) > > What I'm wondering is if I should make this into some sort of generalized > utility method in the client, under the assumption that we'll need this sort > of check in the future for the same backward compatibility requirements. > > So a few questions: > > 1. Does anyone strongly disagree to checking supported template versions as > a way of determining the specifics of the server API. Ok, so some valid concerns have been raised over deriving things using the HOT version (although I do still wonder if the environment itself should be versioned, just like the templates, then we could rev the environment verion and say it supports a list, vs changing anything in the API, but that's probably a separate discussion). Taking a step back for a moment, the original discussion was around providing transparent access to the new interface via heatclient, but that isn't actually a hard requirement - the old interface works fine for many users, so we could just introduce a new interface (which would eventually become the default, after all non-EOL heat versions released support the new API argument): Currently we do: heat stack-create foo -f foo.yaml -e a.yaml -e b.yaml And this implies some client-side resolution of the multiple -e arguments. -e is short for "--environment-file", but we could introduce a new format, e.g "-E", short for "--environment-files": heat stack-create foo -f foo.yaml -E a.yaml -E b.yaml This option would work the same way as the current interface, but it would pass the files unmodified for resolution inside heat (by using the new API format), and as it's opt-in, it's leaving all the current heatclient interfaces alone without any internal fallback logic? Steve __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat] Client checking of server version
On 04/01/16 15:53, Jay Dobies wrote: I ran into an issue in a review about moving environment resolution from client to server [1]. It revolves around clients being able to access older versions of servers (that's a pretty simplistic description; see [2] for the spec). Before the holiday, Steve Hardy and I were talking about the complications involved. In my case, there's no good way to differentiate an older server from a legitimate error. Hmmm, it's true that you'll likely just get a 400 error, but I'd hope that the error message is at least somewhat unique. Since the API isn't versioned to the extent that we can leverage that I mean... it totally is but so far we've chosen not to bump that version. And we mostly got away with it because we were only adding functionality. So far. value, I was looking into using the template versions call. Something along the lines of: supported_versions = hc.template_versions.list() version_nums = [i.to_dict()['version'].split('.')[1] for i in supported_versions] mitaka_or_newer = [i for i in version_nums if i >= '2016-04-08'] Yes, I'm planning on cleaning that up before submitting it :) What I'm wondering is if I should make this into some sort of generalized utility method in the client, under the assumption that we'll need this sort of check in the future for the same backward compatibility requirements. So a few questions: 1. Does anyone strongly disagree to checking supported template versions as a way of determining the specifics of the server API. Yes. Template versions are supposed to be pluggable, and are explicitly under control of the operator. We shouldn't be systematically inferring anything about the server version based on this; in general there's no causal relationship. 2. Does anything like this already exist that I can use? Not really; there's the "heat build-info" command, but that is also explicitly under the control of the operator (and is empty by default). 3. If not, any suggestions on where I should put it? I see a heat.common.utils module but I'm not sure if there is a convention against that module (or common in general) making live server calls. Thanks :D [1] https://review.openstack.org/#/c/239504/ [2] https://review.openstack.org/#/c/226157/ __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat] Client checking of server version
Excerpts from Jay Dobies's message of 2016-01-04 12:53:07 -0800: > I ran into an issue in a review about moving environment resolution from > client to server [1]. It revolves around clients being able to access > older versions of servers (that's a pretty simplistic description; see > [2] for the spec). > > Before the holiday, Steve Hardy and I were talking about the > complications involved. In my case, there's no good way to differentiate > an older server from a legitimate error. > > Since the API isn't versioned to the extent that we can leverage that > value, I was looking into using the template versions call. Something > along the lines of: > >supported_versions = hc.template_versions.list() >version_nums = [i.to_dict()['version'].split('.')[1] for i in > supported_versions] >mitaka_or_newer = [i for i in version_nums if i >= '2016-04-08'] > > Yes, I'm planning on cleaning that up before submitting it :) > > What I'm wondering is if I should make this into some sort of > generalized utility method in the client, under the assumption that > we'll need this sort of check in the future for the same backward > compatibility requirements. > > So a few questions: > > 1. Does anyone strongly disagree to checking supported template versions > as a way of determining the specifics of the server API. > I strongly disagree with anything that requires every clientn in every language that wants to operate with older and newer services to have the same clever knowledge implemented. Try writing the REST documentation for it: "If you want to do something with environments, query the template portion of the API, and if you can use templates newer than 2016-04-18, then you can use the new environments feature. Do not be confused by this, it is totally unrelated to the template format you are using." Sounds like it is time to micro-version Heat's API. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev