Re: [openstack-dev] Ugly Hack to deal with multiple versions
Can you be more specific about what goes wrong here? I'm not entirely sure I understand why an old client of arbitrary age needs to be supported with new OpenStack. The contract is the API, not the client, and an old client that doesn't do version discovery is just a buggy client from what I'm concerned. Time to release a new version. I also wonder if this is an issue with version discovery implementation. It seems like if we think this is going to be affecting multiple services before doing an odd hack for keystone, we should actually figure out a pattern that works for all services, and figure out why this has only just become an issue. Most of the other services have done dual APIs at some point over the last 2 years, and this didn't seem to trip them up too badly. What happened differently in keystone that made this an issue? And what can be learned about how we structure APIs going forward. -Sean On 02/04/2014 04:50 AM, Adam Young wrote: We have to support old clients. Old clients expect that the URL that comes back for the service catalog has the version in it. Old clients don't do version negotiation. Thus, we need an approach to not-break old clients while we politely encourage the rest of the world to move to later APIs. I know Keystone has this problem. I've heard that some of the other services do as well. Here is what I propose. It is ugly, but it is a transition plan, and can be disabled once the old clients are deprecated: HACK: In a new client, look at the URL. If it ends with /v2.0, chop it off and us the substring up to that point. Now, at this point you are probably going: That is ugly, is it really necessary? Can't we do something more correct? No. I mean, we are already doing something more correct in that the later versions of the Keystone client already support version discovery. The problem is that older clients don't. Alternatives: 1. Just chop the url. Now only clients smart enough to do negotiation work. Older clients no longer work. Suck it up. 2. Put multiple endpoints in the service catalog. Ugh. Now we've just doubled the size of the service catalog, and we need new logic to get the identityv3 endpoints, cuz we need to leave identity endpoints for existing clients. 3. Do some sort of magic on the server side to figure out the right URL to respond to the client request. This kind of magic was banned under the wizarding convention of '89. Can we accept that this is necessary, and vow to never let this happen again by removing the versions from the URLs after the current set of clients are deprecated? ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Sean Dague Samsung Research America s...@dague.net / sean.da...@samsung.com http://dague.net signature.asc Description: OpenPGP digital signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Ugly Hack to deal with multiple versions
On Tue, Feb 4, 2014 at 9:00 AM, Sean Dague s...@dague.net wrote: Can you be more specific about what goes wrong here? I'm not entirely sure I understand why an old client of arbitrary age needs to be supported with new OpenStack. The contract is the API, not the client, and an old client that doesn't do version discovery is just a buggy client from what I'm concerned. Time to release a new version. Problem 1: API version discovery is not universally considered to be part of the API and therefore is not defined by most services beyond them responding to a '/' request with a 300 response and a list of versions. No two of these responses look alike except where the source was copied from an existing service. Problem 2: Identity is unique in that it is handed a deployment-defined URL to authenticate and get endpoints for all other services. Most of these auth URLs have a version hard-coded in them because the client didn't do version discovery or negotiation until recently. This is what we're talking about here, how to remove the version from this URL and not break old clients. We can't. Not without doing nasty things like detecting an old client and compensating for it server-side. So we have to work out a way for new clients to do discovery even when handed a URL that has a version in it. I've tested a couple of more generalized approaches, and the best solution I have found so far is to simply special-case the known legacy behaviour then drop in to the general discovery process. I also wonder if this is an issue with version discovery implementation. It seems like if we think this is going to be affecting multiple services before doing an odd hack for keystone, we should actually figure out a pattern that works for all services, and figure out why this has only just become an issue. Most of the other services have done The services that traditionally embed a version inside the URL followed by a tenant ID or something get even deeper into parsing the URL to hack the version. dual APIs at some point over the last 2 years, and this didn't seem to trip them up too badly. What happened differently in keystone that made this an issue? And what can be learned about how we structure APIs going forward. I think the difference is this is the first API we have actually tried to deprecate and we don't have the option to hide it in an updated SC endpoint. The service catalog has hidden a lot of this pain for other services because the clients generally can use whatever endpoint the SC gives it. a) Version discovery needs to be rationalized across the services. We've talked about this at summits before, and proposals have been written. And here we are. We'll do it again in Atlanta, hopefully for the last time. b) Define a common structured endpoint and let the client assemble the components into the final URL. If the service catalog had a base URL for compute, and a list of versions, and the additional bits to be appended the client could make an intelligent choice and assemble the endpoint. It isn't like the client doesn't already have to know how the REST URLs are constructed. b-alt) Stop putting things like tenant IDs in the SC. This has the same issue as the auth URL in how to do this without instantly breaking the existing clients. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Ugly Hack to deal with multiple versions
On 02/05/2014 01:09 AM, Dean Troyer wrote: On Tue, Feb 4, 2014 at 9:00 AM, Sean Dague s...@dague.net mailto:s...@dague.net wrote: Can you be more specific about what goes wrong here? I'm not entirely sure I understand why an old client of arbitrary age needs to be supported with new OpenStack. The contract is the API, not the client, and an old client that doesn't do version discovery is just a buggy client from what I'm concerned. Time to release a new version. Problem 1: API version discovery is not universally considered to be part of the API and therefore is not defined by most services beyond them responding to a '/' request with a 300 response and a list of versions. No two of these responses look alike except where the source was copied from an existing service. Problem 2: Identity is unique in that it is handed a deployment-defined URL to authenticate and get endpoints for all other services. Most of these auth URLs have a version hard-coded in them because the client didn't do version discovery or negotiation until recently. This is what we're talking about here, how to remove the version from this URL and not break old clients. We can't. Not without doing nasty things like detecting an old client and compensating for it server-side. So we have to work out a way for new clients to do discovery even when handed a URL that has a version in it. I've tested a couple of more generalized approaches, and the best solution I have found so far is to simply special-case the known legacy behaviour then drop in to the general discovery process. I also wonder if this is an issue with version discovery implementation. It seems like if we think this is going to be affecting multiple services before doing an odd hack for keystone, we should actually figure out a pattern that works for all services, and figure out why this has only just become an issue. Most of the other services have done The services that traditionally embed a version inside the URL followed by a tenant ID or something get even deeper into parsing the URL to hack the version. dual APIs at some point over the last 2 years, and this didn't seem to trip them up too badly. What happened differently in keystone that made this an issue? And what can be learned about how we structure APIs going forward. I think the difference is this is the first API we have actually tried to deprecate and we don't have the option to hide it in an updated SC endpoint. The service catalog has hidden a lot of this pain for other services because the clients generally can use whatever endpoint the SC gives it. a) Version discovery needs to be rationalized across the services. We've talked about this at summits before, and proposals have been written. And here we are. We'll do it again in Atlanta, hopefully for the last time. b) Define a common structured endpoint and let the client assemble the components into the final URL. If the service catalog had a base URL for compute, and a list of versions, and the additional bits to be appended the client could make an intelligent choice and assemble the endpoint. It isn't like the client doesn't already have to know how the REST URLs are constructed. b-alt) Stop putting things like tenant IDs in the SC. This has the same issue as the auth URL in how to do this without instantly breaking the existing clients. Ok, much clearer now to me (though I'll still claim jetlag for some bits not sinking in). I think a really important thing to keep in mind is any solution that's implemented client side, is something that all the other OpenStack SDKs are going to have to implement as well. So an ugly hack isn't just python-keystone... and be done. It's also just hoisted doing that ugly hack on the php / go sdk teams, jclouds, deltacloud, etc. Something they may not be aware is going to break them, or their users. So we really need version discovery rationalized once and for all, otherwise this nightmare goes far beyond something that's fixable within software that we control. -Sean -- Sean Dague Samsung Research America s...@dague.net / sean.da...@samsung.com http://dague.net signature.asc Description: OpenPGP digital signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Ugly Hack to deal with multiple versions
On 02/04/2014 11:09 AM, Dean Troyer wrote: On Tue, Feb 4, 2014 at 9:00 AM, Sean Dague s...@dague.net mailto:s...@dague.net wrote: Can you be more specific about what goes wrong here? I'm not entirely sure I understand why an old client of arbitrary age needs to be supported with new OpenStack. The contract is the API, not the client, and an old client that doesn't do version discovery is just a buggy client from what I'm concerned. Time to release a new version. Remember: the client is not the only code that Keystone has to worry about. THere is also a whole cotteage industry that is talking direct to the endpoints themselves. We've trained these poor souls to expect the version in the Endpoint. So even if we said we are only supporting the latest Keystone client we'd still have to deal with code that expects the identity endpoint URL to look like this: https://hostname:35357/v2.0/ We want people to be able to call on the V3 API. To be able to do version discovery, the identity endpoint URL should look like this: https://hostname:35357/ And then they can navigate down to https://hostname:35357/v3 Via discovery. So, what I am proposing for Keystone (here: https://review.openstack.org/#/c/62801/) is that we do this If the Catalog endpoint looks like this: https://hostname:35357/v2.0/ chop off /v2.0 and call on it for discover. Now, the code In that link *will* work the same on a https://hostname:35357/v3/ due to the regex (copied from termie who recognized this problem long ago) and that may scare you into saying won't we have this problem in the future? Maybe, if we are stupid. We've been stupid before, so I can't promise anything. However, much more important is that, as the tools for deployment get smarter, they stop putting out Service catalogs with versions in the URLS. They would have to deliberately replace an explicit /v3 into the url in place of the V2.0 Then, yes, the hack would still silently work. But without the hack, if they only check against the the V3 API, it will still work, too. Broken again, and we are back herethis really is something that requires education across the deployers. Problem 1: API version discovery is not universally considered to be part of the API and therefore is not defined by most services beyond them responding to a '/' request with a 300 response and a list of versions. No two of these responses look alike except where the source was copied from an existing service. Problem 2: Identity is unique in that it is handed a deployment-defined URL to authenticate and get endpoints for all other services. Most of these auth URLs have a version hard-coded in them because the client didn't do version discovery or negotiation until recently. This is what we're talking about here, how to remove the version from this URL and not break old clients. We can't. Not without doing nasty things like detecting an old client and compensating for it server-side. So we have to work out a way for new clients to do discovery even when handed a URL that has a version in it. I've tested a couple of more generalized approaches, and the best solution I have found so far is to simply special-case the known legacy behaviour then drop in to the general discovery process. I also wonder if this is an issue with version discovery implementation. It seems like if we think this is going to be affecting multiple services before doing an odd hack for keystone, we should actually figure out a pattern that works for all services, and figure out why this has only just become an issue. Most of the other services have done The services that traditionally embed a version inside the URL followed by a tenant ID or something get even deeper into parsing the URL to hack the version. dual APIs at some point over the last 2 years, and this didn't seem to trip them up too badly. What happened differently in keystone that made this an issue? And what can be learned about how we structure APIs going forward. I think the difference is this is the first API we have actually tried to deprecate and we don't have the option to hide it in an updated SC endpoint. The service catalog has hidden a lot of this pain for other services because the clients generally can use whatever endpoint the SC gives it. a) Version discovery needs to be rationalized across the services. We've talked about this at summits before, and proposals have been written. And here we are. We'll do it again in Atlanta, hopefully for the last time. b) Define a common structured endpoint and let the client assemble the components into the final URL. If the service catalog had a base URL for compute, and a list of versions, and the additional bits to be appended the client could make an intelligent choice and assemble the endpoint. It isn't like the client doesn't
Re: [openstack-dev] Ugly Hack to deal with multiple versions
On Feb 4, 2014, at 10:31 AM, Sean Dague s...@dague.net wrote: On 02/05/2014 01:09 AM, Dean Troyer wrote: On Tue, Feb 4, 2014 at 9:00 AM, Sean Dague s...@dague.net mailto:s...@dague.net wrote: Can you be more specific about what goes wrong here? I'm not entirely sure I understand why an old client of arbitrary age needs to be supported with new OpenStack. The contract is the API, not the client, and an old client that doesn't do version discovery is just a buggy client from what I'm concerned. Time to release a new version. Problem 1: API version discovery is not universally considered to be part of the API and therefore is not defined by most services beyond them responding to a '/' request with a 300 response and a list of versions. No two of these responses look alike except where the source was copied from an existing service. Problem 2: Identity is unique in that it is handed a deployment-defined URL to authenticate and get endpoints for all other services. Most of these auth URLs have a version hard-coded in them because the client didn't do version discovery or negotiation until recently. This is what we're talking about here, how to remove the version from this URL and not break old clients. We can't. Not without doing nasty things like detecting an old client and compensating for it server-side. So we have to work out a way for new clients to do discovery even when handed a URL that has a version in it. I've tested a couple of more generalized approaches, and the best solution I have found so far is to simply special-case the known legacy behaviour then drop in to the general discovery process. I also wonder if this is an issue with version discovery implementation. It seems like if we think this is going to be affecting multiple services before doing an odd hack for keystone, we should actually figure out a pattern that works for all services, and figure out why this has only just become an issue. Most of the other services have done The services that traditionally embed a version inside the URL followed by a tenant ID or something get even deeper into parsing the URL to hack the version. dual APIs at some point over the last 2 years, and this didn't seem to trip them up too badly. What happened differently in keystone that made this an issue? And what can be learned about how we structure APIs going forward. I think the difference is this is the first API we have actually tried to deprecate and we don't have the option to hide it in an updated SC endpoint. The service catalog has hidden a lot of this pain for other services because the clients generally can use whatever endpoint the SC gives it. a) Version discovery needs to be rationalized across the services. We've talked about this at summits before, and proposals have been written. And here we are. We'll do it again in Atlanta, hopefully for the last time. b) Define a common structured endpoint and let the client assemble the components into the final URL. If the service catalog had a base URL for compute, and a list of versions, and the additional bits to be appended the client could make an intelligent choice and assemble the endpoint. It isn't like the client doesn't already have to know how the REST URLs are constructed. b-alt) Stop putting things like tenant IDs in the SC. This has the same issue as the auth URL in how to do this without instantly breaking the existing clients. Ok, much clearer now to me (though I'll still claim jetlag for some bits not sinking in). I think a really important thing to keep in mind is any solution that's implemented client side, is something that all the other OpenStack SDKs are going to have to implement as well. So an ugly hack isn't just python-keystone... and be done. It's also just hoisted doing that ugly hack on the php / go sdk teams, jclouds, deltacloud, etc. Something they may not be aware is going to break them, or their users. Do we have official openstack PHP / go SDKs? So we really need version discovery rationalized once and for all, otherwise this nightmare goes far beyond something that's fixable within software that we control. -Sean -- Sean Dague Samsung Research America s...@dague.net / sean.da...@samsung.com http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Ugly Hack to deal with multiple versions
- Original Message -
From: Dean Troyer dtro...@gmail.com
To: OpenStack Development Mailing List (not for usage questions)
openstack-dev@lists.openstack.org
Sent: Tuesday, February 4, 2014 9:31:57 AM
Subject: Re: [openstack-dev] Ugly Hack to deal with multiple versions
On Mon, Feb 3, 2014 at 1:50 PM, Adam Young ayo...@redhat.com wrote:
HACK: In a new client, look at the URL. If it ends with /v2.0, chop it off
and us the substring up to that point.
Now, at this point you are probably going: That is ugly, is it really
necessary? Can't we do something more correct?
At this point I think we are stuck with hard-coding some legacy compatibility
like this for the near future. Fortunately Identity is an easy one to
handle, Compute is going to be a #$^%! as the commonly documented case has a
version not at the end.
I've been playing with variations on this strategy and I think it is our
least bad option...
Can we accept that this is necessary, and vow to never let this happen again
by removing the versions from the URLs after the current set of clients are
deprecated?
+1
There is another hack to think about: if public_endpoint and/or
admin_endpoint are not set in keystone.conf, all of the discovered urls use
localhost: http://localhost:8770/v2.0/ . Discovery falls over aga
I don't know how common this is but I have encountered it at least once or
twice. Is this the only place those config values are used? It seems like a
better default could be worked out here too; is 'localhost' ever the right
thing to advertise in a real-world deployment?
I've not come across that one so much as something similar i've been wanting to
raise. Version discovery happens across either a public or an internal URL
and we retrieve the version list which has links to version endpoints.
These links in discovery are fully qualified and so it doesn't matter if you
connect to the server across the internal or public address you will always end
up communicating with the links specified in the config's public_endpoint
value.
I propose for this situation we allow (and make default asap) that links in
discovery can be relative links in the standard a href= / manner (ie
python's urljoin rather than the just append it that we do now) joined onto
the URL that took us to the page.
So keystone:
{'versions': {'values': [{'id': 'v3.0',
'links': [{'href': '/v3/',
'rel': 'self'}],
'status': 'stable',
'updated': '2013-03-06T00:00:00Z'},
{'id': 'v2.0',
'links': [{'href': '/v2.0/',
'rel': 'self'},
'status': 'stable',
'updated': '2013-03-06T00:00:00Z'}]}}
dt
--
Dean Troyer
dtro...@gmail.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Ugly Hack to deal with multiple versions
- Original Message - From: Adam Young ayo...@redhat.com To: openstack-dev@lists.openstack.org Sent: Wednesday, February 5, 2014 2:29:18 AM Subject: Re: [openstack-dev] Ugly Hack to deal with multiple versions On 02/04/2014 11:09 AM, Dean Troyer wrote: On Tue, Feb 4, 2014 at 9:00 AM, Sean Dague s...@dague.net wrote: Can you be more specific about what goes wrong here? I'm not entirely sure I understand why an old client of arbitrary age needs to be supported with new OpenStack. The contract is the API, not the client, and an old client that doesn't do version discovery is just a buggy client from what I'm concerned. Time to release a new version. Remember: the client is not the only code that Keystone has to worry about. THere is also a whole cotteage industry that is talking direct to the endpoints themselves. We've trained these poor souls to expect the version in the Endpoint. So even if we said we are only supporting the latest Keystone client we'd still have to deal with code that expects the identity endpoint URL to look like this: https://hostname:35357/v2.0/ We want people to be able to call on the V3 API. To be able to do version discovery, the identity endpoint URL should look like this: https://hostname:35357/ And then they can navigate down to https://hostname:35357/v3 Via discovery. So, what I am proposing for Keystone (here: https://review.openstack.org/#/c/62801/ ) is that we do this If the Catalog endpoint looks like this: https://hostname:35357/v2.0/ chop off /v2.0 and call on it for discover. Now, the code In that link *will* work the same on a https://hostname:35357/v3/ due to the regex (copied from termie who recognized this problem long ago) and that may scare you into saying won't we have this problem in the future? Maybe, if we are stupid. We've been stupid before, so I can't promise anything. However, much more important is that, as the tools for deployment get smarter, they stop putting out Service catalogs with versions in the URLS. They would have to deliberately replace an explicit /v3 into the url in place of the V2.0 Then, yes, the hack would still silently work. But without the hack, if they only check against the the V3 API, it will still work, too. Broken again, and we are back herethis really is something that requires education across the deployers. So we've been around this problem a few times and i've tried every other solution i can think of and it never quite works so i pretty much agree. What i'd like to know and see really well defined is how we get out of this position once we're in it. Yes we can educate deployers and such but at some point we have the situation where our clients are going to be able to accept an endpoint from the service catalog that is wrong and we'll need to deprecate that behaviour. The ripping a band-aid metaphor seems appropriate here, When do we get to deprecate this and are we just delaying (possibly increasing as now we have to deprecate changes on the server and on the client) that pain that we could just declare a one-off hit for? (Note that i am well aware that even our own clients cannot handle this currently, things would still need to be fixed) Problem 1: API version discovery is not universally considered to be part of the API and therefore is not defined by most services beyond them responding to a '/' request with a 300 response and a list of versions. No two of these responses look alike except where the source was copied from an existing service. Problem 2: Identity is unique in that it is handed a deployment-defined URL to authenticate and get endpoints for all other services. Most of these auth URLs have a version hard-coded in them because the client didn't do version discovery or negotiation until recently. This is what we're talking about here, how to remove the version from this URL and not break old clients. We can't. Not without doing nasty things like detecting an old client and compensating for it server-side. So we have to work out a way for new clients to do discovery even when handed a URL that has a version in it. I've tested a couple of more generalized approaches, and the best solution I have found so far is to simply special-case the known legacy behaviour then drop in to the general discovery process. I also wonder if this is an issue with version discovery implementation. It seems like if we think this is going to be affecting multiple services before doing an odd hack for keystone, we should actually figure out a pattern that works for all services, and figure out why this has only just become an issue. Most of the other services have done The services that traditionally embed a version inside the URL followed by a tenant ID or something get even deeper into parsing the URL to hack
Re: [openstack-dev] Ugly Hack to deal with multiple versions
On 02/05/2014 01:50 AM, Jesse Noller wrote: On Feb 4, 2014, at 10:31 AM, Sean Dague s...@dague.net wrote: On 02/05/2014 01:09 AM, Dean Troyer wrote: On Tue, Feb 4, 2014 at 9:00 AM, Sean Dague s...@dague.net mailto:s...@dague.net wrote: Can you be more specific about what goes wrong here? I'm not entirely sure I understand why an old client of arbitrary age needs to be supported with new OpenStack. The contract is the API, not the client, and an old client that doesn't do version discovery is just a buggy client from what I'm concerned. Time to release a new version. Problem 1: API version discovery is not universally considered to be part of the API and therefore is not defined by most services beyond them responding to a '/' request with a 300 response and a list of versions. No two of these responses look alike except where the source was copied from an existing service. Problem 2: Identity is unique in that it is handed a deployment-defined URL to authenticate and get endpoints for all other services. Most of these auth URLs have a version hard-coded in them because the client didn't do version discovery or negotiation until recently. This is what we're talking about here, how to remove the version from this URL and not break old clients. We can't. Not without doing nasty things like detecting an old client and compensating for it server-side. So we have to work out a way for new clients to do discovery even when handed a URL that has a version in it. I've tested a couple of more generalized approaches, and the best solution I have found so far is to simply special-case the known legacy behaviour then drop in to the general discovery process. I also wonder if this is an issue with version discovery implementation. It seems like if we think this is going to be affecting multiple services before doing an odd hack for keystone, we should actually figure out a pattern that works for all services, and figure out why this has only just become an issue. Most of the other services have done The services that traditionally embed a version inside the URL followed by a tenant ID or something get even deeper into parsing the URL to hack the version. dual APIs at some point over the last 2 years, and this didn't seem to trip them up too badly. What happened differently in keystone that made this an issue? And what can be learned about how we structure APIs going forward. I think the difference is this is the first API we have actually tried to deprecate and we don't have the option to hide it in an updated SC endpoint. The service catalog has hidden a lot of this pain for other services because the clients generally can use whatever endpoint the SC gives it. a) Version discovery needs to be rationalized across the services. We've talked about this at summits before, and proposals have been written. And here we are. We'll do it again in Atlanta, hopefully for the last time. b) Define a common structured endpoint and let the client assemble the components into the final URL. If the service catalog had a base URL for compute, and a list of versions, and the additional bits to be appended the client could make an intelligent choice and assemble the endpoint. It isn't like the client doesn't already have to know how the REST URLs are constructed. b-alt) Stop putting things like tenant IDs in the SC. This has the same issue as the auth URL in how to do this without instantly breaking the existing clients. Ok, much clearer now to me (though I'll still claim jetlag for some bits not sinking in). I think a really important thing to keep in mind is any solution that's implemented client side, is something that all the other OpenStack SDKs are going to have to implement as well. So an ugly hack isn't just python-keystone... and be done. It's also just hoisted doing that ugly hack on the php / go sdk teams, jclouds, deltacloud, etc. Something they may not be aware is going to break them, or their users. Do we have official openstack PHP / go SDKs? Official is a strong word, but we do have stackforge teams active on it: * https://github.com/stackforge/openstack-sdk-php * https://github.com/stackforge/golang-client And I think we should should be mindful of their work to make OpenStack easily accessible from their language communities. -Sean -- Sean Dague Samsung Research America s...@dague.net / sean.da...@samsung.com http://dague.net signature.asc Description: OpenPGP digital signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Ugly Hack to deal with multiple versions
On Feb 4, 2014, at 1:28 PM, Sean Dague s...@dague.net wrote: On 02/05/2014 01:50 AM, Jesse Noller wrote: On Feb 4, 2014, at 10:31 AM, Sean Dague s...@dague.net wrote: On 02/05/2014 01:09 AM, Dean Troyer wrote: On Tue, Feb 4, 2014 at 9:00 AM, Sean Dague s...@dague.net mailto:s...@dague.net wrote: Can you be more specific about what goes wrong here? I'm not entirely sure I understand why an old client of arbitrary age needs to be supported with new OpenStack. The contract is the API, not the client, and an old client that doesn't do version discovery is just a buggy client from what I'm concerned. Time to release a new version. Problem 1: API version discovery is not universally considered to be part of the API and therefore is not defined by most services beyond them responding to a '/' request with a 300 response and a list of versions. No two of these responses look alike except where the source was copied from an existing service. Problem 2: Identity is unique in that it is handed a deployment-defined URL to authenticate and get endpoints for all other services. Most of these auth URLs have a version hard-coded in them because the client didn't do version discovery or negotiation until recently. This is what we're talking about here, how to remove the version from this URL and not break old clients. We can't. Not without doing nasty things like detecting an old client and compensating for it server-side. So we have to work out a way for new clients to do discovery even when handed a URL that has a version in it. I've tested a couple of more generalized approaches, and the best solution I have found so far is to simply special-case the known legacy behaviour then drop in to the general discovery process. I also wonder if this is an issue with version discovery implementation. It seems like if we think this is going to be affecting multiple services before doing an odd hack for keystone, we should actually figure out a pattern that works for all services, and figure out why this has only just become an issue. Most of the other services have done The services that traditionally embed a version inside the URL followed by a tenant ID or something get even deeper into parsing the URL to hack the version. dual APIs at some point over the last 2 years, and this didn't seem to trip them up too badly. What happened differently in keystone that made this an issue? And what can be learned about how we structure APIs going forward. I think the difference is this is the first API we have actually tried to deprecate and we don't have the option to hide it in an updated SC endpoint. The service catalog has hidden a lot of this pain for other services because the clients generally can use whatever endpoint the SC gives it. a) Version discovery needs to be rationalized across the services. We've talked about this at summits before, and proposals have been written. And here we are. We'll do it again in Atlanta, hopefully for the last time. b) Define a common structured endpoint and let the client assemble the components into the final URL. If the service catalog had a base URL for compute, and a list of versions, and the additional bits to be appended the client could make an intelligent choice and assemble the endpoint. It isn't like the client doesn't already have to know how the REST URLs are constructed. b-alt) Stop putting things like tenant IDs in the SC. This has the same issue as the auth URL in how to do this without instantly breaking the existing clients. Ok, much clearer now to me (though I'll still claim jetlag for some bits not sinking in). I think a really important thing to keep in mind is any solution that's implemented client side, is something that all the other OpenStack SDKs are going to have to implement as well. So an ugly hack isn't just python-keystone... and be done. It's also just hoisted doing that ugly hack on the php / go sdk teams, jclouds, deltacloud, etc. Something they may not be aware is going to break them, or their users. Do we have official openstack PHP / go SDKs? Official is a strong word, but we do have stackforge teams active on it: * https://github.com/stackforge/openstack-sdk-php * https://github.com/stackforge/golang-client And I think we should should be mindful of their work to make OpenStack easily accessible from their language communities. Oh for sure - but I was wondering because there’s also: jclouds (java): http://jclouds.apache.org/ php-opencloud (php): https://github.com/rackspace/php-opencloud fog (ruby): http://fog.io/ libcloud (python): http://libcloud.apache.org/ gophercloud (go): https://github.com/rackspace/gophercloud openstack .net (.net): https://github.com/rackspace/openstack.net So finding a go client and php one one stackforge is surprising - wondering if we can combine efforts - on my end
Re: [openstack-dev] Ugly Hack to deal with multiple versions
It would really be nice to be able to hit an endpoint like https://hostname:35357/ and get a list of available versions back. It would make discovery easier. But, for scripting languages like PHP this would add a lot of extra http requests unless you cache the location between page views/requests. We'd need to be smart about it. I would mind the both and approach. if /v3 is appended it's smart enough in an SDK to know what to do. If it's not there it's smart enough to hit the top level where a list might be available of what's supported. On Tue, Feb 4, 2014 at 2:55 PM, Jesse Noller jesse.nol...@rackspace.comwrote: On Feb 4, 2014, at 1:28 PM, Sean Dague s...@dague.net wrote: On 02/05/2014 01:50 AM, Jesse Noller wrote: On Feb 4, 2014, at 10:31 AM, Sean Dague s...@dague.net wrote: On 02/05/2014 01:09 AM, Dean Troyer wrote: On Tue, Feb 4, 2014 at 9:00 AM, Sean Dague s...@dague.net mailto:s...@dague.net wrote: Can you be more specific about what goes wrong here? I'm not entirely sure I understand why an old client of arbitrary age needs to be supported with new OpenStack. The contract is the API, not the client, and an old client that doesn't do version discovery is just a buggy client from what I'm concerned. Time to release a new version. Problem 1: API version discovery is not universally considered to be part of the API and therefore is not defined by most services beyond them responding to a '/' request with a 300 response and a list of versions. No two of these responses look alike except where the source was copied from an existing service. Problem 2: Identity is unique in that it is handed a deployment-defined URL to authenticate and get endpoints for all other services. Most of these auth URLs have a version hard-coded in them because the client didn't do version discovery or negotiation until recently. This is what we're talking about here, how to remove the version from this URL and not break old clients. We can't. Not without doing nasty things like detecting an old client and compensating for it server-side. So we have to work out a way for new clients to do discovery even when handed a URL that has a version in it. I've tested a couple of more generalized approaches, and the best solution I have found so far is to simply special-case the known legacy behaviour then drop in to the general discovery process. I also wonder if this is an issue with version discovery implementation. It seems like if we think this is going to be affecting multiple services before doing an odd hack for keystone, we should actually figure out a pattern that works for all services, and figure out why this has only just become an issue. Most of the other services have done The services that traditionally embed a version inside the URL followed by a tenant ID or something get even deeper into parsing the URL to hack the version. dual APIs at some point over the last 2 years, and this didn't seem to trip them up too badly. What happened differently in keystone that made this an issue? And what can be learned about how we structure APIs going forward. I think the difference is this is the first API we have actually tried to deprecate and we don't have the option to hide it in an updated SC endpoint. The service catalog has hidden a lot of this pain for other services because the clients generally can use whatever endpoint the SC gives it. a) Version discovery needs to be rationalized across the services. We've talked about this at summits before, and proposals have been written. And here we are. We'll do it again in Atlanta, hopefully for the last time. b) Define a common structured endpoint and let the client assemble the components into the final URL. If the service catalog had a base URL for compute, and a list of versions, and the additional bits to be appended the client could make an intelligent choice and assemble the endpoint. It isn't like the client doesn't already have to know how the REST URLs are constructed. b-alt) Stop putting things like tenant IDs in the SC. This has the same issue as the auth URL in how to do this without instantly breaking the existing clients. Ok, much clearer now to me (though I'll still claim jetlag for some bits not sinking in). I think a really important thing to keep in mind is any solution that's implemented client side, is something that all the other OpenStack SDKs are going to have to implement as well. So an ugly hack isn't just python-keystone... and be done. It's also just hoisted doing that ugly hack on the php / go sdk teams, jclouds, deltacloud, etc. Something they may not be aware is going to break them, or their users. Do we have official openstack PHP / go SDKs? Official is a strong word, but we do have stackforge teams active on it:
Re: [openstack-dev] Ugly Hack to deal with multiple versions
Excerpts from Jesse Noller's message of 2014-02-04 11:55:57 -0800: On Feb 4, 2014, at 1:28 PM, Sean Dague s...@dague.net wrote: On 02/05/2014 01:50 AM, Jesse Noller wrote: On Feb 4, 2014, at 10:31 AM, Sean Dague s...@dague.net wrote: On 02/05/2014 01:09 AM, Dean Troyer wrote: On Tue, Feb 4, 2014 at 9:00 AM, Sean Dague s...@dague.net mailto:s...@dague.net wrote: Can you be more specific about what goes wrong here? I'm not entirely sure I understand why an old client of arbitrary age needs to be supported with new OpenStack. The contract is the API, not the client, and an old client that doesn't do version discovery is just a buggy client from what I'm concerned. Time to release a new version. Problem 1: API version discovery is not universally considered to be part of the API and therefore is not defined by most services beyond them responding to a '/' request with a 300 response and a list of versions. No two of these responses look alike except where the source was copied from an existing service. Problem 2: Identity is unique in that it is handed a deployment-defined URL to authenticate and get endpoints for all other services. Most of these auth URLs have a version hard-coded in them because the client didn't do version discovery or negotiation until recently. This is what we're talking about here, how to remove the version from this URL and not break old clients. We can't. Not without doing nasty things like detecting an old client and compensating for it server-side. So we have to work out a way for new clients to do discovery even when handed a URL that has a version in it. I've tested a couple of more generalized approaches, and the best solution I have found so far is to simply special-case the known legacy behaviour then drop in to the general discovery process. I also wonder if this is an issue with version discovery implementation. It seems like if we think this is going to be affecting multiple services before doing an odd hack for keystone, we should actually figure out a pattern that works for all services, and figure out why this has only just become an issue. Most of the other services have done The services that traditionally embed a version inside the URL followed by a tenant ID or something get even deeper into parsing the URL to hack the version. dual APIs at some point over the last 2 years, and this didn't seem to trip them up too badly. What happened differently in keystone that made this an issue? And what can be learned about how we structure APIs going forward. I think the difference is this is the first API we have actually tried to deprecate and we don't have the option to hide it in an updated SC endpoint. The service catalog has hidden a lot of this pain for other services because the clients generally can use whatever endpoint the SC gives it. a) Version discovery needs to be rationalized across the services. We've talked about this at summits before, and proposals have been written. And here we are. We'll do it again in Atlanta, hopefully for the last time. b) Define a common structured endpoint and let the client assemble the components into the final URL. If the service catalog had a base URL for compute, and a list of versions, and the additional bits to be appended the client could make an intelligent choice and assemble the endpoint. It isn't like the client doesn't already have to know how the REST URLs are constructed. b-alt) Stop putting things like tenant IDs in the SC. This has the same issue as the auth URL in how to do this without instantly breaking the existing clients. Ok, much clearer now to me (though I'll still claim jetlag for some bits not sinking in). I think a really important thing to keep in mind is any solution that's implemented client side, is something that all the other OpenStack SDKs are going to have to implement as well. So an ugly hack isn't just python-keystone... and be done. It's also just hoisted doing that ugly hack on the php / go sdk teams, jclouds, deltacloud, etc. Something they may not be aware is going to break them, or their users. Do we have official openstack PHP / go SDKs? Official is a strong word, but we do have stackforge teams active on it: * https://github.com/stackforge/openstack-sdk-php * https://github.com/stackforge/golang-client And I think we should should be mindful of their work to make OpenStack easily accessible from their language communities. Oh for sure - but I was wondering because there’s also: jclouds (java): http://jclouds.apache.org/ php-opencloud (php): https://github.com/rackspace/php-opencloud fog (ruby): http://fog.io/ libcloud (python): http://libcloud.apache.org/ gophercloud (go):
Re: [openstack-dev] Ugly Hack to deal with multiple versions
On Mon, Feb 3, 2014 at 1:50 PM, Adam Young ayo...@redhat.com wrote: HACK: In a new client, look at the URL. If it ends with /v2.0, chop it off and us the substring up to that point. Now, at this point you are probably going: That is ugly, is it really necessary? Can't we do something more correct? At this point I think we are stuck with hard-coding some legacy compatibility like this for the near future. Fortunately Identity is an easy one to handle, Compute is going to be a #$^%! as the commonly documented case has a version not at the end. I've been playing with variations on this strategy and I think it is our least bad option... Can we accept that this is necessary, and vow to never let this happen again by removing the versions from the URLs after the current set of clients are deprecated? +1 There is another hack to think about: if public_endpoint and/or admin_endpoint are not set in keystone.conf, all of the discovered urls use localhost: http://localhost:8770/v2.0/. Discovery falls over aga I don't know how common this is but I have encountered it at least once or twice. Is this the only place those config values are used? It seems like a better default could be worked out here too; is 'localhost' ever the right thing to advertise in a real-world deployment? dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Ugly Hack to deal with multiple versions
On Tue, Feb 4, 2014 at 6:20 AM, Adam Young ayo...@redhat.com wrote: We have to support old clients. Old clients expect that the URL that comes back for the service catalog has the version in it. Old clients don't do version negotiation. Thus, we need an approach to not-break old clients while we politely encourage the rest of the world to move to later APIs. I know Keystone has this problem. I've heard that some of the other services do as well. Here is what I propose. It is ugly, but it is a transition plan, and can be disabled once the old clients are deprecated: HACK: In a new client, look at the URL. If it ends with /v2.0, chop it off and us the substring up to that point. +1 to this. I agree its ugly, but I think its the least-worst solution. Nova certainly has this problem with the url including the version suffix in the service catalog. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
