Re: [openstack-dev] Solving the client libs stable support dilemma
On Mon, Nov 17, 2014 at 8:51 AM, Doug Hellmann d...@doughellmann.com wrote: On Nov 17, 2014, at 6:16 AM, Sean Dague s...@dague.net wrote: On 11/16/2014 11:23 AM, Doug Hellmann wrote: On Nov 16, 2014, at 9:54 AM, Jeremy Stanley fu...@yuggoth.org wrote: On 2014-11-16 09:06:02 -0500 (-0500), Doug Hellmann wrote: So we would pin the client libraries used by the servers and installed globally, but then install the more recent client libraries in a virtualenv and test using those versions? That's what I was thinking anyway, yes. I like that. Honestly I don't, but it sucks less than the other solutions which sprang to mind. Hopefully someone will come along with a more elegant suggestion... in the meantime I don't see any obvious reasons why it wouldn't work. Really, it’s a much more accurate test of what we want. We have, as an artifact of our test configuration, to install everything on a single box. But what we’re trying to test is that a user can install the new clients and talk to an old cloud. We don’t expect deployers of old clouds to install new clients — at least we shouldn’t, and by pinning the requirements we can make that clear. Using the virtualenv for the new clients gives us separation between the “user” and “cloud” parts of the test configuration that we don’t have now. Anyway, if we’re prepared to go along with this I think it’s safe for us to stop using alpha version numbers for Oslo libraries as a matter of course. We may still opt to do it in cases where we aren’t sure of a new API or feature, but we won’t have to do it for every release. Doug I think this idea sounds good on the surface, though what a working system looks like is going to be a little interesting to make sure you are in / out of the venv. I actually think you might find it simpler to invert this. Create 1 global venv for servers, specify the venv before launching a service. Install all the clients into system level space, then running nova list doesn't require that it is put inside the venv. This should have the same results, but be less confusing for people poking at devstacks manually. That makes sense. I’m a little worried that it’s a bigger change to devstack vs. the job that’s testing the clients, but I’ll defer to you on what’s actually easier since you’re more familiar with the code. Either way, installing the servers and the clients into separate packaging spaces would allow us to pin the clients in the stable branches. Another piece is middleware, for example the auth_token middleware in the keystonemiddleware package. - Brant Doug -Sean -- Sean Dague http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Solving the client libs stable support dilemma
On 11/19/2014 08:55 PM, Brant Knudson wrote: On Mon, Nov 17, 2014 at 8:51 AM, Doug Hellmann d...@doughellmann.com mailto:d...@doughellmann.com wrote: On Nov 17, 2014, at 6:16 AM, Sean Dague s...@dague.net mailto:s...@dague.net wrote: On 11/16/2014 11:23 AM, Doug Hellmann wrote: On Nov 16, 2014, at 9:54 AM, Jeremy Stanley fu...@yuggoth.org mailto:fu...@yuggoth.org wrote: On 2014-11-16 09:06:02 -0500 (-0500), Doug Hellmann wrote: So we would pin the client libraries used by the servers and installed globally, but then install the more recent client libraries in a virtualenv and test using those versions? That's what I was thinking anyway, yes. I like that. Honestly I don't, but it sucks less than the other solutions which sprang to mind. Hopefully someone will come along with a more elegant suggestion... in the meantime I don't see any obvious reasons why it wouldn't work. Really, it’s a much more accurate test of what we want. We have, as an artifact of our test configuration, to install everything on a single box. But what we’re trying to test is that a user can install the new clients and talk to an old cloud. We don’t expect deployers of old clouds to install new clients — at least we shouldn’t, and by pinning the requirements we can make that clear. Using the virtualenv for the new clients gives us separation between the “user” and “cloud” parts of the test configuration that we don’t have now. Anyway, if we’re prepared to go along with this I think it’s safe for us to stop using alpha version numbers for Oslo libraries as a matter of course. We may still opt to do it in cases where we aren’t sure of a new API or feature, but we won’t have to do it for every release. Doug I think this idea sounds good on the surface, though what a working system looks like is going to be a little interesting to make sure you are in / out of the venv. I actually think you might find it simpler to invert this. Create 1 global venv for servers, specify the venv before launching a service. Install all the clients into system level space, then running nova list doesn't require that it is put inside the venv. This should have the same results, but be less confusing for people poking at devstacks manually. That makes sense. I’m a little worried that it’s a bigger change to devstack vs. the job that’s testing the clients, but I’ll defer to you on what’s actually easier since you’re more familiar with the code. Either way, installing the servers and the clients into separate packaging spaces would allow us to pin the clients in the stable branches. Another piece is middleware, for example the auth_token middleware in the keystonemiddleware package. Right, so that would get dragged into the system libs for the CLI pieces at whatever the clients want, and would get additionally dragged into the global venv for the server based on the server's needs. I think the only question would be if LIBS_FROM_GIT did both system and global venv in this case. Perhaps we'd need another SYSLIBS_FROM_GIT or something to specify the differences. Anyway, I think I have most of the ideas in my head, but will write all this out in a spec before implementing, because there are probably enough edge cases that we'll want plenty of eyes looking at it. -Sean -- Sean Dague http://dague.net signature.asc Description: OpenPGP digital signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Solving the client libs stable support dilemma
On 11/16/2014 11:23 AM, Doug Hellmann wrote: On Nov 16, 2014, at 9:54 AM, Jeremy Stanley fu...@yuggoth.org wrote: On 2014-11-16 09:06:02 -0500 (-0500), Doug Hellmann wrote: So we would pin the client libraries used by the servers and installed globally, but then install the more recent client libraries in a virtualenv and test using those versions? That's what I was thinking anyway, yes. I like that. Honestly I don't, but it sucks less than the other solutions which sprang to mind. Hopefully someone will come along with a more elegant suggestion... in the meantime I don't see any obvious reasons why it wouldn't work. Really, it’s a much more accurate test of what we want. We have, as an artifact of our test configuration, to install everything on a single box. But what we’re trying to test is that a user can install the new clients and talk to an old cloud. We don’t expect deployers of old clouds to install new clients — at least we shouldn’t, and by pinning the requirements we can make that clear. Using the virtualenv for the new clients gives us separation between the “user” and “cloud” parts of the test configuration that we don’t have now. Anyway, if we’re prepared to go along with this I think it’s safe for us to stop using alpha version numbers for Oslo libraries as a matter of course. We may still opt to do it in cases where we aren’t sure of a new API or feature, but we won’t have to do it for every release. Doug I think this idea sounds good on the surface, though what a working system looks like is going to be a little interesting to make sure you are in / out of the venv. I actually think you might find it simpler to invert this. Create 1 global venv for servers, specify the venv before launching a service. Install all the clients into system level space, then running nova list doesn't require that it is put inside the venv. This should have the same results, but be less confusing for people poking at devstacks manually. -Sean -- Sean Dague http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Solving the client libs stable support dilemma
On Nov 17, 2014, at 6:16 AM, Sean Dague s...@dague.net wrote: On 11/16/2014 11:23 AM, Doug Hellmann wrote: On Nov 16, 2014, at 9:54 AM, Jeremy Stanley fu...@yuggoth.org wrote: On 2014-11-16 09:06:02 -0500 (-0500), Doug Hellmann wrote: So we would pin the client libraries used by the servers and installed globally, but then install the more recent client libraries in a virtualenv and test using those versions? That's what I was thinking anyway, yes. I like that. Honestly I don't, but it sucks less than the other solutions which sprang to mind. Hopefully someone will come along with a more elegant suggestion... in the meantime I don't see any obvious reasons why it wouldn't work. Really, it’s a much more accurate test of what we want. We have, as an artifact of our test configuration, to install everything on a single box. But what we’re trying to test is that a user can install the new clients and talk to an old cloud. We don’t expect deployers of old clouds to install new clients — at least we shouldn’t, and by pinning the requirements we can make that clear. Using the virtualenv for the new clients gives us separation between the “user” and “cloud” parts of the test configuration that we don’t have now. Anyway, if we’re prepared to go along with this I think it’s safe for us to stop using alpha version numbers for Oslo libraries as a matter of course. We may still opt to do it in cases where we aren’t sure of a new API or feature, but we won’t have to do it for every release. Doug I think this idea sounds good on the surface, though what a working system looks like is going to be a little interesting to make sure you are in / out of the venv. I actually think you might find it simpler to invert this. Create 1 global venv for servers, specify the venv before launching a service. Install all the clients into system level space, then running nova list doesn't require that it is put inside the venv. This should have the same results, but be less confusing for people poking at devstacks manually. That makes sense. I’m a little worried that it’s a bigger change to devstack vs. the job that’s testing the clients, but I’ll defer to you on what’s actually easier since you’re more familiar with the code. Either way, installing the servers and the clients into separate packaging spaces would allow us to pin the clients in the stable branches. Doug -Sean -- Sean Dague http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Solving the client libs stable support dilemma
On Nov 15, 2014, at 5:50 PM, Jeremy Stanley fu...@yuggoth.org wrote: During the Kilo design summit session on Oslo versioning changes, we mostly resolved to switch to semver without alphas and pin stable branch libs to less than the next nontrivial version number (allowing for backports to fix bugs with subsequent point releases on a stable branch when needed). This however raises a question for other library dependencies, including dependencies on client libraries. Specifically, if we're going to limit the Oslo libraries consumed by stable branches of integrated release servers (bear in mind we already do this in one way by declaring they can't suddenly start requiring new dependencies), then how does that play out when we allow client libraries which are also dependencies of our stable release servers to depend on later versions of (or for that matter additional) Oslo libs? It seems likely we need to apply a similar versioning and branching model across our client libraries to address this, but this raises the end-user/app-dev interoperability concern we discuss from time to time: with a recent enough client library, you should be able to interact with multiple OpenStack deployments made from different integrated releases (or non-releases, continuous deployment, et al). The challenge, then, is to test interactions using our client libraries under development against older stable servers while installing those stable servers using different, older versions of the same client libraries. Perhaps something similar to how devstack+tempest jobs install devstack/server requirements globally on the system but then tempest requirements are installed separately into a virtualenv? So we would pin the client libraries used by the servers and installed globally, but then install the more recent client libraries in a virtualenv and test using those versions? I like that. Doug -- Jeremy Stanley ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Solving the client libs stable support dilemma
On 2014-11-16 09:06:02 -0500 (-0500), Doug Hellmann wrote: So we would pin the client libraries used by the servers and installed globally, but then install the more recent client libraries in a virtualenv and test using those versions? That's what I was thinking anyway, yes. I like that. Honestly I don't, but it sucks less than the other solutions which sprang to mind. Hopefully someone will come along with a more elegant suggestion... in the meantime I don't see any obvious reasons why it wouldn't work. -- Jeremy Stanley ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Solving the client libs stable support dilemma
On Nov 16, 2014, at 9:54 AM, Jeremy Stanley fu...@yuggoth.org wrote: On 2014-11-16 09:06:02 -0500 (-0500), Doug Hellmann wrote: So we would pin the client libraries used by the servers and installed globally, but then install the more recent client libraries in a virtualenv and test using those versions? That's what I was thinking anyway, yes. I like that. Honestly I don't, but it sucks less than the other solutions which sprang to mind. Hopefully someone will come along with a more elegant suggestion... in the meantime I don't see any obvious reasons why it wouldn't work. Really, it’s a much more accurate test of what we want. We have, as an artifact of our test configuration, to install everything on a single box. But what we’re trying to test is that a user can install the new clients and talk to an old cloud. We don’t expect deployers of old clouds to install new clients — at least we shouldn’t, and by pinning the requirements we can make that clear. Using the virtualenv for the new clients gives us separation between the “user” and “cloud” parts of the test configuration that we don’t have now. Anyway, if we’re prepared to go along with this I think it’s safe for us to stop using alpha version numbers for Oslo libraries as a matter of course. We may still opt to do it in cases where we aren’t sure of a new API or feature, but we won’t have to do it for every release. Doug -- Jeremy Stanley ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] Solving the client libs stable support dilemma
During the Kilo design summit session on Oslo versioning changes, we mostly resolved to switch to semver without alphas and pin stable branch libs to less than the next nontrivial version number (allowing for backports to fix bugs with subsequent point releases on a stable branch when needed). This however raises a question for other library dependencies, including dependencies on client libraries. Specifically, if we're going to limit the Oslo libraries consumed by stable branches of integrated release servers (bear in mind we already do this in one way by declaring they can't suddenly start requiring new dependencies), then how does that play out when we allow client libraries which are also dependencies of our stable release servers to depend on later versions of (or for that matter additional) Oslo libs? It seems likely we need to apply a similar versioning and branching model across our client libraries to address this, but this raises the end-user/app-dev interoperability concern we discuss from time to time: with a recent enough client library, you should be able to interact with multiple OpenStack deployments made from different integrated releases (or non-releases, continuous deployment, et al). The challenge, then, is to test interactions using our client libraries under development against older stable servers while installing those stable servers using different, older versions of the same client libraries. Perhaps something similar to how devstack+tempest jobs install devstack/server requirements globally on the system but then tempest requirements are installed separately into a virtualenv? -- Jeremy Stanley ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev