Re: [openstack-dev] Solving the client libs stable support dilemma

2014-11-19 Thread Brant Knudson
On Mon, Nov 17, 2014 at 8:51 AM, Doug Hellmann d...@doughellmann.com
wrote:


 On Nov 17, 2014, at 6:16 AM, Sean Dague s...@dague.net wrote:

  On 11/16/2014 11:23 AM, Doug Hellmann wrote:
 
  On Nov 16, 2014, at 9:54 AM, Jeremy Stanley fu...@yuggoth.org wrote:
 
  On 2014-11-16 09:06:02 -0500 (-0500), Doug Hellmann wrote:
  So we would pin the client libraries used by the servers and
  installed globally, but then install the more recent client
  libraries in a virtualenv and test using those versions?
 
  That's what I was thinking anyway, yes.
 
  I like that.
 
  Honestly I don't, but it sucks less than the other solutions which
  sprang to mind. Hopefully someone will come along with a more
  elegant suggestion... in the meantime I don't see any obvious
  reasons why it wouldn't work.
 
  Really, it’s a much more accurate test of what we want. We have, as an
 artifact of our test configuration, to install everything on a single box.
 But what we’re trying to test is that a user can install the new clients
 and talk to an old cloud. We don’t expect deployers of old clouds to
 install new clients — at least we shouldn’t, and by pinning the
 requirements we can make that clear. Using the virtualenv for the new
 clients gives us separation between the “user” and “cloud” parts of the
 test configuration that we don’t have now.
 
  Anyway, if we’re prepared to go along with this I think it’s safe for
 us to stop using alpha version numbers for Oslo libraries as a matter of
 course. We may still opt to do it in cases where we aren’t sure of a new
 API or feature, but we won’t have to do it for every release.
 
  Doug
 
  I think this idea sounds good on the surface, though what a working
  system looks like is going to be a little interesting to make sure you
  are in / out of the venv.
 
  I actually think you might find it simpler to invert this.
 
  Create 1 global venv for servers, specify the venv before launching a
  service.
 
  Install all the clients into system level space, then running nova list
  doesn't require that it is put inside the venv.
 
  This should have the same results, but be less confusing for people
  poking at devstacks manually.

 That makes sense. I’m a little worried that it’s a bigger change to
 devstack vs. the job that’s testing the clients, but I’ll defer to you on
 what’s actually easier since you’re more familiar with the code. Either
 way, installing the servers and the clients into separate packaging spaces
 would allow us to pin the clients in the stable branches.


Another piece is middleware, for example the auth_token middleware in the
keystonemiddleware package.

 - Brant



 Doug

 
-Sean
 
  --
  Sean Dague
  http://dague.net
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Solving the client libs stable support dilemma

2014-11-19 Thread Sean Dague
On 11/19/2014 08:55 PM, Brant Knudson wrote:


 On Mon, Nov 17, 2014 at 8:51 AM, Doug Hellmann d...@doughellmann.com
 mailto:d...@doughellmann.com wrote:


 On Nov 17, 2014, at 6:16 AM, Sean Dague s...@dague.net
 mailto:s...@dague.net wrote:

  On 11/16/2014 11:23 AM, Doug Hellmann wrote:
 
  On Nov 16, 2014, at 9:54 AM, Jeremy Stanley fu...@yuggoth.org
 mailto:fu...@yuggoth.org wrote:
 
  On 2014-11-16 09:06:02 -0500 (-0500), Doug Hellmann wrote:
  So we would pin the client libraries used by the servers and
  installed globally, but then install the more recent client
  libraries in a virtualenv and test using those versions?
 
  That's what I was thinking anyway, yes.
 
  I like that.
 
  Honestly I don't, but it sucks less than the other solutions which
  sprang to mind. Hopefully someone will come along with a more
  elegant suggestion... in the meantime I don't see any obvious
  reasons why it wouldn't work.
 
  Really, it’s a much more accurate test of what we want. We
 have, as an artifact of our test configuration, to install
 everything on a single box. But what we’re trying to test is that
 a user can install the new clients and talk to an old cloud. We
 don’t expect deployers of old clouds to install new clients — at
 least we shouldn’t, and by pinning the requirements we can make
 that clear. Using the virtualenv for the new clients gives us
 separation between the “user” and “cloud” parts of the test
 configuration that we don’t have now.
 
  Anyway, if we’re prepared to go along with this I think it’s
 safe for us to stop using alpha version numbers for Oslo libraries
 as a matter of course. We may still opt to do it in cases where we
 aren’t sure of a new API or feature, but we won’t have to do it
 for every release.
 
  Doug
 
  I think this idea sounds good on the surface, though what a working
  system looks like is going to be a little interesting to make
 sure you
  are in / out of the venv.
 
  I actually think you might find it simpler to invert this.
 
  Create 1 global venv for servers, specify the venv before
 launching a
  service.
 
  Install all the clients into system level space, then running
 nova list
  doesn't require that it is put inside the venv.
 
  This should have the same results, but be less confusing for people
  poking at devstacks manually.

 That makes sense. I’m a little worried that it’s a bigger change
 to devstack vs. the job that’s testing the clients, but I’ll defer
 to you on what’s actually easier since you’re more familiar with
 the code. Either way, installing the servers and the clients into
 separate packaging spaces would allow us to pin the clients in the
 stable branches.


 Another piece is middleware, for example the auth_token middleware in
 the keystonemiddleware package.

Right, so that would get dragged into the system libs for the CLI pieces
at whatever the clients want, and would get additionally dragged into
the global venv for the server based on the server's needs. I think the
only question would be if LIBS_FROM_GIT did both system and global venv
in this case. Perhaps we'd need another SYSLIBS_FROM_GIT or something to
specify the differences.

Anyway, I think I have most of the ideas in my head, but will write all
this out in a spec before implementing, because there are probably
enough edge cases that we'll want plenty of eyes looking at it.

-Sean

-- 
Sean Dague
http://dague.net



signature.asc
Description: OpenPGP digital signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Solving the client libs stable support dilemma

2014-11-17 Thread Sean Dague
On 11/16/2014 11:23 AM, Doug Hellmann wrote:
 
 On Nov 16, 2014, at 9:54 AM, Jeremy Stanley fu...@yuggoth.org wrote:
 
 On 2014-11-16 09:06:02 -0500 (-0500), Doug Hellmann wrote:
 So we would pin the client libraries used by the servers and
 installed globally, but then install the more recent client
 libraries in a virtualenv and test using those versions?

 That's what I was thinking anyway, yes.

 I like that.

 Honestly I don't, but it sucks less than the other solutions which
 sprang to mind. Hopefully someone will come along with a more
 elegant suggestion... in the meantime I don't see any obvious
 reasons why it wouldn't work.
 
 Really, it’s a much more accurate test of what we want. We have, as an 
 artifact of our test configuration, to install everything on a single box. 
 But what we’re trying to test is that a user can install the new clients and 
 talk to an old cloud. We don’t expect deployers of old clouds to install new 
 clients — at least we shouldn’t, and by pinning the requirements we can make 
 that clear. Using the virtualenv for the new clients gives us separation 
 between the “user” and “cloud” parts of the test configuration that we don’t 
 have now.
 
 Anyway, if we’re prepared to go along with this I think it’s safe for us to 
 stop using alpha version numbers for Oslo libraries as a matter of course. We 
 may still opt to do it in cases where we aren’t sure of a new API or feature, 
 but we won’t have to do it for every release.
 
 Doug

I think this idea sounds good on the surface, though what a working
system looks like is going to be a little interesting to make sure you
are in / out of the venv.

I actually think you might find it simpler to invert this.

Create 1 global venv for servers, specify the venv before launching a
service.

Install all the clients into system level space, then running nova list
doesn't require that it is put inside the venv.

This should have the same results, but be less confusing for people
poking at devstacks manually.

-Sean

-- 
Sean Dague
http://dague.net

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Solving the client libs stable support dilemma

2014-11-17 Thread Doug Hellmann

On Nov 17, 2014, at 6:16 AM, Sean Dague s...@dague.net wrote:

 On 11/16/2014 11:23 AM, Doug Hellmann wrote:
 
 On Nov 16, 2014, at 9:54 AM, Jeremy Stanley fu...@yuggoth.org wrote:
 
 On 2014-11-16 09:06:02 -0500 (-0500), Doug Hellmann wrote:
 So we would pin the client libraries used by the servers and
 installed globally, but then install the more recent client
 libraries in a virtualenv and test using those versions?
 
 That's what I was thinking anyway, yes.
 
 I like that.
 
 Honestly I don't, but it sucks less than the other solutions which
 sprang to mind. Hopefully someone will come along with a more
 elegant suggestion... in the meantime I don't see any obvious
 reasons why it wouldn't work.
 
 Really, it’s a much more accurate test of what we want. We have, as an 
 artifact of our test configuration, to install everything on a single box. 
 But what we’re trying to test is that a user can install the new clients and 
 talk to an old cloud. We don’t expect deployers of old clouds to install new 
 clients — at least we shouldn’t, and by pinning the requirements we can make 
 that clear. Using the virtualenv for the new clients gives us separation 
 between the “user” and “cloud” parts of the test configuration that we don’t 
 have now.
 
 Anyway, if we’re prepared to go along with this I think it’s safe for us to 
 stop using alpha version numbers for Oslo libraries as a matter of course. 
 We may still opt to do it in cases where we aren’t sure of a new API or 
 feature, but we won’t have to do it for every release.
 
 Doug
 
 I think this idea sounds good on the surface, though what a working
 system looks like is going to be a little interesting to make sure you
 are in / out of the venv.
 
 I actually think you might find it simpler to invert this.
 
 Create 1 global venv for servers, specify the venv before launching a
 service.
 
 Install all the clients into system level space, then running nova list
 doesn't require that it is put inside the venv.
 
 This should have the same results, but be less confusing for people
 poking at devstacks manually.

That makes sense. I’m a little worried that it’s a bigger change to devstack 
vs. the job that’s testing the clients, but I’ll defer to you on what’s 
actually easier since you’re more familiar with the code. Either way, 
installing the servers and the clients into separate packaging spaces would 
allow us to pin the clients in the stable branches.

Doug

 
   -Sean
 
 -- 
 Sean Dague
 http://dague.net
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Solving the client libs stable support dilemma

2014-11-16 Thread Doug Hellmann

On Nov 15, 2014, at 5:50 PM, Jeremy Stanley fu...@yuggoth.org wrote:

 During the Kilo design summit session on Oslo versioning changes, we
 mostly resolved to switch to semver without alphas and pin stable
 branch libs to less than the next nontrivial version number
 (allowing for backports to fix bugs with subsequent point releases
 on a stable branch when needed). This however raises a question for
 other library dependencies, including dependencies on client
 libraries.
 
 Specifically, if we're going to limit the Oslo libraries consumed by
 stable branches of integrated release servers (bear in mind we
 already do this in one way by declaring they can't suddenly start
 requiring new dependencies), then how does that play out when we
 allow client libraries which are also dependencies of our stable
 release servers to depend on later versions of (or for that matter
 additional) Oslo libs?
 
 It seems likely we need to apply a similar versioning and branching
 model across our client libraries to address this, but this raises
 the end-user/app-dev interoperability concern we discuss from time
 to time: with a recent enough client library, you should be able to
 interact with multiple OpenStack deployments made from different
 integrated releases (or non-releases, continuous deployment, et al).
 
 The challenge, then, is to test interactions using our client
 libraries under development against older stable servers while
 installing those stable servers using different, older versions of
 the same client libraries. Perhaps something similar to how
 devstack+tempest jobs install devstack/server requirements globally
 on the system but then tempest requirements are installed separately
 into a virtualenv?

So we would pin the client libraries used by the servers and installed 
globally, but then install the more recent client libraries in a virtualenv and 
test using those versions? I like that.

Doug

 -- 
 Jeremy Stanley
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Solving the client libs stable support dilemma

2014-11-16 Thread Jeremy Stanley
On 2014-11-16 09:06:02 -0500 (-0500), Doug Hellmann wrote:
 So we would pin the client libraries used by the servers and
 installed globally, but then install the more recent client
 libraries in a virtualenv and test using those versions?

That's what I was thinking anyway, yes.

 I like that.

Honestly I don't, but it sucks less than the other solutions which
sprang to mind. Hopefully someone will come along with a more
elegant suggestion... in the meantime I don't see any obvious
reasons why it wouldn't work.
-- 
Jeremy Stanley

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Solving the client libs stable support dilemma

2014-11-16 Thread Doug Hellmann

On Nov 16, 2014, at 9:54 AM, Jeremy Stanley fu...@yuggoth.org wrote:

 On 2014-11-16 09:06:02 -0500 (-0500), Doug Hellmann wrote:
 So we would pin the client libraries used by the servers and
 installed globally, but then install the more recent client
 libraries in a virtualenv and test using those versions?
 
 That's what I was thinking anyway, yes.
 
 I like that.
 
 Honestly I don't, but it sucks less than the other solutions which
 sprang to mind. Hopefully someone will come along with a more
 elegant suggestion... in the meantime I don't see any obvious
 reasons why it wouldn't work.

Really, it’s a much more accurate test of what we want. We have, as an artifact 
of our test configuration, to install everything on a single box. But what 
we’re trying to test is that a user can install the new clients and talk to an 
old cloud. We don’t expect deployers of old clouds to install new clients — at 
least we shouldn’t, and by pinning the requirements we can make that clear. 
Using the virtualenv for the new clients gives us separation between the “user” 
and “cloud” parts of the test configuration that we don’t have now.

Anyway, if we’re prepared to go along with this I think it’s safe for us to 
stop using alpha version numbers for Oslo libraries as a matter of course. We 
may still opt to do it in cases where we aren’t sure of a new API or feature, 
but we won’t have to do it for every release.

Doug

 -- 
 Jeremy Stanley
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev