Re: [openstack-dev] why do we use pip -U in tox.ini?

2014-01-05 Thread Jay Pipes
On Sun, 2014-01-05 at 10:47 -0500, Sean Dague wrote:
 While waiting some inordinate amount of time to compile lxml for a 
 tempest pep8 test, I started to wonder why is it that we are using -U on 
 all our pip install commands in tox (it's a pretty common pattern across 
 projects).
 
 It seems like requirements should be sufficient. Forcing -U to the 
 latest all the time seems like it would actually mask a ton of problems, 
 and makes sitepackages = True to be largely useless.
 
 I'm sure it went in for a reason, so I'd love to understand the reason, 
 to make sure that my instinct in disabling it in Tempest doesn't have a 
 ton of side effects.

+1 to removing it. Makes runtime of tests longer as well...

-jay


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] why do we use pip -U in tox.ini?

2014-01-05 Thread Clint Byrum
Excerpts from Sean Dague's message of 2014-01-05 07:47:22 -0800:
 While waiting some inordinate amount of time to compile lxml for a 
 tempest pep8 test, I started to wonder why is it that we are using -U on 
 all our pip install commands in tox (it's a pretty common pattern across 
 projects).
 
 It seems like requirements should be sufficient. Forcing -U to the 
 latest all the time seems like it would actually mask a ton of problems, 
 and makes sitepackages = True to be largely useless.
 
 I'm sure it went in for a reason, so I'd love to understand the reason, 
 to make sure that my instinct in disabling it in Tempest doesn't have a 
 ton of side effects.

I tend to just use tox without -r, which means re-using the existing
virtualenv. It occasionally means a failed run because I've missed a
requirements change, but that just means when I get a weird failure I
run with -r, which then builds me a new virtualenv.

Would be cool if we could just have tox auto-detect when it even needs to
rebuild the virtualenv, making this dance unnecessary, as I think that
would be preferable to what I suspect you're doing, which is running
with -r all the time?

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] why do we use pip -U in tox.ini?

2014-01-05 Thread Jeremy Stanley
On 2014-01-05 10:47:22 -0500 (-0500), Sean Dague wrote:
[...]
 It seems like requirements should be sufficient. Forcing -U to the
 latest all the time seems like it would actually mask a ton of
 problems, and makes sitepackages = True to be largely useless.
[...]

In fact, I believe it was mostly for the benefit of
sitepackages=True that we started adding it. If my recollection is
correct, faced with a requirement of foo=3 and a system install of
foo==2 pip under tox would refuse to upgrade to meet the stated
requirement and would instead just use the (non-conforming) version
from the system (system-wide installs of old Jinja on CentOS 6 seems
like where nova ran into this).
-- 
Jeremy Stanley

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] why do we use pip -U in tox.ini?

2014-01-05 Thread Sean Dague

On 01/05/2014 11:25 AM, Jeremy Stanley wrote:

On 2014-01-05 10:47:22 -0500 (-0500), Sean Dague wrote:
[...]

It seems like requirements should be sufficient. Forcing -U to the
latest all the time seems like it would actually mask a ton of
problems, and makes sitepackages = True to be largely useless.

[...]

In fact, I believe it was mostly for the benefit of
sitepackages=True that we started adding it. If my recollection is
correct, faced with a requirement of foo=3 and a system install of
foo==2 pip under tox would refuse to upgrade to meet the stated
requirement and would instead just use the (non-conforming) version
from the system (system-wide installs of old Jinja on CentOS 6 seems
like where nova ran into this).



Well that's really unfortunate.

So if I understand that correctly there is *no* way to configure tox + 
pip to:


 * use site packages if they are sufficient to requirements.txt
 * install packages into venv if they are not (i.e. only use it if you 
have to)


Because that's what I want. And it doesn't seem like a lot to ask.

-Sean

--
Sean Dague
Samsung Research America
s...@dague.net / sean.da...@samsung.com
http://dague.net

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] why do we use pip -U in tox.ini?

2014-01-05 Thread Jeremy Stanley
On 2014-01-05 12:17:41 -0500 (-0500), Sean Dague wrote:
 Well that's really unfortunate.
 
 So if I understand that correctly there is *no* way to configure tox
 + pip to:
 
  * use site packages if they are sufficient to requirements.txt
  * install packages into venv if they are not (i.e. only use it if
 you have to)
 
 Because that's what I want. And it doesn't seem like a lot to ask.

I think we ran into that issue circa pip 1.3 (though my sense of
time is really, really horrible, so not sure). It's entirely
possible this is no longer an issue and we should certainly
reevaluate.

We also want to get to a place where we no longer depend on
sitepackages=True in the tox.ini, and the MySQL-python package seems
very close now to no longer needing it (but this gets us back to
needing to solve the newer libvirt issues with Ubuntu Cloud Archive
again--hopefully someone will figure out what's wrong there soon so
we can actually start using UCA to get the bits we need for newer
PyPIfied MySQL-python to work for us).
-- 
Jeremy Stanley

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev