Re: [openstack-dev] [all] Testtools 1.7.0 may error if you installed it before reading this email

2015-03-12 Thread Sean Dague
On 03/10/2015 08:52 PM, Joe Gordon wrote:
 
 On Tue, Mar 10, 2015 at 5:09 PM, Alan Pevec ape...@gmail.com
 mailto:ape...@gmail.com wrote:
 
  The wheel has been removed from PyPI and anyone installing testtools
  1.7.0 now will install from source which works fine.
 
 On stable/icehouse devstack fails[*] with
 pkg_resources.VersionConflict: (unittest2 0.5.1
 (/usr/lib/python2.7/dist-packages),
 Requirement.parse('unittest2=1.0.0'))
 when installing testtools 1.7.0
 
 unittest2 is not capped in stable/icehouse requirements, why is it not
 upgraded by pip?
 
 
 Tracking bug: https://bugs.launchpad.net/devstack/+bug/1430592

So, we can work around this in devstack, but it seems like there is a
more fundamental bug here that setup project isn't following dependencies.

Is this just another 'pip is drunk' issue in it not actually satisfying
requirements?

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] Testtools 1.7.0 may error if you installed it before reading this email

2015-03-11 Thread Ian Cordasco
I think part of the problem is that we’re still relying on distribution
packaged dependencies. If you look earlier in the logs for:
http://logs.openstack.org/51/146651/15/check/check-grenade-dsvm/2485c9e/log
s/grenade.sh.txt.gz#_2015-03-11_00_58_45_129 you’ll see that
“python-unittest2” is being installed and that it is installed in
dist-packages instead of site-packages. Haven’t there been multiple
problems with distribution based packages in the past? It would seem wiser
to stop relying on the system packages for installation if at all possible.

On 3/11/15, 09:43, Chris Dent chd...@redhat.com wrote:

On Wed, 11 Mar 2015, Alan Pevec wrote:

 So, we can work around this in devstack, but it seems like there is a
 more fundamental bug here that setup project isn't following
dependencies.

 Dep chain was: testtools (from
 zake=0.1-tooz=0.12,=0.3-ceilometer==2014.2.3.dev2)
 Unneeded _runtime_ dependency on testtools was removed in
 
https://github.com/yahoo/Zake/commit/215620ca51c3c883279ba62ccc860a274219
ecc1

 Is this just another 'pip is drunk' issue in it not actually satisfying
 requirements?

 Seems that pip is drunk by design, clarkb explained that pip only
 updates deps if you pass the --upgrade flag.

That's why I did this for devstack:
https://review.openstack.org/#/c/161195/

Presumably it might be useful other places?

-- 
Chris Dent tw:@anticdent freenode:cdent
https://tank.peermore.com/tanks/cdent

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] Testtools 1.7.0 may error if you installed it before reading this email

2015-03-11 Thread Ian Wienand
On 03/11/2015 08:10 AM, Robert Collins wrote:
 The wheel has been removed from PyPI and anyone installing testtools
 1.7.0 now will install from source which works fine.

I noticed the centos7 job failed with the source version.

The failing job was [1] where the back-trace looks like ~45 songs on
Python's greatest-hits album (pasted below).  The next run [2] it got
the 1.7.1 wheel and just worked.

Maybe this jumps out at someone as a known issue ...

-i

[1] 
http://logs.openstack.org/49/163249/1/check/check-tempest-dsvm-centos7/8dceac8/logs/devstacklog.txt.gz
[2] 
http://logs.openstack.org/49/163249/1/check/check-tempest-dsvm-centos7/f3b86d5/logs/devstacklog.txt.gz

---
Colecting testtools=0.9.22 (from 
fixtures=0.3.14-oslo.concurrency=1.4.1-keystone==2015.1.dev395)
Downloading 
http://pypi.IAD.openstack.org/packages/source/t/testtools/testtools-1.7.0.tar.gz
 (202kB)

 Installed 
/tmp/easy_install-mV2rSm/unittest2-1.0.0/.eggs/traceback2-1.4.0-py2.7.egg

 Installed 
/tmp/easy_install-mV2rSm/unittest2-1.0.0/.eggs/linecache2-1.0.0-py2.7.egg
 /usr/lib/python2.7/site-packages/setuptools/dist.py:291: UserWarning: The 
version specified (__main__.late_version instance at 0x34654d0) is an invalid 
version, this may not work as expected with newer versions of setuptools, pip, 
and PyPI. Please see PEP 440 for more details.
   details. % self.metadata.version
 Traceback (most recent call last):
   File string, line 20, in module
   File /tmp/pip-build-aGC1zC/testtools/setup.py, line 92, in module
 setup_requires=deps,
   File /usr/lib64/python2.7/distutils/core.py, line 112, in setup
 _setup_distribution = dist = klass(attrs)
   File /usr/lib/python2.7/site-packages/setuptools/dist.py, line 265, in 
__init__
 self.fetch_build_eggs(attrs['setup_requires'])
   File /usr/lib/python2.7/site-packages/setuptools/dist.py, line 310, in 
fetch_build_eggs
 replace_conflicting=True,
   File /usr/lib/python2.7/site-packages/pkg_resources/__init__.py, line 799, 
in resolve
 dist = best[req.key] = env.best_match(req, ws, installer)
   File /usr/lib/python2.7/site-packages/pkg_resources/__init__.py, line 
1049, in best_match
 return self.obtain(req, installer)
   File /usr/lib/python2.7/site-packages/pkg_resources/__init__.py, line 
1061, in obtain
 return installer(requirement)
   File /usr/lib/python2.7/site-packages/setuptools/dist.py, line 377, in 
fetch_build_egg
 return cmd.easy_install(req)
   File /usr/lib/python2.7/site-packages/setuptools/command/easy_install.py, 
line 620, in easy_install
 return self.install_item(spec, dist.location, tmpdir, deps)
   File /usr/lib/python2.7/site-packages/setuptools/command/easy_install.py, 
line 650, in install_item
 dists = self.install_eggs(spec, download, tmpdir)
   File /usr/lib/python2.7/site-packages/setuptools/command/easy_install.py, 
line 835, in install_eggs
 return self.build_and_install(setup_script, setup_base)
   File /usr/lib/python2.7/site-packages/setuptools/command/easy_install.py, 
line 1063, in build_and_install
 self.run_setup(setup_script, setup_base, args)
   File /usr/lib/python2.7/site-packages/setuptools/command/easy_install.py, 
line 1049, in run_setup
 run_setup(setup_script, args)
   File /usr/lib/python2.7/site-packages/setuptools/sandbox.py, line 240, in 
run_setup
 raise
   File /usr/lib64/python2.7/contextlib.py, line 35, in __exit__
 self.gen.throw(type, value, traceback)
   File /usr/lib/python2.7/site-packages/setuptools/sandbox.py, line 193, in 
setup_context
 yield
   File /usr/lib64/python2.7/contextlib.py, line 35, in __exit__
 self.gen.throw(type, value, traceback)
   File /usr/lib/python2.7/site-packages/setuptools/sandbox.py, line 164, in 
save_modules
 saved_exc.resume()
   File /usr/lib/python2.7/site-packages/setuptools/sandbox.py, line 139, in 
resume
 compat.reraise(type, exc, self._tb)
   File /usr/lib/python2.7/site-packages/setuptools/sandbox.py, line 152, in 
save_modules
 yield saved
   File /usr/lib/python2.7/site-packages/setuptools/sandbox.py, line 193, in 
setup_context
 yield
   File /usr/lib/python2.7/site-packages/setuptools/sandbox.py, line 237, in 
run_setup
 DirectorySandbox(setup_dir).run(runner)
   File /usr/lib/python2.7/site-packages/setuptools/sandbox.py, line 267, in 
run
 return func()
   File /usr/lib/python2.7/site-packages/setuptools/sandbox.py, line 236, in 
runner
 _execfile(setup_script, ns)
   File /usr/lib/python2.7/site-packages/setuptools/sandbox.py, line 46, in 
_execfile
 exec(code, globals, locals)
   File /tmp/easy_install-mV2rSm/unittest2-1.0.0/setup.py, line 87, in 
module
 'testtools.tests.matchers',
   File /usr/lib64/python2.7/distutils/core.py, line 152, in setup
 dist.run_commands()
   File /usr/lib64/python2.7/distutils/dist.py, line 953, in run_commands
 self.run_command(cmd)
   File /usr/lib64/python2.7/distutils/dist.py, line 971, in run_command
 cmd_obj.ensure_finalized()
  

Re: [openstack-dev] [all] Testtools 1.7.0 may error if you installed it before reading this email

2015-03-11 Thread Robert Collins
On 12 March 2015 at 12:37, Ian Wienand iwien...@redhat.com wrote:
 On 03/11/2015 08:10 AM, Robert Collins wrote:
 The wheel has been removed from PyPI and anyone installing testtools
 1.7.0 now will install from source which works fine.

 I noticed the centos7 job failed with the source version.

 The failing job was [1] where the back-trace looks like ~45 songs on
 Python's greatest-hits album (pasted below).  The next run [2] it got
 the 1.7.1 wheel and just worked.

 Maybe this jumps out at someone as a known issue ...

 -i

 [1] 
 http://logs.openstack.org/49/163249/1/check/check-tempest-dsvm-centos7/8dceac8/logs/devstacklog.txt.gz
...
File /tmp/easy_install-mV2rSm/unittest2-1.0.0/unittest2/case.py, line 
 16, in module
  ImportError: cannot import name range
  Complete output from command python setup.py egg_info:

Thats a missing symbol in six. Which means unittest2 needs a versioned
dep on six - please file a unittest2 bug, and if you have the time,
figure out when range was added to six.

Cheers,
Rob

-- 
Robert Collins rbtcoll...@hp.com
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] Testtools 1.7.0 may error if you installed it before reading this email

2015-03-11 Thread Alan Pevec
 So, we can work around this in devstack, but it seems like there is a
 more fundamental bug here that setup project isn't following dependencies.

Dep chain was: testtools (from
zake=0.1-tooz=0.12,=0.3-ceilometer==2014.2.3.dev2)
Unneeded _runtime_ dependency on testtools was removed in
https://github.com/yahoo/Zake/commit/215620ca51c3c883279ba62ccc860a274219ecc1

 Is this just another 'pip is drunk' issue in it not actually satisfying
 requirements?

Seems that pip is drunk by design, clarkb explained that pip only
updates deps if you pass the --upgrade flag.


Cheers,
Alan

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] Testtools 1.7.0 may error if you installed it before reading this email

2015-03-11 Thread Chris Dent

On Wed, 11 Mar 2015, Alan Pevec wrote:


So, we can work around this in devstack, but it seems like there is a
more fundamental bug here that setup project isn't following dependencies.


Dep chain was: testtools (from
zake=0.1-tooz=0.12,=0.3-ceilometer==2014.2.3.dev2)
Unneeded _runtime_ dependency on testtools was removed in
https://github.com/yahoo/Zake/commit/215620ca51c3c883279ba62ccc860a274219ecc1


Is this just another 'pip is drunk' issue in it not actually satisfying
requirements?


Seems that pip is drunk by design, clarkb explained that pip only
updates deps if you pass the --upgrade flag.


That's why I did this for devstack: https://review.openstack.org/#/c/161195/

Presumably it might be useful other places?

--
Chris Dent tw:@anticdent freenode:cdent
https://tank.peermore.com/tanks/cdent

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] Testtools 1.7.0 may error if you installed it before reading this email

2015-03-11 Thread Robert Collins
On 12 March 2015 at 13:18, Ian Wienand iwien...@redhat.com wrote:
 On 03/12/2015 10:37 AM, Ian Wienand wrote:
 File /tmp/easy_install-mV2rSm/unittest2-1.0.0/unittest2/case.py, line 
 16, in module
   ImportError: cannot import name range
   Complete output from command python setup.py egg_info:

 OK, so this suggests the version of six is wrong.

 unittest2 does have an uncapped dependency [1]

 looking at six, range was added between 1.3.0  1.4.0
...
 but it hadn't got there yet.  Filed [2].

 I think the wheel works because pip gets to it after six has been
 upgraded.

 Even then, I'm not sure that would fix it.  I've had similar issues
 before; I still don't fully understand why but pip install a b does
 *not* install a then b, apparently by design [2].  This deep in I'm
 not sure how the dependencies will order.

Thanks for digging up the release of six. I've added the versioned dep
and released unittest2 1.0.1 which has that and only that fix in it.
Hopefully that will prevent the issue.

The late version object shim is fragile though, so its likely not to
work in many places - wheels will always be a better path forward for
installing it until I get around to some proper magic to fix the
version lookup stuff.

-Rob

-- 
Robert Collins rbtcoll...@hp.com
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] Testtools 1.7.0 may error if you installed it before reading this email

2015-03-10 Thread Robert Collins
On 11 March 2015 at 13:59, Robert Collins robe...@robertcollins.net wrote:
 On 11 March 2015 at 10:27, Dolph Mathews dolph.math...@gmail.com wrote:
 Great to hear that this has been addressed, as this impacted a few tests in
 keystone.

 (but why was the fix not released as 1.7.1?)

 There will be a new release indeed later today to fix a small UI issue
 on pypy3 which affects testtools CI, but the actual bug affecting
 1.7.0 on Python2 was entirely in the build process, not in the release
 itself.

 The built wheel was corrupt (missing files), as opposed to there being
 a bug in the code itself.

And indeed now that we've worked around Sphinx breaking pypy3 and
Python 3.2, I've been able to cut a new release of testtools, which
will bring with it a new wheel, and that will workaround the bug in
capping in stable (because thats how 1.6.1 was working - wheels better
:)).

-Rob


-- 
Robert Collins rbtcoll...@hp.com
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] Testtools 1.7.0 may error if you installed it before reading this email

2015-03-10 Thread Joe Gordon
On Tue, Mar 10, 2015 at 5:09 PM, Alan Pevec ape...@gmail.com wrote:

  The wheel has been removed from PyPI and anyone installing testtools
  1.7.0 now will install from source which works fine.

 On stable/icehouse devstack fails[*] with
 pkg_resources.VersionConflict: (unittest2 0.5.1
 (/usr/lib/python2.7/dist-packages),
 Requirement.parse('unittest2=1.0.0'))
 when installing testtools 1.7.0

 unittest2 is not capped in stable/icehouse requirements, why is it not
 upgraded by pip?


Tracking bug: https://bugs.launchpad.net/devstack/+bug/1430592




 Cheers,
 Alan


 [*] e.g.
 http://logs.openstack.org/14/144714/3/check/check-tempest-dsvm-neutron/4d195b5/logs/devstacklog.txt.gz

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] Testtools 1.7.0 may error if you installed it before reading this email

2015-03-10 Thread Robert Collins
On 11 March 2015 at 10:27, Dolph Mathews dolph.math...@gmail.com wrote:
 Great to hear that this has been addressed, as this impacted a few tests in
 keystone.

 (but why was the fix not released as 1.7.1?)

There will be a new release indeed later today to fix a small UI issue
on pypy3 which affects testtools CI, but the actual bug affecting
1.7.0 on Python2 was entirely in the build process, not in the release
itself.

The built wheel was corrupt (missing files), as opposed to there being
a bug in the code itself.

-Rob

-- 
Robert Collins rbtcoll...@hp.com
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] Testtools 1.7.0 may error if you installed it before reading this email

2015-03-10 Thread Dolph Mathews
Great to hear that this has been addressed, as this impacted a few tests in
keystone.

(but why was the fix not released as 1.7.1?)

On Tue, Mar 10, 2015 at 4:10 PM, Robert Collins robe...@robertcollins.net
wrote:

 There was a broken wheel built when testtools 1.7.0 was released. The
 wheel was missing the _compat2x.py file used for 2.x only syntax in
 exception handling, for an unknown reason. (We know how to trigger it
 - build the wheel with Python 3.4).

 The wheel has been removed from PyPI and anyone installing testtools
 1.7.0 now will install from source which works fine.

 Cheers,
 Rob

 --
 Robert Collins rbtcoll...@hp.com
 Distinguished Technologist
 HP Converged Cloud

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] Testtools 1.7.0 may error if you installed it before reading this email

2015-03-10 Thread Alan Pevec
 The wheel has been removed from PyPI and anyone installing testtools
 1.7.0 now will install from source which works fine.

On stable/icehouse devstack fails[*] with
pkg_resources.VersionConflict: (unittest2 0.5.1
(/usr/lib/python2.7/dist-packages),
Requirement.parse('unittest2=1.0.0'))
when installing testtools 1.7.0

unittest2 is not capped in stable/icehouse requirements, why is it not
upgraded by pip?


Cheers,
Alan


[*] e.g. 
http://logs.openstack.org/14/144714/3/check/check-tempest-dsvm-neutron/4d195b5/logs/devstacklog.txt.gz

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev