Re: [openstack-dev] [Horizon][stable] Horizon kilo gate fails due to testrepository dependency

[Tony Breeds]

On Tue, Jan 05, 2016 at 12:26:24PM +1300, Robert Collins wrote:
> On 5 January 2016 at 12:04, Robert Collins  wrote:
> ...
> > Indeed - 
> > https://bitbucket.org/pypa/setuptools/commits/fb35fcade302fa828d34e6aff952ec2398f2c877?at=get_command_list
> > - the failing bit AFAICT is indeed new code :/.
> 
> 
> Ok, so I've paged this all in. Here's whats up, and some thoughts on fixing 
> it.
> 
> Old pbr does indeed have a bug where 'setup.py test' will error with
> that unguarded import of what isn't meant to be a dependency.
> 
> The reason this started failing is that a bugfix to setuptools - so
> that the existing pbr code that wraps commands can wrap commands only
> added by setuptools plugins like 'wheel' was merged and included in a
> setuptools release.
> 
> This causes the pbr testr command to be loaded, which fails in old pbr.
> 
> The right answer is a back port of the import guard to pbr < 1.0.0 and
> a point release - 0.11.1.
> 
> IMO that is :)

I noticed:

http://logs.openstack.org/periodic-stable/periodic-horizon-python27-kilo/45f1797/tox/py27-2.log
so I went to do the back port and found it's already been done #winning!

Sachi King did it here:
 * https://review.openstack.org/#/c/263928/ which is ready to merge BUT it 
depends on
 * https://review.openstack.org/#/c/264010 which needs additional +2 +W's

It's be great if we could get a kilo (0.11.1) release to unblock 
periodic-horizon-python27-kilo

Yours Tony.


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon][stable] Horizon kilo gate fails due to testrepository dependency

[Matthias Runge]

On Tue, Jan 05, 2016 at 12:26:24PM +1300, Robert Collins wrote:
> On 5 January 2016 at 12:04, Robert Collins  wrote:
> ...
> > Indeed - 
> > https://bitbucket.org/pypa/setuptools/commits/fb35fcade302fa828d34e6aff952ec2398f2c877?at=get_command_list
> > - the failing bit AFAICT is indeed new code :/.
> 
> 
> Ok, so I've paged this all in. Here's whats up, and some thoughts on fixing 
> it.
> 
> Old pbr does indeed have a bug where 'setup.py test' will error with
> that unguarded import of what isn't meant to be a dependency.
> 
> The reason this started failing is that a bugfix to setuptools - so
> that the existing pbr code that wraps commands can wrap commands only
> added by setuptools plugins like 'wheel' was merged and included in a
> setuptools release.
> 
> This causes the pbr testr command to be loaded, which fails in old pbr.
> 
> The right answer is a back port of the import guard to pbr < 1.0.0 and
> a point release - 0.11.1.
> 
> IMO that is :)
> 
> I see that a workaround has been committed - installing testrepository
> - but I'd hate for folk to cargo cult the idea that pbr could have a
> runtime dependency on test-only tools like that.

Rob,

thank you for looking deeper into this. Your conclusions sound logical
to me and much saner than just pulling in testrepository.

Matthias
-- 
Matthias Runge 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon][stable] Horizon kilo gate fails due to testrepository dependency

[Jordan Pittier]

On Tue, Jan 5, 2016 at 9:16 AM, Matthias Runge  wrote:

> On Tue, Jan 05, 2016 at 12:26:24PM +1300, Robert Collins wrote:
> > On 5 January 2016 at 12:04, Robert Collins 
> wrote:
> > ...
> > > Indeed -
> https://bitbucket.org/pypa/setuptools/commits/fb35fcade302fa828d34e6aff952ec2398f2c877?at=get_command_list
> > > - the failing bit AFAICT is indeed new code :/.
> >
> >
> > Ok, so I've paged this all in. Here's whats up, and some thoughts on
> fixing it.
> >
> > Old pbr does indeed have a bug where 'setup.py test' will error with
> > that unguarded import of what isn't meant to be a dependency.
> >
> > The reason this started failing is that a bugfix to setuptools - so
> > that the existing pbr code that wraps commands can wrap commands only
> > added by setuptools plugins like 'wheel' was merged and included in a
> > setuptools release.
> >
> > This causes the pbr testr command to be loaded, which fails in old pbr.
> >
> > The right answer is a back port of the import guard to pbr < 1.0.0 and
> > a point release - 0.11.1.
>

There's no 0.11 branch in PBR to which we could cherry-pick
https://git.openstack.org/cgit/openstack-dev/pbr/commit/?id=946cf80b750f3735a5d3b0c2173f4eaa7fad4a81
There's only a 0.10 branch.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon][stable] Horizon kilo gate fails due to testrepository dependency

[Itxaka Serrano Garcia]

Same issue on django_openstack_auth, kilo branch:

https://review.openstack.org/#/c/262778/


And same error too [0]:

2016-01-04 10:47:17.621 | Obtaining file:///opt/stack/new/keystone
2016-01-04 10:47:18.251 | Complete output from command python
setup.py egg_info:
2016-01-04 10:47:18.251 | ERROR:root:Error parsing
2016-01-04 10:47:18.251 | Traceback (most recent call last):
2016-01-04 10:47:18.251 |   File
"/usr/local/lib/python2.7/dist-packages/pbr/core.py", line 109, in pbr
2016-01-04 10:47:18.251 | attrs = util.cfg_to_args(path)
2016-01-04 10:47:18.251 |   File
"/usr/local/lib/python2.7/dist-packages/pbr/util.py", line 261, in
cfg_to_args
2016-01-04 10:47:18.251 | wrap_commands(kwargs)
2016-01-04 10:47:18.251 |   File
"/usr/local/lib/python2.7/dist-packages/pbr/util.py", line 482, in
wrap_commands
2016-01-04 10:47:18.251 | for cmd, _ in dist.get_command_list():
2016-01-04 10:47:18.251 |   File
"/usr/local/lib/python2.7/dist-packages/setuptools/dist.py", line 446,
in get_command_list
2016-01-04 10:47:18.251 | cmdclass = ep.resolve()
2016-01-04 10:47:18.251 |   File
"/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line
2386, in resolve
2016-01-04 10:47:18.251 | module = __import__(self.module_name,
fromlist=['__name__'], level=0)
2016-01-04 10:47:18.251 |   File
"/usr/local/lib/python2.7/dist-packages/pbr/testr_command.py", line 47,
in 
2016-01-04 10:47:18.251 | from testrepository import commands
2016-01-04 10:47:18.251 | ImportError: No module named testrepository
2016-01-04 10:47:18.251 | error in setup command: Error parsing
/opt/stack/new/keystone/setup.cfg: ImportError: No module named
testrepository




[0]
http://logs.openstack.org/78/262778/2/check/gate-tempest-dsvm-neutron-src-django_openstack_auth/01220e8/logs/devstacklog.txt.gz


On 01/04/2016 09:13 AM, Matthias Runge wrote:

Hello,

did we had a recent change in stable tests for Kilo?

Horizon tests for kilo are now failing due to a missing dependency to
testrepository. Horizon never used testrepository (until recently,
where I added testr support, but only in mitaka branch).

As a test, I added a test dependency for kilo branch, but that fails
somewhere else due to missing testrepository:

https://review.openstack.org/#/c/262296/

The error is in [1] somewhere at the bottom:

É5-12-30 09:53:27.883 | Obtaining file:///opt/stack/new/keystone
2015-12-30 09:53:28.504 | Complete output from command python
setup.py egg_info:
2015-12-30 09:53:28.504 | ERROR:root:Error parsing
2015-12-30 09:53:28.504 | Traceback (most recent call last):
2015-12-30 09:53:28.504 |   File
"/usr/local/lib/python2.7/dist-packages/pbr/core.py", line 109, in pbr
2015-12-30 09:53:28.504 | attrs = util.cfg_to_args(path)
2015-12-30 09:53:28.504 |   File
"/usr/local/lib/python2.7/dist-packages/pbr/util.py", line 261, in
cfg_to_args
2015-12-30 09:53:28.504 | wrap_commands(kwargs)
2015-12-30 09:53:28.504 |   File
"/usr/local/lib/python2.7/dist-packages/pbr/util.py", line 482, in
wrap_commands
2015-12-30 09:53:28.504 | for cmd, _ in dist.get_command_list():
2015-12-30 09:53:28.504 |   File
"/usr/local/lib/python2.7/dist-packages/setuptools/dist.py", line 446,
in get_command_list
2015-12-30 09:53:28.504 | cmdclass = ep.resolve()
2015-12-30 09:53:28.505 |   File
"/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line
2386, in resolve
2015-12-30 09:53:28.505 | module = __import__(self.module_name,
fromlist=['__name__'], level=0)
2015-12-30 09:53:28.505 |   File
"/usr/local/lib/python2.7/dist-packages/pbr/testr_command.py", line 47,
in 
2015-12-30 09:53:28.505 | from testrepository import commands
2015-12-30 09:53:28.505 | ImportError: No module named
testrepository
2015-12-30 09:53:28.505 | error in setup command: Error parsing
/opt/stack/new/keystone/setup.cfg: ImportError: No module named
testrepository

Any suggestions here?

[1]
http://logs.openstack.org/96/262296/2/check/gate-tempest-dsvm-full/e47e5c6/logs/devstacklog.txt.gz





__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon][stable] Horizon kilo gate fails due to testrepository dependency

[Ihar Hrachyshka]

Matthias Runge  wrote:


Hello,

did we had a recent change in stable tests for Kilo?

Horizon tests for kilo are now failing due to a missing dependency to
testrepository. Horizon never used testrepository (until recently,
where I added testr support, but only in mitaka branch).

As a test, I added a test dependency for kilo branch, but that fails
somewhere else due to missing testrepository:

https://review.openstack.org/#/c/262296/

The error is in [1] somewhere at the bottom:

É5-12-30 09:53:27.883 | Obtaining file:///opt/stack/new/keystone
2015-12-30 09:53:28.504 | Complete output from command python
setup.py egg_info:
2015-12-30 09:53:28.504 | ERROR:root:Error parsing
2015-12-30 09:53:28.504 | Traceback (most recent call last):
2015-12-30 09:53:28.504 |   File
"/usr/local/lib/python2.7/dist-packages/pbr/core.py", line 109, in pbr
2015-12-30 09:53:28.504 | attrs = util.cfg_to_args(path)
2015-12-30 09:53:28.504 |   File
"/usr/local/lib/python2.7/dist-packages/pbr/util.py", line 261, in
cfg_to_args
2015-12-30 09:53:28.504 | wrap_commands(kwargs)
2015-12-30 09:53:28.504 |   File
"/usr/local/lib/python2.7/dist-packages/pbr/util.py", line 482, in
wrap_commands
2015-12-30 09:53:28.504 | for cmd, _ in dist.get_command_list():
2015-12-30 09:53:28.504 |   File
"/usr/local/lib/python2.7/dist-packages/setuptools/dist.py", line 446,
in get_command_list
2015-12-30 09:53:28.504 | cmdclass = ep.resolve()
2015-12-30 09:53:28.505 |   File
"/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line
2386, in resolve
2015-12-30 09:53:28.505 | module = __import__(self.module_name,
fromlist=['__name__'], level=0)
2015-12-30 09:53:28.505 |   File
"/usr/local/lib/python2.7/dist-packages/pbr/testr_command.py", line 47,
in 
2015-12-30 09:53:28.505 | from testrepository import commands
2015-12-30 09:53:28.505 | ImportError: No module named
testrepository
2015-12-30 09:53:28.505 | error in setup command: Error parsing
/opt/stack/new/keystone/setup.cfg: ImportError: No module named
testrepository

Any suggestions here?


Seems like pbr importing testrepository, hence the dependency belongs to  
pbr, not horizon (and as a runtime dependency, not just test only).


But note that since pbr 1.1.0, they no longer depend on the package and  
fail gracefully:


https://github.com/openstack-dev/pbr/commit/946cf80b750f3735a5d3b0c2173f4eaa7fad4a81

So the proper way would be indeed to make your package to install testr for  
tests. Not sure why it worked before, but I would bet that some other  
components installed it for you (devstack? devstack-gate? job definition?  
some other component previously installed before keystone? Not that it’s  
too important.)


Ihar

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon][stable] Horizon kilo gate fails due to testrepository dependency

[Robert Collins]

On 5 January 2016 at 11:59, Robert Collins  wrote:
> This is odd indeed. pbr is not meant to have a dep on testrepository,
> and you can see in
> https://git.openstack.org/cgit/openstack-dev/pbr/tree/pbr/testr_command.py#n150
> that we only access it if it is installed and we use latest pbr
> everywhere because otherwise we can't deal with ecosystem changes to
> e.g. setuptools or pip.
>
> http://logs.openstack.org/96/262296/2/check/gate-tempest-dsvm-full/e47e5c6/logs/devstacklog.txt.gz#_2015-12-30_09_53_14_200
> shows that the latest pbr (1.8.1 which would work) is being
> downgraded. Thats when the problem is introduced. However, kilo has
> that special thing where pbr is capped because a lot of dependencies
> error due to version disagreements (not API breaks - pbr is
> compatible!), so this is expected :(.
>
> I'm not sure what has caused testrepository to not be installed in
> this scenario, but thats what I'd be looking at.
>

Further to that, I suspect setuptools may have changed -
https://pypi.python.org/pypi/setuptools - 19.2 was released
suspiciously close to the point errors were reported (25th dec).

Looking now..

-Rob


-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon][stable] Horizon kilo gate fails due to testrepository dependency

[Ihar Hrachyshka]

Matthias Runge  wrote:


On Mon, Jan 04, 2016 at 12:29:27PM +0100, Ihar Hrachyshka wrote:

Matthias Runge  wrote:

testrepository

Any suggestions here?


Seems like pbr importing testrepository, hence the dependency belongs to
pbr, not horizon (and as a runtime dependency, not just test only).

But note that since pbr 1.1.0, they no longer depend on the package and  
fail

gracefully:

https://github.com/openstack-dev/pbr/commit/946cf80b750f3735a5d3b0c2173f4eaa7fad4a81

So the proper way would be indeed to make your package to install testr  
for

tests. Not sure why it worked before, but I would bet that some other
components installed it for you (devstack? devstack-gate? job definition?
some other component previously installed before keystone? Not that it’s  
too

important.)

Ihar


Thank you.

I'm a bit confused, why it worked before e.g Dec 20th last year, but
fails after.
And it's only failing in kilo, not on liberty.

And even when adding testrepository to test-requirements, it fails,
because it's missing?

If pbr uses testrepository at run-time, it should be pulled in as
run-time requirement.


Note that it’s keystone installation that fails, not horizon, and it seems  
that it’s for grenade (I see /opt/stack/new/keystone in the logs). I would  
expect keystone gate to be broken too, so you could add the dep there and  
validate whether it fixes the thing for you.


The best alternative outcome is probably to get a new ‘kilo’ (<1.0) pbr  
release that would pull in the dependency for you.


Ihar

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon][stable] Horizon kilo gate fails due to testrepository dependency

[Matthias Runge]

On Mon, Jan 04, 2016 at 12:29:27PM +0100, Ihar Hrachyshka wrote:
> Matthias Runge  wrote:
> >testrepository
> >
> >Any suggestions here?
> 
> Seems like pbr importing testrepository, hence the dependency belongs to
> pbr, not horizon (and as a runtime dependency, not just test only).
> 
> But note that since pbr 1.1.0, they no longer depend on the package and fail
> gracefully:
> 
> https://github.com/openstack-dev/pbr/commit/946cf80b750f3735a5d3b0c2173f4eaa7fad4a81
> 
> So the proper way would be indeed to make your package to install testr for
> tests. Not sure why it worked before, but I would bet that some other
> components installed it for you (devstack? devstack-gate? job definition?
> some other component previously installed before keystone? Not that it’s too
> important.)
> 
> Ihar

Thank you.

I'm a bit confused, why it worked before e.g Dec 20th last year, but
fails after.
And it's only failing in kilo, not on liberty.

And even when adding testrepository to test-requirements, it fails,
because it's missing?

If pbr uses testrepository at run-time, it should be pulled in as
run-time requirement.
-- 
Matthias Runge 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon][stable] Horizon kilo gate fails due to testrepository dependency

[Matthias Runge]

On 04/01/16 15:41, Ihar Hrachyshka wrote:
> UPD: Turns out it breaks Liberty gate too, f.e. for Neutron. It’s
> interesting that it did not break the thing for e.g. Neutron master.
> 
> Matthias Runge  wrote:
> 
>> Hello,
>>

Horizon in Kilo *only* fails since Dec 26th. Horizon liberty seems to be
fine.

Matthias


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon][stable] Horizon kilo gate fails due to testrepository dependency

[Ihar Hrachyshka]

UPD: Turns out it breaks Liberty gate too, f.e. for Neutron. It’s  
interesting that it did not break the thing for e.g. Neutron master.


Matthias Runge  wrote:


Hello,

did we had a recent change in stable tests for Kilo?

Horizon tests for kilo are now failing due to a missing dependency to
testrepository. Horizon never used testrepository (until recently,
where I added testr support, but only in mitaka branch).

As a test, I added a test dependency for kilo branch, but that fails
somewhere else due to missing testrepository:

https://review.openstack.org/#/c/262296/

The error is in [1] somewhere at the bottom:

É5-12-30 09:53:27.883 | Obtaining file:///opt/stack/new/keystone
2015-12-30 09:53:28.504 | Complete output from command python
setup.py egg_info:
2015-12-30 09:53:28.504 | ERROR:root:Error parsing
2015-12-30 09:53:28.504 | Traceback (most recent call last):
2015-12-30 09:53:28.504 |   File
"/usr/local/lib/python2.7/dist-packages/pbr/core.py", line 109, in pbr
2015-12-30 09:53:28.504 | attrs = util.cfg_to_args(path)
2015-12-30 09:53:28.504 |   File
"/usr/local/lib/python2.7/dist-packages/pbr/util.py", line 261, in
cfg_to_args
2015-12-30 09:53:28.504 | wrap_commands(kwargs)
2015-12-30 09:53:28.504 |   File
"/usr/local/lib/python2.7/dist-packages/pbr/util.py", line 482, in
wrap_commands
2015-12-30 09:53:28.504 | for cmd, _ in dist.get_command_list():
2015-12-30 09:53:28.504 |   File
"/usr/local/lib/python2.7/dist-packages/setuptools/dist.py", line 446,
in get_command_list
2015-12-30 09:53:28.504 | cmdclass = ep.resolve()
2015-12-30 09:53:28.505 |   File
"/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line
2386, in resolve
2015-12-30 09:53:28.505 | module = __import__(self.module_name,
fromlist=['__name__'], level=0)
2015-12-30 09:53:28.505 |   File
"/usr/local/lib/python2.7/dist-packages/pbr/testr_command.py", line 47,
in 
2015-12-30 09:53:28.505 | from testrepository import commands
2015-12-30 09:53:28.505 | ImportError: No module named
testrepository
2015-12-30 09:53:28.505 | error in setup command: Error parsing
/opt/stack/new/keystone/setup.cfg: ImportError: No module named
testrepository

Any suggestions here?

[1]
http://logs.openstack.org/96/262296/2/check/gate-tempest-dsvm-full/e47e5c6/logs/devstacklog.txt.gz
--
Matthias Runge 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon][stable] Horizon kilo gate fails due to testrepository dependency

[Michał Dulko]

On 01/04/2016 03:41 PM, Ihar Hrachyshka wrote:
> UPD: Turns out it breaks Liberty gate too, f.e. for Neutron. It’s
> interesting that it did not break the thing for e.g. Neutron master.
>
> Matthias Runge  wrote:

We observe this on Cinder's stable/liberty in Grenade tests (e.g. [1])
and on stable/kilo in whole Tempest (e.g. [2]).

[1] https://review.openstack.org/#/c/262162/
[2] https://review.openstack.org/#/c/246646/

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev