Re: [openstack-dev] [oslo.messaging][infra] Adding support for AMQP 1.0 Messaging to Oslo.Messaging and infra/config
On Wed, 30 Jul 2014 15:04:41 -0700, Matt Riedemann wrote: On 7/30/2014 11:59 AM, Ken Giusti wrote: On Wed, 30 Jul 2014 14:25:46 +0100, Daniel P. Berrange wrote: On Wed, Jul 30, 2014 at 08:54:01AM -0400, Ken Giusti wrote: Greetings, snip At this point, there are no integration tests that exercise the driver. However, the new unit tests include a test 'broker', which allow the unit tests to fully exercise the new driver, right down to the network. That's a bonus of AMQP 1.0 - it can support peer-2-peer messaging. So its the new unit tests that have the 'hard' requirement of the proton libraries.And mocking-out the proton libraries really doesn't allow us to do any meaningful tests of the driver. snip If your unit tests are dependent on a specific dependent library aren't they no longer unit tests but integration tests anyway? Good point - yes, they are certainly more than just unit tests. I'd consider them more functional tests than integration tests, tho: they only test from the new driver API down to the wire (and back up again via the fake loopback broker). For integration testing, I'd want to put a real broker in there, and run real subprojects over oslo.messaging using the new driver (neutron, etc). I'd really like to avoid the classic unit test approach of mocking out the underlying messaging client api if possible. Even though that would avoid the dependency, I think it could result in the same issues we've had with the existing impl_qpid tests passing in mock, but failing when run against qpidd. Just wondering, not trying to put up road-blocks because I'd like to see how this code performs but haven't had time yet to play with it. np, a good question, thanks! When you do get a chance to kick the tires, feel free to ping me with any questions/issues you have. Thanks! -- Thanks, Matt Riedemann ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo.messaging][infra] Adding support for AMQP 1.0 Messaging to Oslo.Messaging and infra/config
It seems like this is precisely what the functional test setup was designed to handle. Is there a reason you don't want to run them as functional tests instead of unit tests? As functional tests, nobody would need new prereqs just to make it through unit tests and anyone that wants to do the full tests can install them and run 'tox functional'. This is how neutron is starting to test the behavior of OVS and it seems to work well. On Aug 1, 2014 6:01 AM, Ken Giusti kgiu...@gmail.com wrote: On Wed, 30 Jul 2014 15:04:41 -0700, Matt Riedemann wrote: On 7/30/2014 11:59 AM, Ken Giusti wrote: On Wed, 30 Jul 2014 14:25:46 +0100, Daniel P. Berrange wrote: On Wed, Jul 30, 2014 at 08:54:01AM -0400, Ken Giusti wrote: Greetings, snip At this point, there are no integration tests that exercise the driver. However, the new unit tests include a test 'broker', which allow the unit tests to fully exercise the new driver, right down to the network. That's a bonus of AMQP 1.0 - it can support peer-2-peer messaging. So its the new unit tests that have the 'hard' requirement of the proton libraries.And mocking-out the proton libraries really doesn't allow us to do any meaningful tests of the driver. snip If your unit tests are dependent on a specific dependent library aren't they no longer unit tests but integration tests anyway? Good point - yes, they are certainly more than just unit tests. I'd consider them more functional tests than integration tests, tho: they only test from the new driver API down to the wire (and back up again via the fake loopback broker). For integration testing, I'd want to put a real broker in there, and run real subprojects over oslo.messaging using the new driver (neutron, etc). I'd really like to avoid the classic unit test approach of mocking out the underlying messaging client api if possible. Even though that would avoid the dependency, I think it could result in the same issues we've had with the existing impl_qpid tests passing in mock, but failing when run against qpidd. Just wondering, not trying to put up road-blocks because I'd like to see how this code performs but haven't had time yet to play with it. np, a good question, thanks! When you do get a chance to kick the tires, feel free to ping me with any questions/issues you have. Thanks! -- Thanks, Matt Riedemann ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo.messaging][infra] Adding support for AMQP 1.0 Messaging to Oslo.Messaging and infra/config
On Wed, 30 Jul 2014 22:14:51 +, Jeremy Stanley wrote: On 2014-07-30 14:59:09 -0400 (-0400), Ken Giusti wrote: Thanks Daniel. It was my understanding - which may be wrong - that having devstack install the 'out of band' packages would only help in the case of the devstack-based integration tests, not in the case of CI running the unit tests. Is that indeed the case? [...] I'm open to any thoughts on how best to solve this, thanks. Since they're in EPEL and we run Python 2.6 unit tests today on CentOS 6 servers, if the proton libraries install successfully there perhaps we could opportunistically exercise it only under Python 2.6 for now? Not ideal, but it does get it enforced upstream with minimal fuss. I'd really rather not start accumulating arbitrary PPA sources on our job workers... I know we've done it for a handful of multi-project efforts where we needed select backports from non-LTS releases, but we've so far limited that to only PPAs maintained by the same package teams as the mainline distro packages themselves. Yeah, it's becoming pretty clear that adding this PPA to infra is not The Right Thing To Do. How does this sound as an alternative: 1) _for_ _now_, make the dependent unit tests optional for oslo.messaging. Specifically, by default tox will not run them, but I'll add a new testenv that adds a requirement for the dependent packages and runs all the unit tests (default tests + new amqp1.0 tests). Eg, do 'tox -e amqp1' to pull in the python packages that require the libraries, and run all unit tests. This allows those developers that have installed the proton libraries to run the tests, and avoid making life hard for those devs who don't have the libraries installed. 2) Propose a new optional configuration flag in devstack that enables the AMQP 1.0 messaging protocol (default off). Something like $RPC_MESSAGING_PROTOCOL == AMQP1. When this is set in the devstack config, rpc_backend will install the AMQP 1.0 libraries, adding the Qpid PPA in the case of ubuntu for now. 3) Create a non-voting oslo.messaging gate test [0] that merely runs the 'tox -e amqp1' tests. Of course, additional integration tests are a Good Thing, but at the very least we should start with this. This would give us a heads up should new patches break the amqp 1.0 driver. This test could eventually become gating once the driver matures and the packages find their way into all the proper repos. 4) Profit (couldn't resist :) Opinions? [0] I honestly have no idea how to do this, or if it's even feasible btw - I've never written a gating test before. I'd appreciate any pointers to get me started, thanks! Longer term, I'd suggest getting it sponsored into Debian unstable/testing ASAP, interesting the Ubuntu OpenStack team in importing it into the development tree for the next Ubuntu release, and then incorporating it into the Trusty Ubuntu Cloud Archive. We're not using UCA yet, but on Trusty we probably should consider adding it sooner rather than later since when we tried to tack on the Precise UCA in the last couple cycles we had too many headaches from trying to jump ahead substantially on fundamental bits like libvirt. Breaking sooner and more often means those incremental issues are easier to identify and address, usually. Ah - I didn't know that, thanks! I know one of the Qpid devs is currently engaged in getting these packages into Debian. I'll reach out to him and see if he can work on getting it into UCA next. Thanks again - very valuable info! -- Jeremy Stanley -- Ken Giusti (kgiu...@gmail.com) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo.messaging][infra] Adding support for AMQP 1.0 Messaging to Oslo.Messaging and infra/config
On 08/01/2014 03:29 PM, Ken Giusti wrote: On Wed, 30 Jul 2014 22:14:51 +, Jeremy Stanley wrote: On 2014-07-30 14:59:09 -0400 (-0400), Ken Giusti wrote: Thanks Daniel. It was my understanding - which may be wrong - that having devstack install the 'out of band' packages would only help in the case of the devstack-based integration tests, not in the case of CI running the unit tests. Is that indeed the case? [...] I'm open to any thoughts on how best to solve this, thanks. Since they're in EPEL and we run Python 2.6 unit tests today on CentOS 6 servers, if the proton libraries install successfully there perhaps we could opportunistically exercise it only under Python 2.6 for now? Not ideal, but it does get it enforced upstream with minimal fuss. I'd really rather not start accumulating arbitrary PPA sources on our job workers... I know we've done it for a handful of multi-project efforts where we needed select backports from non-LTS releases, but we've so far limited that to only PPAs maintained by the same package teams as the mainline distro packages themselves. Yeah, it's becoming pretty clear that adding this PPA to infra is not The Right Thing To Do. How does this sound as an alternative: 1) _for_ _now_, make the dependent unit tests optional for oslo.messaging. Specifically, by default tox will not run them, but I'll add a new testenv that adds a requirement for the dependent packages and runs all the unit tests (default tests + new amqp1.0 tests). Eg, do 'tox -e amqp1' to pull in the python packages that require the libraries, and run all unit tests. This allows those developers that have installed the proton libraries to run the tests, and avoid making life hard for those devs who don't have the libraries installed. 2) Propose a new optional configuration flag in devstack that enables the AMQP 1.0 messaging protocol (default off). Something like $RPC_MESSAGING_PROTOCOL == AMQP1. When this is set in the devstack config, rpc_backend will install the AMQP 1.0 libraries, adding the Qpid PPA in the case of ubuntu for now. 3) Create a non-voting oslo.messaging gate test [0] that merely runs the 'tox -e amqp1' tests. Of course, additional integration tests are a Good Thing, but at the very least we should start with this. This would give us a heads up should new patches break the amqp 1.0 driver. This test could eventually become gating once the driver matures and the packages find their way into all the proper repos. 4) Profit (couldn't resist :) +1 As long as we get the tests running, I'm happy. This sounds like something more acceptable for the infrastructure - at least based on the discussions on this thread. The plan sounds good to me. I think it's also possible to run amqp10 gate *just* for the changes happening in the *amqp* package but it's probably worth it to just make it non-voting and run it for every patch, as you mentioned. Flavio -- @flaper87 Flavio Percoco ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo.messaging][infra] Adding support for AMQP 1.0 Messaging to Oslo.Messaging and infra/config
On Wed, Jul 30, 2014 at 08:54:01AM -0400, Ken Giusti wrote: Greetings, Apologies for the cross-post: this should be of interest to both infra and olso.messaging developers. The blueprint [0] that adds support for version 1.0 of the AMQP messaging protocol is blocked due to CI test failures [1]. These failures are due to a new package dependency this blueprint adds to oslo.messaging. The AMQP 1.0 functionality is provided by the Apache Qpid's Proton AMQP 1.0 toolkit. The blueprint uses the Python bindings for this toolkit, which are available on Pypi. These bindings, however, include a C extension that depends on the Proton toolkit development libraries in order to build and install. The lack of this toolkit is the cause of the blueprint's current CI failures. This toolkit is written in C, and thus requires platform-specific libraries. Now here's the problem: packages for Proton are not included by default in most distro's base repositories (yet). The Apache Qpid team has provided packages for EPEL, and has a PPA available for Ubuntu. Packages for Debian are also being proposed. I'm proposing this patch to openstack-infra/config to address the dependency problem [2]. It adds the proton toolkit packages to the common slave configuration. Does this make sense? Are there any better alternatives? For other cases where we need more native packages, we tyically use devstack to ensure they are installed. This is preferrable since it works for ordinary developers as well as the CI system. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo.messaging][infra] Adding support for AMQP 1.0 Messaging to Oslo.Messaging and infra/config
On Wed, 30 Jul 2014 14:25:46 +0100, Daniel P. Berrange wrote: On Wed, Jul 30, 2014 at 08:54:01AM -0400, Ken Giusti wrote: Greetings, Apologies for the cross-post: this should be of interest to both infra and olso.messaging developers. The blueprint [0] that adds support for version 1.0 of the AMQP messaging protocol is blocked due to CI test failures [1]. These failures are due to a new package dependency this blueprint adds to oslo.messaging. The AMQP 1.0 functionality is provided by the Apache Qpid's Proton AMQP 1.0 toolkit. The blueprint uses the Python bindings for this toolkit, which are available on Pypi. These bindings, however, include a C extension that depends on the Proton toolkit development libraries in order to build and install. The lack of this toolkit is the cause of the blueprint's current CI failures. This toolkit is written in C, and thus requires platform-specific libraries. Now here's the problem: packages for Proton are not included by default in most distro's base repositories (yet). The Apache Qpid team has provided packages for EPEL, and has a PPA available for Ubuntu. Packages for Debian are also being proposed. I'm proposing this patch to openstack-infra/config to address the dependency problem [2]. It adds the proton toolkit packages to the common slave configuration. Does this make sense? Are there any better alternatives? For other cases where we need more native packages, we tyically use devstack to ensure they are installed. This is preferrable since it works for ordinary developers as well as the CI system. Thanks Daniel. It was my understanding - which may be wrong - that having devstack install the 'out of band' packages would only help in the case of the devstack-based integration tests, not in the case of CI running the unit tests. Is that indeed the case? At this point, there are no integration tests that exercise the driver. However, the new unit tests include a test 'broker', which allow the unit tests to fully exercise the new driver, right down to the network. That's a bonus of AMQP 1.0 - it can support peer-2-peer messaging. So its the new unit tests that have the 'hard' requirement of the proton libraries.And mocking-out the proton libraries really doesn't allow us to do any meaningful tests of the driver. But if devstack is the preferred method for getting 'special case' packages installed, would it be acceptable to have the new unit tests run as a separate oslo.messaging integration test, and remove them from the unit tests? I'm open to any thoughts on how best to solve this, thanks. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo.messaging][infra] Adding support for AMQP 1.0 Messaging to Oslo.Messaging and infra/config
On 7/30/2014 11:59 AM, Ken Giusti wrote: On Wed, 30 Jul 2014 14:25:46 +0100, Daniel P. Berrange wrote: On Wed, Jul 30, 2014 at 08:54:01AM -0400, Ken Giusti wrote: Greetings, Apologies for the cross-post: this should be of interest to both infra and olso.messaging developers. The blueprint [0] that adds support for version 1.0 of the AMQP messaging protocol is blocked due to CI test failures [1]. These failures are due to a new package dependency this blueprint adds to oslo.messaging. The AMQP 1.0 functionality is provided by the Apache Qpid's Proton AMQP 1.0 toolkit. The blueprint uses the Python bindings for this toolkit, which are available on Pypi. These bindings, however, include a C extension that depends on the Proton toolkit development libraries in order to build and install. The lack of this toolkit is the cause of the blueprint's current CI failures. This toolkit is written in C, and thus requires platform-specific libraries. Now here's the problem: packages for Proton are not included by default in most distro's base repositories (yet). The Apache Qpid team has provided packages for EPEL, and has a PPA available for Ubuntu. Packages for Debian are also being proposed. I'm proposing this patch to openstack-infra/config to address the dependency problem [2]. It adds the proton toolkit packages to the common slave configuration. Does this make sense? Are there any better alternatives? For other cases where we need more native packages, we tyically use devstack to ensure they are installed. This is preferrable since it works for ordinary developers as well as the CI system. Thanks Daniel. It was my understanding - which may be wrong - that having devstack install the 'out of band' packages would only help in the case of the devstack-based integration tests, not in the case of CI running the unit tests. Is that indeed the case? At this point, there are no integration tests that exercise the driver. However, the new unit tests include a test 'broker', which allow the unit tests to fully exercise the new driver, right down to the network. That's a bonus of AMQP 1.0 - it can support peer-2-peer messaging. So its the new unit tests that have the 'hard' requirement of the proton libraries.And mocking-out the proton libraries really doesn't allow us to do any meaningful tests of the driver. But if devstack is the preferred method for getting 'special case' packages installed, would it be acceptable to have the new unit tests run as a separate oslo.messaging integration test, and remove them from the unit tests? I'm open to any thoughts on how best to solve this, thanks. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev If your unit tests are dependent on a specific dependent library aren't they no longer unit tests but integration tests anyway? Just wondering, not trying to put up road-blocks because I'd like to see how this code performs but haven't had time yet to play with it. -- Thanks, Matt Riedemann ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo.messaging][infra] Adding support for AMQP 1.0 Messaging to Oslo.Messaging and infra/config
On 2014-07-30 14:59:09 -0400 (-0400), Ken Giusti wrote: Thanks Daniel. It was my understanding - which may be wrong - that having devstack install the 'out of band' packages would only help in the case of the devstack-based integration tests, not in the case of CI running the unit tests. Is that indeed the case? [...] I'm open to any thoughts on how best to solve this, thanks. Since they're in EPEL and we run Python 2.6 unit tests today on CentOS 6 servers, if the proton libraries install successfully there perhaps we could opportunistically exercise it only under Python 2.6 for now? Not ideal, but it does get it enforced upstream with minimal fuss. I'd really rather not start accumulating arbitrary PPA sources on our job workers... I know we've done it for a handful of multi-project efforts where we needed select backports from non-LTS releases, but we've so far limited that to only PPAs maintained by the same package teams as the mainline distro packages themselves. Longer term, I'd suggest getting it sponsored into Debian unstable/testing ASAP, interesting the Ubuntu OpenStack team in importing it into the development tree for the next Ubuntu release, and then incorporating it into the Trusty Ubuntu Cloud Archive. We're not using UCA yet, but on Trusty we probably should consider adding it sooner rather than later since when we tried to tack on the Precise UCA in the last couple cycles we had too many headaches from trying to jump ahead substantially on fundamental bits like libvirt. Breaking sooner and more often means those incremental issues are easier to identify and address, usually. -- Jeremy Stanley ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
