Re: [openstack-dev] [neutron][testing] How to modify DSVM tests to use a DevStack plugin?

2015-07-27 Thread Sean Dague 
You would build variants of the jobs you want that specifically enable
your plugin.

That being said, you should focus on jobs that substantially test your
component, not just the giant list of all jobs. Part of our focus in on
decoupling so that for something like vpnaas you can start with the
assumption that neutron base services are sufficiently tested elsewhere,
and the only thing you should test is the additional function and
complexity that your component brings to the mix.

-Sean

On 07/27/2015 07:44 AM, Paul Michali wrote:
 Yes, the plugin enables the service, and for the neutron-vpnaas DSVM
 based jobs, I have the enable_plugin line added to the job so that
 everything works.
 
 However, for the DevStack repo, which runs a bunch of other DSVM jobs,
 this fails, as there is (obviously) no enable_plugin line.:
 
   * gate-tempest-dsvm-full
 
 http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-full/98be491/ 
 SUCCESS in
 58m 37s
   * gate-tempest-dsvm-postgres-full
 
 http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-postgres-full/85c5b92/
  SUCCESS in
 50m 45s
   * gate-tempest-dsvm-neutron-full
 
 http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-neutron-full/0050bfe/
  FAILURE in
 1h 25m 30s
   * gate-grenade-dsvm
 http://logs.openstack.org/19/201119/1/check/gate-grenade-dsvm/b224606/ 
 SUCCESS in
 44m 23s
   * gate-tempest-dsvm-large-ops
 
 http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-large-ops/a250cf5/
  SUCCESS in
 26m 49s
   * gate-tempest-dsvm-neutron-large-ops
 
 http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-neutron-large-ops/6faa1be/
  SUCCESS in
 25m 51s
   * gate-devstack-bashate
 
 http://logs.openstack.org/19/201119/1/check/gate-devstack-bashate/65ad952/ 
 SUCCESS in
 13s
   * gate-devstack-unit-tests
 
 http://logs.openstack.org/19/201119/1/check/gate-devstack-unit-tests/ccdbe4e/
  SUCCESS in
 1m 02s
   * gate-devstack-dsvm-cells
 
 http://logs.openstack.org/19/201119/1/check/gate-devstack-dsvm-cells/a6ca00c/
  SUCCESS in
 24m 08s
   * gate-grenade-dsvm-partial-ncpu
 
 http://logs.openstack.org/19/201119/1/check/gate-grenade-dsvm-partial-ncpu/744deb8/
  SUCCESS in
 48m 36s
   * gate-tempest-dsvm-ironic-pxe_ssh
 
 http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-ironic-pxe_ssh/8eb4315/
  FAILURE in
 40m 10s
   * gate-devstack-dsvm-updown
 
 http://logs.openstack.org/19/201119/1/check/gate-devstack-dsvm-updown/85f1de5/
  SUCCESS in
 21m 12s
   * gate-tempest-dsvm-f21
 
 http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-f21/35a04c4/ 
 FAILURE in
 51m 01s (non-voting)
   * gate-tempest-dsvm-centos7
 
 http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-centos7/b9c99c9/
  SUCCESS in
 30m 23s (non-voting)
   * gate-devstack-publish-docs
 
 http://docs-draft.openstack.org/19/201119/1/check/gate-devstack-publish-docs/f794b1c//doc/build/html/
  SUCCESS in
 2m 23s
   * gate-swift-dsvm-functional-nv
 
 http://logs.openstack.org/19/201119/1/check/gate-swift-dsvm-functional-nv/13d2c58/
  SUCCESS in
 27m 12s (non-voting)
   * gate-grenade-dsvm-neutron
 
 http://logs.openstack.org/19/201119/1/check/gate-grenade-dsvm-neutron/8675f0c/
  FAILURE in
 47m 49s
   * gate-tempest-dsvm-multinode-smoke
 
 http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-multinode-smoke/bd69c45/
  SUCCESS in
 36m 53s (non-voting)
   * gate-tempest-dsvm-neutron-multinode-smoke
 
 http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-neutron-multinode-smoke/01e1d45/
  FAILURE in
 44m 16s (non-voting)
 
 
 I'm wondering what's the best way to modify those jobs... is there some
 common location where I can enable the plugin to handle all DSVM based
 jobs, do I just update the 5 failing tests, do I update just the 3
 voting tests, or do I update all 16 DSVM based jobs?
 
 Regards,
 PCM
 
 On Fri, Jul 24, 2015 at 5:12 PM Clark Boylan cboy...@sapwetik.org
 mailto:cboy...@sapwetik.org wrote:
 
 On Fri, Jul 24, 2015, at 02:05 PM, Paul Michali wrote:
  Hi,
 
  I've created a DevStack plugin for the neutron-vpnaas repo. Now, I'm
  trying
  to remove the q-vpn service setup from the DevStack repo (
  https://review.openstack.org/#/c/201119/).
 
  However, I'm hitting an issue in that (almost) every test that uses
  DevStack fails, because it is no longer setting up q-vpn.
 
  How should I modify the tests, so that they setup the q-vpn
 service, in
  light of the fact that there is a DevStack plugin available for it. Is
  there some common place that I can do the enable_plugin
  neutron-vpnaas...
  line?
 
 Your devstack plugin should enable the service. Then in your jobs you
 just need to enable the plugin which will then enable the vpn service.
 There should be plenty of prior

Re: [openstack-dev] [neutron][testing] How to modify DSVM tests to use a DevStack plugin?

2015-07-27 Thread Paul Michali 
Maybe I'm not explaining myself well (sorry)...

For VPN commits, there are functional jobs that (now) enable the devstack
plugin for neutron-vpnaas as needed (and grenade job will do the same).
From the neutron-vpnaas repo standpoint everything is in place.

Now that there is a devstack plugin for neutron-vpnaas, I want to remove
all the VPN setup from the *DevStack* repo's setup, as the user of DevStack
can specify the enable_plugin in their local.conf file now. The commit is
https://review.openstack.org/#/c/201119/.

The issue I see though, is that the DevStack repo's jobs are failing,
because they are using devstack, are relying on VPN being set up, and the
enable_plugin line for VPN isn't part of any of the jobs shown in my last
post.

How do we resolve that issue?

Regards,

PCM


On Mon, Jul 27, 2015 at 8:09 AM Sean Dague s...@dague.net wrote:

 You would build variants of the jobs you want that specifically enable
 your plugin.

 That being said, you should focus on jobs that substantially test your
 component, not just the giant list of all jobs. Part of our focus in on
 decoupling so that for something like vpnaas you can start with the
 assumption that neutron base services are sufficiently tested elsewhere,
 and the only thing you should test is the additional function and
 complexity that your component brings to the mix.

 -Sean

 On 07/27/2015 07:44 AM, Paul Michali wrote:
  Yes, the plugin enables the service, and for the neutron-vpnaas DSVM
  based jobs, I have the enable_plugin line added to the job so that
  everything works.
 
  However, for the DevStack repo, which runs a bunch of other DSVM jobs,
  this fails, as there is (obviously) no enable_plugin line.:
 
* gate-tempest-dsvm-full
  
 http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-full/98be491/
 SUCCESS in
  58m 37s
* gate-tempest-dsvm-postgres-full
  
 http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-postgres-full/85c5b92/
 SUCCESS in
  50m 45s
* gate-tempest-dsvm-neutron-full
  
 http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-neutron-full/0050bfe/
 FAILURE in
  1h 25m 30s
* gate-grenade-dsvm
  
 http://logs.openstack.org/19/201119/1/check/gate-grenade-dsvm/b224606/
 SUCCESS in
  44m 23s
* gate-tempest-dsvm-large-ops
  
 http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-large-ops/a250cf5/
 SUCCESS in
  26m 49s
* gate-tempest-dsvm-neutron-large-ops
  
 http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-neutron-large-ops/6faa1be/
 SUCCESS in
  25m 51s
* gate-devstack-bashate
  
 http://logs.openstack.org/19/201119/1/check/gate-devstack-bashate/65ad952/
 SUCCESS in
  13s
* gate-devstack-unit-tests
  
 http://logs.openstack.org/19/201119/1/check/gate-devstack-unit-tests/ccdbe4e/
 SUCCESS in
  1m 02s
* gate-devstack-dsvm-cells
  
 http://logs.openstack.org/19/201119/1/check/gate-devstack-dsvm-cells/a6ca00c/
 SUCCESS in
  24m 08s
* gate-grenade-dsvm-partial-ncpu
  
 http://logs.openstack.org/19/201119/1/check/gate-grenade-dsvm-partial-ncpu/744deb8/
 SUCCESS in
  48m 36s
* gate-tempest-dsvm-ironic-pxe_ssh
  
 http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-ironic-pxe_ssh/8eb4315/
 FAILURE in
  40m 10s
* gate-devstack-dsvm-updown
  
 http://logs.openstack.org/19/201119/1/check/gate-devstack-dsvm-updown/85f1de5/
 SUCCESS in
  21m 12s
* gate-tempest-dsvm-f21
  
 http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-f21/35a04c4/
 FAILURE in
  51m 01s (non-voting)
* gate-tempest-dsvm-centos7
  
 http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-centos7/b9c99c9/
 SUCCESS in
  30m 23s (non-voting)
* gate-devstack-publish-docs
  
 http://docs-draft.openstack.org/19/201119/1/check/gate-devstack-publish-docs/f794b1c//doc/build/html/
 SUCCESS in
  2m 23s
* gate-swift-dsvm-functional-nv
  
 http://logs.openstack.org/19/201119/1/check/gate-swift-dsvm-functional-nv/13d2c58/
 SUCCESS in
  27m 12s (non-voting)
* gate-grenade-dsvm-neutron
  
 http://logs.openstack.org/19/201119/1/check/gate-grenade-dsvm-neutron/8675f0c/
 FAILURE in
  47m 49s
* gate-tempest-dsvm-multinode-smoke
  
 http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-multinode-smoke/bd69c45/
 SUCCESS in
  36m 53s (non-voting)
* gate-tempest-dsvm-neutron-multinode-smoke
  
 http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-neutron-multinode-smoke/01e1d45/
 FAILURE in
  44m 16s (non-voting)
 
 
  I'm wondering what's the best way to modify those jobs... is there some
  common location where I can enable the plugin to handle all DSVM based
  jobs, do I just update the 5 failing tests, do I update just the 3
  voting tests, or do I update all 16 DSVM based jobs?
 
  Regards,
  PCM
 
  On Fri, Jul 24, 2015 at 5:12 PM Clark Boylan

Re: [openstack-dev] [neutron][testing] How to modify DSVM tests to use a DevStack plugin?

2015-07-27 Thread Paul Michali 
Yes, the plugin enables the service, and for the neutron-vpnaas DSVM based
jobs, I have the enable_plugin line added to the job so that everything
works.

However, for the DevStack repo, which runs a bunch of other DSVM jobs, this
fails, as there is (obviously) no enable_plugin line.:


   - gate-tempest-dsvm-full
   http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-full/98be491/
SUCCESS in 58m 37s
   - gate-tempest-dsvm-postgres-full
   
http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-postgres-full/85c5b92/
SUCCESS in 50m 45s
   - gate-tempest-dsvm-neutron-full
   
http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-neutron-full/0050bfe/
FAILURE in 1h 25m 30s
   - gate-grenade-dsvm
   http://logs.openstack.org/19/201119/1/check/gate-grenade-dsvm/b224606/
   SUCCESS in 44m 23s
   - gate-tempest-dsvm-large-ops
   
http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-large-ops/a250cf5/
SUCCESS in 26m 49s
   - gate-tempest-dsvm-neutron-large-ops
   
http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-neutron-large-ops/6faa1be/
SUCCESS in 25m 51s
   - gate-devstack-bashate
   http://logs.openstack.org/19/201119/1/check/gate-devstack-bashate/65ad952/
SUCCESS in 13s
   - gate-devstack-unit-tests
   
http://logs.openstack.org/19/201119/1/check/gate-devstack-unit-tests/ccdbe4e/
SUCCESS in 1m 02s
   - gate-devstack-dsvm-cells
   
http://logs.openstack.org/19/201119/1/check/gate-devstack-dsvm-cells/a6ca00c/
SUCCESS in 24m 08s
   - gate-grenade-dsvm-partial-ncpu
   
http://logs.openstack.org/19/201119/1/check/gate-grenade-dsvm-partial-ncpu/744deb8/
SUCCESS in 48m 36s
   - gate-tempest-dsvm-ironic-pxe_ssh
   
http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-ironic-pxe_ssh/8eb4315/
FAILURE in 40m 10s
   - gate-devstack-dsvm-updown
   
http://logs.openstack.org/19/201119/1/check/gate-devstack-dsvm-updown/85f1de5/
SUCCESS in 21m 12s
   - gate-tempest-dsvm-f21
   http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-f21/35a04c4/
FAILURE in 51m 01s (non-voting)
   - gate-tempest-dsvm-centos7
   
http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-centos7/b9c99c9/
SUCCESS in 30m 23s (non-voting)
   - gate-devstack-publish-docs
   
http://docs-draft.openstack.org/19/201119/1/check/gate-devstack-publish-docs/f794b1c//doc/build/html/
SUCCESS in 2m 23s
   - gate-swift-dsvm-functional-nv
   
http://logs.openstack.org/19/201119/1/check/gate-swift-dsvm-functional-nv/13d2c58/
SUCCESS in 27m 12s (non-voting)
   - gate-grenade-dsvm-neutron
   
http://logs.openstack.org/19/201119/1/check/gate-grenade-dsvm-neutron/8675f0c/
FAILURE in 47m 49s
   - gate-tempest-dsvm-multinode-smoke
   
http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-multinode-smoke/bd69c45/
SUCCESS in 36m 53s (non-voting)
   - gate-tempest-dsvm-neutron-multinode-smoke
   
http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-neutron-multinode-smoke/01e1d45/
FAILURE in 44m 16s (non-voting)


I'm wondering what's the best way to modify those jobs... is there some
common location where I can enable the plugin to handle all DSVM based
jobs, do I just update the 5 failing tests, do I update just the 3 voting
tests, or do I update all 16 DSVM based jobs?

Regards,
PCM

On Fri, Jul 24, 2015 at 5:12 PM Clark Boylan cboy...@sapwetik.org wrote:

 On Fri, Jul 24, 2015, at 02:05 PM, Paul Michali wrote:
  Hi,
 
  I've created a DevStack plugin for the neutron-vpnaas repo. Now, I'm
  trying
  to remove the q-vpn service setup from the DevStack repo (
  https://review.openstack.org/#/c/201119/).
 
  However, I'm hitting an issue in that (almost) every test that uses
  DevStack fails, because it is no longer setting up q-vpn.
 
  How should I modify the tests, so that they setup the q-vpn service, in
  light of the fact that there is a DevStack plugin available for it. Is
  there some common place that I can do the enable_plugin
  neutron-vpnaas...
  line?
 
 Your devstack plugin should enable the service. Then in your jobs you
 just need to enable the plugin which will then enable the vpn service.
 There should be plenty of prior art with the ec2api plugin, glusterfs
 plugin, and others.

 Clark

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][testing] How to modify DSVM tests to use a DevStack plugin?

2015-07-27 Thread Sean Dague 
On 07/27/2015 08:21 AM, Paul Michali wrote:
 Maybe I'm not explaining myself well (sorry)...
 
 For VPN commits, there are functional jobs that (now) enable the
 devstack plugin for neutron-vpnaas as needed (and grenade job will do
 the same). From the neutron-vpnaas repo standpoint everything is in place.
 
 Now that there is a devstack plugin for neutron-vpnaas, I want to remove
 all the VPN setup from the *DevStack* repo's setup, as the user of
 DevStack can specify the enable_plugin in their local.conf file now. The
 commit is https://review.openstack.org/#/c/201119/.
 
 The issue I see though, is that the DevStack repo's jobs are failing,
 because they are using devstack, are relying on VPN being set up, and
 the enable_plugin line for VPN isn't part of any of the jobs shown in my
 last post.
 
 How do we resolve that issue?

Presumably there is a flag in Tempest for whether or not this service
should be tested? That would be where I'd look.

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][testing] How to modify DSVM tests to use a DevStack plugin?

2015-07-27 Thread Paul Michali 
Not being very familiar with how this all works, can someone provide a bit
more hand holding here?

The overall question is, do we remove VPN from all the DevStack based tests
(except for those run by VPN repo)?

Thanks,

PCM


On Mon, Jul 27, 2015 at 8:26 AM Sean Dague s...@dague.net wrote:

 On 07/27/2015 08:21 AM, Paul Michali wrote:
  Maybe I'm not explaining myself well (sorry)...
 
  For VPN commits, there are functional jobs that (now) enable the
  devstack plugin for neutron-vpnaas as needed (and grenade job will do
  the same). From the neutron-vpnaas repo standpoint everything is in
 place.
 
  Now that there is a devstack plugin for neutron-vpnaas, I want to remove
  all the VPN setup from the *DevStack* repo's setup, as the user of
  DevStack can specify the enable_plugin in their local.conf file now. The
  commit is https://review.openstack.org/#/c/201119/.
 
  The issue I see though, is that the DevStack repo's jobs are failing,
  because they are using devstack, are relying on VPN being set up, and
  the enable_plugin line for VPN isn't part of any of the jobs shown in my
  last post.
 
  How do we resolve that issue?

 Presumably there is a flag in Tempest for whether or not this service
 should be tested? That would be where I'd look.

 -Sean

 --
 Sean Dague
 http://dague.net

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][testing] How to modify DSVM tests to use a DevStack plugin?

2015-07-24 Thread Clark Boylan 
On Fri, Jul 24, 2015, at 02:05 PM, Paul Michali wrote:
 Hi,
 
 I've created a DevStack plugin for the neutron-vpnaas repo. Now, I'm
 trying
 to remove the q-vpn service setup from the DevStack repo (
 https://review.openstack.org/#/c/201119/).
 
 However, I'm hitting an issue in that (almost) every test that uses
 DevStack fails, because it is no longer setting up q-vpn.
 
 How should I modify the tests, so that they setup the q-vpn service, in
 light of the fact that there is a DevStack plugin available for it. Is
 there some common place that I can do the enable_plugin
 neutron-vpnaas...
 line?

Your devstack plugin should enable the service. Then in your jobs you
just need to enable the plugin which will then enable the vpn service.
There should be plenty of prior art with the ec2api plugin, glusterfs
plugin, and others.

Clark

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev