Re: [openstack-dev] [qa][Neutron][Tempest][Network] Break down NetworkBasicOps to smaller test cases

2014-01-22 Thread Jay Pipes
On Tue, 2014-01-21 at 01:15 -0500, Yair Fried wrote: I seem to be unable to convey my point using generalization, so I will give a specific example: I would like to have update dns server as an additional network scenario. Currently I could add it to the existing module: 1. tests

Re: [openstack-dev] [qa][Neutron][Tempest][Network] Break down NetworkBasicOps to smaller test cases

2014-01-22 Thread David Kranz
On 01/22/2014 03:19 PM, Jay Pipes wrote: On Tue, 2014-01-21 at 01:15 -0500, Yair Fried wrote: I seem to be unable to convey my point using generalization, so I will give a specific example: I would like to have update dns server as an additional network scenario. Currently I could add it to

Re: [openstack-dev] [qa][Neutron][Tempest][Network] Break down NetworkBasicOps to smaller test cases

2014-01-20 Thread Jay Pipes
On Sun, 2014-01-19 at 07:17 -0500, Yair Fried wrote: OK, but considering my pending patch (#3 and #4) what about: #1 - #2 #1 - #3 #1 - #4 instead of #1 - #2 - #3 - #4 a failure in #2 will prevent #3 and #4 from running even though they are completely unrelated Seems to me,

Re: [openstack-dev] [qa][Neutron][Tempest][Network] Break down NetworkBasicOps to smaller test cases

2014-01-20 Thread Salvatore Orlando
Yair is probably referring to statistically independent tests, or whatever case for which the following is true (P(x) is the probably that a test succeeds): P(4|3|2|1) = P(4|1) * P(3|1) * P(2|1) This might apply to the tests we are adding to network_basic_ops scenario; however it is worth noting

Re: [openstack-dev] [qa][Neutron][Tempest][Network] Break down NetworkBasicOps to smaller test cases

2014-01-20 Thread Yair Fried
) openstack-dev@lists.openstack.org Sent: Monday, January 20, 2014 7:22:22 PM Subject: Re: [openstack-dev] [qa][Neutron][Tempest][Network] Break down NetworkBasicOps to smaller test cases Yair is probably referring to statistically independent tests, or whatever case for which the following is true (P

Re: [openstack-dev] [qa][Neutron][Tempest][Network] Break down NetworkBasicOps to smaller test cases

2014-01-19 Thread Sean Dague
questions) openstack-dev@lists.openstack.org Sent: Friday, January 17, 2014 6:17:55 AM Subject: Re: [openstack-dev] [qa][Neutron][Tempest][Network] Break down NetworkBasicOps to smaller test cases On Wed, Jan 15, 2014 at 11:20:22AM -0500, Yair Fried wrote: Hi Guys As Maru pointed out

Re: [openstack-dev] [qa][Neutron][Tempest][Network] Break down NetworkBasicOps to smaller test cases

2014-01-19 Thread Yair Fried
-dev@lists.openstack.org Sent: Friday, January 17, 2014 6:17:55 AM Subject: Re: [openstack-dev] [qa][Neutron][Tempest][Network] Break down NetworkBasicOps to smaller test cases On Wed, Jan 15, 2014 at 11:20:22AM -0500, Yair Fried wrote: Hi Guys As Maru pointed out - NetworkBasicOps

Re: [openstack-dev] [qa][Neutron][Tempest][Network] Break down NetworkBasicOps to smaller test cases

2014-01-18 Thread Yair Fried
Sent: Friday, January 17, 2014 6:17:55 AM Subject: Re: [openstack-dev] [qa][Neutron][Tempest][Network] Break down NetworkBasicOps to smaller test cases On Wed, Jan 15, 2014 at 11:20:22AM -0500, Yair Fried wrote: Hi Guys As Maru pointed out - NetworkBasicOps scenario has grown out of proportion

Re: [openstack-dev] [qa][Neutron][Tempest][Network] Break down NetworkBasicOps to smaller test cases

2014-01-16 Thread Matthew Treinish
On Wed, Jan 15, 2014 at 11:20:22AM -0500, Yair Fried wrote: Hi Guys As Maru pointed out - NetworkBasicOps scenario has grown out of proportion and is no longer basic ops. Is your issue here that it's just called basic ops and you don't think that's reflective of what is being tested in that

[openstack-dev] [qa][Neutron][Tempest][Network] Break down NetworkBasicOps to smaller test cases

2014-01-15 Thread Yair Fried
Hi Guys As Maru pointed out - NetworkBasicOps scenario has grown out of proportion and is no longer basic ops. So, I started breaking it down to smaller test cases that can fail independently. Those test cases share the same setup and tear-down code: 1. create network resources (and verify