Re: [openstack-dev] [Neutron] [Tempest] where fwaas tempest tests should be?

[Takashi Yamamoto]

hi,

On Sat, Oct 24, 2015 at 9:25 PM, Doug Wiegley
 wrote:
>
>> On Oct 19, 2015, at 5:33 PM, Ihar Hrachyshka  wrote:
>>
>>> On 16 Oct 2015, at 10:50, Takashi Yamamoto  wrote:
>>>
>>> if i move fwaas tests from neutron to neutron-fwaas, [1]
>>> is there easy way to run them together with the rest of neutron api tests
>>> for gate-neutron-dsvm-api job?
>>
>> Before we jump in to reflect current gating status-quo, do we have a 
>> definite answer why we want to keep the gate coupling in place? Can’t we 
>> leave the fwaas api job for fwaas repo only? Do we think core is unstable 
>> enough to justify additional coupling? Is core bad at handling backwards 
>> compatibility when it comes to (re)moving code that is unneeded for the core 
>> repo?
>
> Core is absolutely dreadful at backwards compatibility (I’m talking recent 
> history here.) The co-gate is necessary to guard against that at the moment.

do you still want to keep the co-gate?
i was trying to keep the co-gate [2] but armando doesn't seem to like it.
any suggestions?

[2] https://review.openstack.org/#/c/242925/

>
> Thanks,
> doug
>
>>
>> That may actually be a dumb question, so don’t hesitate to point out obvious 
>> reasons to keep co-gating.
>>
>> Ihar
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] [Tempest] where fwaas tempest tests should be?

[Ihar Hrachyshka]

> On 24 Oct 2015, at 21:25, Doug Wiegley  wrote:
> 
> 
>> On Oct 19, 2015, at 5:33 PM, Ihar Hrachyshka  wrote:
>> 
>>> On 16 Oct 2015, at 10:50, Takashi Yamamoto  wrote:
>>> 
>>> if i move fwaas tests from neutron to neutron-fwaas, [1]
>>> is there easy way to run them together with the rest of neutron api tests
>>> for gate-neutron-dsvm-api job?
>> 
>> Before we jump in to reflect current gating status-quo, do we have a 
>> definite answer why we want to keep the gate coupling in place? Can’t we 
>> leave the fwaas api job for fwaas repo only? Do we think core is unstable 
>> enough to justify additional coupling? Is core bad at handling backwards 
>> compatibility when it comes to (re)moving code that is unneeded for the core 
>> repo?
> 
> Core is absolutely dreadful at backwards compatibility (I’m talking recent 
> history here.) The co-gate is necessary to guard against that at the moment.

Interesting. Any details? Is it just a matter of some dumb code move/rename, or 
is it something really architectural, like single API manager for all services?

I wonder what we may do to make the difference for eventual breakage of the 
tie. Should we start collecting post-mortem details on breakage cases to start 
understanding where we lag?

Ihar


signature.asc
Description: Message signed with OpenPGP using GPGMail
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] [Tempest] where fwaas tempest tests should be?

[Doug Wiegley]

> On Oct 19, 2015, at 5:33 PM, Ihar Hrachyshka  wrote:
> 
>> On 16 Oct 2015, at 10:50, Takashi Yamamoto  wrote:
>> 
>> if i move fwaas tests from neutron to neutron-fwaas, [1]
>> is there easy way to run them together with the rest of neutron api tests
>> for gate-neutron-dsvm-api job?
> 
> Before we jump in to reflect current gating status-quo, do we have a definite 
> answer why we want to keep the gate coupling in place? Can’t we leave the 
> fwaas api job for fwaas repo only? Do we think core is unstable enough to 
> justify additional coupling? Is core bad at handling backwards compatibility 
> when it comes to (re)moving code that is unneeded for the core repo?

Core is absolutely dreadful at backwards compatibility (I’m talking recent 
history here.) The co-gate is necessary to guard against that at the moment.

Thanks,
doug

> 
> That may actually be a dumb question, so don’t hesitate to point out obvious 
> reasons to keep co-gating.
> 
> Ihar
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] [Tempest] where fwaas tempest tests should be?

[Sean M. Collins]

My only concern is how tightly coupled the FwaaS code is with the L3 agent. We 
have a bug already filed because the L3 agent inheriting the FwaaS code causes 
circular dependencies, but it also be the reason why FwaaS co-gates with the 
main Neutron repo.

Let's try and fix the tight coupling of the L3 agent with the FwaaS code so we 
can get FwaaS to just gate by itself.
-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] [Tempest] where fwaas tempest tests should be?

[Takashi Yamamoto]

hi,

On Mon, Oct 19, 2015 at 5:33 PM, Ihar Hrachyshka  wrote:
>> On 16 Oct 2015, at 10:50, Takashi Yamamoto  wrote:
>>
>> if i move fwaas tests from neutron to neutron-fwaas, [1]
>> is there easy way to run them together with the rest of neutron api tests
>> for gate-neutron-dsvm-api job?
>
> Before we jump in to reflect current gating status-quo, do we have a definite 
> answer why we want to keep the gate coupling in place? Can’t we leave the 
> fwaas api job for fwaas repo only? Do we think core is unstable enough to 
> justify additional coupling? Is core bad at handling backwards compatibility 
> when it comes to (re)moving code that is unneeded for the core repo?
>
> That may actually be a dumb question, so don’t hesitate to point out obvious 
> reasons to keep co-gating.

it sounds like a good question.
i tend to think fwaas jobs for fwaas repo only is good enough.

>
> Ihar
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] [Tempest] where fwaas tempest tests should be?

[Takashi Yamamoto]

hi,

On Mon, Oct 19, 2015 at 2:52 PM, Takashi Yamamoto  wrote:
> hi,
>
> On Thu, Oct 15, 2015 at 11:22 PM, Matthew Treinish  
> wrote:
>> On Thu, Oct 15, 2015 at 09:02:22AM -0400, Assaf Muller wrote:
>>> On Thu, Oct 15, 2015 at 7:25 AM, Takashi Yamamoto 
>>> wrote:
>>>
>>> > hi,
>>> >
>>> > i'm looking in fwaas tempest tests and have a question about code 
>>> > location.
>>> >
>>> > currently,
>>> >
>>> > - fwaas api tests and its rest client are in neutron repo
>>> > - there are no fwaas scenario tests
>>> >
>>> > eventually,
>>> >
>>> > - fwaas api tests should be moved into neutron-fwaas repo
>>> > - fwaaa scenario tests should be in neutron-fwaas repo too.
>>> >
>>>
>>> I believe scenario tests that invoke APIs outside of Neutron should
>>> stay (Or be introduced to) Tempest.
>>
>> So testing the neutron advanced services was actually one of the first things
>> we decided was out of scope for tempest. (like well over a year ago) It took
>> some time to get equivalent testing setup elsewhere, but tests and support 
>> for
>> the advanced services were removed from tempest on purpose. I'd suggest that
>> you look at the tempest plugin interface:
>>
>> http://docs.openstack.org/developer/tempest/plugin.html
>>
>> if you'd like to make the fwaas tests (or any other adv. service tests)
>> integrate more cleanly with the rest of tempest.
>
> is there a clean way for a plugin to extend NetworkClientJSON for fwaas?

i created separate clients for each resources
as it seems the way tempest is going to take.
https://review.openstack.org/#/c/236890/

>
>>
>> -Matt Treinish
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] [Tempest] where fwaas tempest tests should be?

[Ihar Hrachyshka]

> On 16 Oct 2015, at 10:50, Takashi Yamamoto  wrote:
> 
> if i move fwaas tests from neutron to neutron-fwaas, [1]
> is there easy way to run them together with the rest of neutron api tests
> for gate-neutron-dsvm-api job?

Before we jump in to reflect current gating status-quo, do we have a definite 
answer why we want to keep the gate coupling in place? Can’t we leave the fwaas 
api job for fwaas repo only? Do we think core is unstable enough to justify 
additional coupling? Is core bad at handling backwards compatibility when it 
comes to (re)moving code that is unneeded for the core repo?

That may actually be a dumb question, so don’t hesitate to point out obvious 
reasons to keep co-gating.

Ihar


signature.asc
Description: Message signed with OpenPGP using GPGMail
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] [Tempest] where fwaas tempest tests should be?

[Takashi Yamamoto]

hi,

On Sat, Oct 17, 2015 at 12:50 AM, Eichberger, German
 wrote:
> Hi,
>
> Just moving the tests is not sufficient since the gate jobs also need
> adjustment.

yea.
do yo have any advice how it should be done?
i submitted an experimental job [2] so that the movement can be tested at least.
how do you think?

[2] https://review.openstack.org/#/c/236762/

>
> German
>
> On 10/16/15, 1:50 AM, "Takashi Yamamoto"  wrote:
>
>>hi,
>>
>>On Thu, Oct 15, 2015 at 11:17 PM, Matthew Treinish 
>>wrote:
>>> On Thu, Oct 15, 2015 at 08:25:40PM +0900, Takashi Yamamoto wrote:
 hi,

 i'm looking in fwaas tempest tests and have a question about code
location.

 currently,

 - fwaas api tests and its rest client are in neutron repo
>>>
>>> This is a bad situation because it means that we're directly coupling
>>>fwaas to
>>> the neutron repo. Everything that depends on fwaas should be separate
>>>and self
>>> contained outside of the neutron repo. The longer that things are like
>>>this it
>>> will just lead to more headaches down the road.
>>>
 - there are no fwaas scenario tests

 eventually,

 - fwaas api tests should be moved into neutron-fwaas repo
 - fwaaa scenario tests should be in neutron-fwaas repo too.
>>>
>>> This is definitely the right direction.
>>
>>if i move fwaas tests from neutron to neutron-fwaas, [1]
>>is there easy way to run them together with the rest of neutron api tests
>>for gate-neutron-dsvm-api job?
>>
>>[1] https://review.openstack.org/#/c/235790/
>>
>>>
 - the rest client will be in tempest-lib
>>>
>>> The discussion on which clients are in scope for tempest-lib hasn't
>>>fully
>>> happened yet. For right now we're taking a conservative approach and
>>>the clients
>>> can live in tempest-lib if there are tests in the tempest tree using
>>>them.
>>> (which is not fwaas) This might change eventually, (there will be a
>>>summit
>>> session on it) but for right now I'd say the fwaas clients should live
>>>in the
>>> fwaas repo. (they can and likely should still be based on the
>>>tempest-lib rest
>>> client)
>>>
>>> -Matt Treinish
>>>
>>>
>>>_
>>>_
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>>openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>__
>>OpenStack Development Mailing List (not for usage questions)
>>Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] [Tempest] where fwaas tempest tests should be?

[Takashi Yamamoto]

hi,

On Thu, Oct 15, 2015 at 11:22 PM, Matthew Treinish  wrote:
> On Thu, Oct 15, 2015 at 09:02:22AM -0400, Assaf Muller wrote:
>> On Thu, Oct 15, 2015 at 7:25 AM, Takashi Yamamoto 
>> wrote:
>>
>> > hi,
>> >
>> > i'm looking in fwaas tempest tests and have a question about code location.
>> >
>> > currently,
>> >
>> > - fwaas api tests and its rest client are in neutron repo
>> > - there are no fwaas scenario tests
>> >
>> > eventually,
>> >
>> > - fwaas api tests should be moved into neutron-fwaas repo
>> > - fwaaa scenario tests should be in neutron-fwaas repo too.
>> >
>>
>> I believe scenario tests that invoke APIs outside of Neutron should
>> stay (Or be introduced to) Tempest.
>
> So testing the neutron advanced services was actually one of the first things
> we decided was out of scope for tempest. (like well over a year ago) It took
> some time to get equivalent testing setup elsewhere, but tests and support for
> the advanced services were removed from tempest on purpose. I'd suggest that
> you look at the tempest plugin interface:
>
> http://docs.openstack.org/developer/tempest/plugin.html
>
> if you'd like to make the fwaas tests (or any other adv. service tests)
> integrate more cleanly with the rest of tempest.

is there a clean way for a plugin to extend NetworkClientJSON for fwaas?

>
> -Matt Treinish
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] [Tempest] where fwaas tempest tests should be?

[Takashi Yamamoto]

hi,

On Thu, Oct 15, 2015 at 11:17 PM, Matthew Treinish  wrote:
> On Thu, Oct 15, 2015 at 08:25:40PM +0900, Takashi Yamamoto wrote:
>> hi,
>>
>> i'm looking in fwaas tempest tests and have a question about code location.
>>
>> currently,
>>
>> - fwaas api tests and its rest client are in neutron repo
>
> This is a bad situation because it means that we're directly coupling fwaas to
> the neutron repo. Everything that depends on fwaas should be separate and self
> contained outside of the neutron repo. The longer that things are like this it
> will just lead to more headaches down the road.
>
>> - there are no fwaas scenario tests
>>
>> eventually,
>>
>> - fwaas api tests should be moved into neutron-fwaas repo
>> - fwaaa scenario tests should be in neutron-fwaas repo too.
>
> This is definitely the right direction.

if i move fwaas tests from neutron to neutron-fwaas, [1]
is there easy way to run them together with the rest of neutron api tests
for gate-neutron-dsvm-api job?

[1] https://review.openstack.org/#/c/235790/

>
>> - the rest client will be in tempest-lib
>
> The discussion on which clients are in scope for tempest-lib hasn't fully
> happened yet. For right now we're taking a conservative approach and the 
> clients
> can live in tempest-lib if there are tests in the tempest tree using them.
> (which is not fwaas) This might change eventually, (there will be a summit
> session on it) but for right now I'd say the fwaas clients should live in the
> fwaas repo. (they can and likely should still be based on the tempest-lib rest
> client)
>
> -Matt Treinish
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] [Tempest] where fwaas tempest tests should be?

[Eichberger, German]

Hi,

Just moving the tests is not sufficient since the gate jobs also need
adjustment.

German

On 10/16/15, 1:50 AM, "Takashi Yamamoto"  wrote:

>hi,
>
>On Thu, Oct 15, 2015 at 11:17 PM, Matthew Treinish 
>wrote:
>> On Thu, Oct 15, 2015 at 08:25:40PM +0900, Takashi Yamamoto wrote:
>>> hi,
>>>
>>> i'm looking in fwaas tempest tests and have a question about code
>>>location.
>>>
>>> currently,
>>>
>>> - fwaas api tests and its rest client are in neutron repo
>>
>> This is a bad situation because it means that we're directly coupling
>>fwaas to
>> the neutron repo. Everything that depends on fwaas should be separate
>>and self
>> contained outside of the neutron repo. The longer that things are like
>>this it
>> will just lead to more headaches down the road.
>>
>>> - there are no fwaas scenario tests
>>>
>>> eventually,
>>>
>>> - fwaas api tests should be moved into neutron-fwaas repo
>>> - fwaaa scenario tests should be in neutron-fwaas repo too.
>>
>> This is definitely the right direction.
>
>if i move fwaas tests from neutron to neutron-fwaas, [1]
>is there easy way to run them together with the rest of neutron api tests
>for gate-neutron-dsvm-api job?
>
>[1] https://review.openstack.org/#/c/235790/
>
>>
>>> - the rest client will be in tempest-lib
>>
>> The discussion on which clients are in scope for tempest-lib hasn't
>>fully
>> happened yet. For right now we're taking a conservative approach and
>>the clients
>> can live in tempest-lib if there are tests in the tempest tree using
>>them.
>> (which is not fwaas) This might change eventually, (there will be a
>>summit
>> session on it) but for right now I'd say the fwaas clients should live
>>in the
>> fwaas repo. (they can and likely should still be based on the
>>tempest-lib rest
>> client)
>>
>> -Matt Treinish
>>
>> 
>>_
>>_
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: 
>>openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>__
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Neutron] [Tempest] where fwaas tempest tests should be?

[Takashi Yamamoto]

hi,

i'm looking in fwaas tempest tests and have a question about code location.

currently,

- fwaas api tests and its rest client are in neutron repo
- there are no fwaas scenario tests

eventually,

- fwaas api tests should be moved into neutron-fwaas repo
- fwaaa scenario tests should be in neutron-fwaas repo too.
- the rest client will be in tempest-lib

is it right?

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] [Tempest] where fwaas tempest tests should be?

[Assaf Muller]

On Thu, Oct 15, 2015 at 7:25 AM, Takashi Yamamoto 
wrote:

> hi,
>
> i'm looking in fwaas tempest tests and have a question about code location.
>
> currently,
>
> - fwaas api tests and its rest client are in neutron repo
> - there are no fwaas scenario tests
>
> eventually,
>
> - fwaas api tests should be moved into neutron-fwaas repo
> - fwaaa scenario tests should be in neutron-fwaas repo too.
>

I believe scenario tests that invoke APIs outside of Neutron should
stay (Or be introduced to) Tempest.


> - the rest client will be in tempest-lib
>
> is it right?
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] [Tempest] where fwaas tempest tests should be?

[Matthew Treinish]

On Thu, Oct 15, 2015 at 08:25:40PM +0900, Takashi Yamamoto wrote:
> hi,
> 
> i'm looking in fwaas tempest tests and have a question about code location.
> 
> currently,
> 
> - fwaas api tests and its rest client are in neutron repo

This is a bad situation because it means that we're directly coupling fwaas to
the neutron repo. Everything that depends on fwaas should be separate and self
contained outside of the neutron repo. The longer that things are like this it
will just lead to more headaches down the road.

> - there are no fwaas scenario tests
> 
> eventually,
> 
> - fwaas api tests should be moved into neutron-fwaas repo
> - fwaaa scenario tests should be in neutron-fwaas repo too.

This is definitely the right direction.

> - the rest client will be in tempest-lib

The discussion on which clients are in scope for tempest-lib hasn't fully
happened yet. For right now we're taking a conservative approach and the clients
can live in tempest-lib if there are tests in the tempest tree using them.
(which is not fwaas) This might change eventually, (there will be a summit
session on it) but for right now I'd say the fwaas clients should live in the
fwaas repo. (they can and likely should still be based on the tempest-lib rest
client)

-Matt Treinish


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] [Tempest] where fwaas tempest tests should be?

[Matthew Treinish]

On Thu, Oct 15, 2015 at 09:02:22AM -0400, Assaf Muller wrote:
> On Thu, Oct 15, 2015 at 7:25 AM, Takashi Yamamoto 
> wrote:
> 
> > hi,
> >
> > i'm looking in fwaas tempest tests and have a question about code location.
> >
> > currently,
> >
> > - fwaas api tests and its rest client are in neutron repo
> > - there are no fwaas scenario tests
> >
> > eventually,
> >
> > - fwaas api tests should be moved into neutron-fwaas repo
> > - fwaaa scenario tests should be in neutron-fwaas repo too.
> >
> 
> I believe scenario tests that invoke APIs outside of Neutron should
> stay (Or be introduced to) Tempest.

So testing the neutron advanced services was actually one of the first things
we decided was out of scope for tempest. (like well over a year ago) It took
some time to get equivalent testing setup elsewhere, but tests and support for
the advanced services were removed from tempest on purpose. I'd suggest that
you look at the tempest plugin interface:

http://docs.openstack.org/developer/tempest/plugin.html

if you'd like to make the fwaas tests (or any other adv. service tests)
integrate more cleanly with the rest of tempest.

-Matt Treinish


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] [Tempest] where fwaas tempest tests should be?

[Madhusudhan Kandadai]

On Thu, Oct 15, 2015 at 10:13 AM, Assaf Muller  wrote:

>
>
> On Thu, Oct 15, 2015 at 10:22 AM, Matthew Treinish 
> wrote:
>
>> On Thu, Oct 15, 2015 at 09:02:22AM -0400, Assaf Muller wrote:
>> > On Thu, Oct 15, 2015 at 7:25 AM, Takashi Yamamoto <
>> yamam...@midokura.com>
>> > wrote:
>> >
>> > > hi,
>> > >
>> > > i'm looking in fwaas tempest tests and have a question about code
>> location.
>> > >
>> > > currently,
>> > >
>> > > - fwaas api tests and its rest client are in neutron repo
>> > > - there are no fwaas scenario tests
>> > >
>> > > eventually,
>> > >
>> > > - fwaas api tests should be moved into neutron-fwaas repo
>> > > - fwaaa scenario tests should be in neutron-fwaas repo too.
>> > >
>> >
>> > I believe scenario tests that invoke APIs outside of Neutron should
>> > stay (Or be introduced to) Tempest.
>>
>> So testing the neutron advanced services was actually one of the first
>> things
>> we decided was out of scope for tempest. (like well over a year ago) It
>> took
>> some time to get equivalent testing setup elsewhere, but tests and
>> support for
>> the advanced services were removed from tempest on purpose.
>
>
> This is for both *aaS API tests and scenario tests?
>

I think yes. Scenario tests for LBaaS and VPNaaS are at this path:
LBaaS:
https://github.com/openstack/neutron-lbaas/tree/master/neutron_lbaas/tests/tempest/v2/scenario
VPNaaS: https://github.com/openstack/neutron-vpnaas/blob/master/rally-jobs/
Moving VPN api jobs in VPNaaS tree: https://review.openstack.org/#/c/211381/
(WIP)

>
>
>> I'd suggest that
>> you look at the tempest plugin interface:
>>
>> http://docs.openstack.org/developer/tempest/plugin.html
>>
>> if you'd like to make the fwaas tests (or any other adv. service tests)
>> integrate more cleanly with the rest of tempest.
>>
>> -Matt Treinish
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] [Tempest] where fwaas tempest tests should be?

[Assaf Muller]

On Thu, Oct 15, 2015 at 10:22 AM, Matthew Treinish 
wrote:

> On Thu, Oct 15, 2015 at 09:02:22AM -0400, Assaf Muller wrote:
> > On Thu, Oct 15, 2015 at 7:25 AM, Takashi Yamamoto  >
> > wrote:
> >
> > > hi,
> > >
> > > i'm looking in fwaas tempest tests and have a question about code
> location.
> > >
> > > currently,
> > >
> > > - fwaas api tests and its rest client are in neutron repo
> > > - there are no fwaas scenario tests
> > >
> > > eventually,
> > >
> > > - fwaas api tests should be moved into neutron-fwaas repo
> > > - fwaaa scenario tests should be in neutron-fwaas repo too.
> > >
> >
> > I believe scenario tests that invoke APIs outside of Neutron should
> > stay (Or be introduced to) Tempest.
>
> So testing the neutron advanced services was actually one of the first
> things
> we decided was out of scope for tempest. (like well over a year ago) It
> took
> some time to get equivalent testing setup elsewhere, but tests and support
> for
> the advanced services were removed from tempest on purpose.


This is for both *aaS API tests and scenario tests?


> I'd suggest that
> you look at the tempest plugin interface:
>
> http://docs.openstack.org/developer/tempest/plugin.html
>
> if you'd like to make the fwaas tests (or any other adv. service tests)
> integrate more cleanly with the rest of tempest.
>
> -Matt Treinish
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev