[openstack-dev] [qa] resigning from Tempest core

[Koderer, Marc]

Hi folks,

as already mentioned during the current code sprint:
I am currently lacking time for reviews and code contributions.

I think it better to step back and let other’s in that are more active.

Thanks for all the support and it was really fun the past 3 years working with
you!

Regards
Marc


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron][ml2][Manila] API to query segments used during port binding

[Koderer, Marc]

Hi folks,

I had a deep dive session with Bob (thx Bob).

We have a plan to solve the issue without any change of APIs or 
manila driver reworks.

The process will look like the following:

1.) In case of multi-segment/hpb Manila creates a port like Ironic ML2
would do it [1]:
  vif_type = baremetal
  binding_profile = list of sw ports
  device_owner = manila:ID

2.) Manila waits until all ports are actively bound
2.a) Neutron binds the port through the segments
2.b) A manila-neutron mech driver (a really simple one) fulfils the binding
   and sets:
 vif_details = {“share_segmentation_id" = XX,
   “share_network_type" = YY}

3.) When the port becomes active Manila has all it needs to proceed

In 2.b. the manila MD will only search for device_owner == “manila:”,
sets the needed details and fulfils the binding (~10 LOC).

We also discussed about using ML2 GW [2] or trunk port [3].
I consider the design above as simple enough to get merged smoothly
in Newton.

@Ben: Will be back for bug hunting now.

Regards
Marc

[1]: 
https://specs.openstack.org/openstack/ironic-specs/specs/not-implemented/ironic-ml2-integration.html
 
<https://specs.openstack.org/openstack/ironic-specs/specs/not-implemented/ironic-ml2-integration.html>
[2]: https://wiki.openstack.org/wiki/Neutron/L2-GW
[3]: https://wiki.openstack.org/wiki/Neutron/TrunkPort


> On 09 Mar 2016, at 09:25, Koderer, Marc  wrote:
> 
>> 
>> On 09 Mar 2016, at 08:43, Sukhdev Kapur > <mailto:sukhdevka...@gmail.com>> wrote:
>> 
>> Hi Marc, 
>> 
>> I am driving the ironic-ml2 integration and introduced baremetal type for 
>> vmic_type. 
> 
> Basically that’s my plan. So in my current implementation
> I use the baremetal vnic_type [1] and add a binding profile [2].
> 
>> You can very much use the same integration here - however, I am not 
>> completely clear about your use case. 
>> Do you want neutron/ML2 to plumb the network for Manila or do you want to 
>> find out what VLAN (segmentation id) is used on the segment which connects 
>> TOR to the storage device? 
> 
> Generally I want to find the best architecture for this feature :)
> Introducing a neutron-agent that does the plumbing will mean this agent needs
> to have a connection to the storage node itself. So we will end-up in a
> storage agent with a driver model (or an agent for each storage device). On
> the other side it follows the idea that neutron takes care about networking.
> 
> From a Manila perspective the easiest solution would be to have an interface 
> to
> get the segmentation id of the lowest bound segment.
> 
>> 
>> You had this on the agenda of ML2 meeting for tomorrow and I was going to 
>> discuss this with you in the meeting. But, I noticed that you removed it 
>> from the agenda. Do you have what you need? If not, you may want to join us 
>> in the ML2 meeting tomorrow and we can discuss this use case there. 
> 
> Yeah I am sorry - I have to move the topic +1 week due to an internal meeting 
> :(
> But we can have a chat on IRC (mkoderer on freenode).
> 
> Regards
> Marc
> 
> [1]: https://review.openstack.org/#/c/283494/ 
> <https://review.openstack.org/#/c/283494/>
> [2]: https://review.openstack.org/#/c/284034/ 
> <https://review.openstack.org/#/c/284034/>
> 
> 
>> 
>> -Sukhdev
>> 
>> 
>> On Tue, Mar 1, 2016 at 1:08 AM, Koderer, Marc > <mailto:m...@koderer.com>> wrote:
>> 
>>> On 01 Mar 2016, at 06:22, Kevin Benton >> <mailto:ke...@benton.pub>> wrote:
>>> 
>>> >This seems gross and backwards. It makes sense as a short term hack but 
>>> >given that we have time to design this correctly I'd prefer to get this 
>>> >information in a more straighforward way.
>>> 
>>> Well it depends on what is happening here. If Manilla is wiring up a 
>>> specific VLAN for a port, that makes it part of the port binding process, 
>>> in which case it should be an ML2 driver. Can you provide some more details 
>>> about what Manilla is doing with this info?
>> 
>> The VLAN segment ID and IP address is used in the share driver to configure 
>> the
>> corresponding interface resources within the storage. Just to give some
>> examples:
>> 
>>  - NetApp driver uses it to create a logical interface and assign it to a
>>“storage virtual machine” [1]
>>  - EMC driver does it in similar manner [2]
>> 
>> My idea was to use the same principle as ironic ml2 intregration is doing [3]
>> by setting the vnic_type to “baremetal”.
>> 
>> In Manila's current implementation storage driv

Re: [openstack-dev] [Neutron][ml2][Manila] API to query segments used during port binding

[Koderer, Marc]

> On 09 Mar 2016, at 08:43, Sukhdev Kapur  wrote:
> 
> Hi Marc, 
> 
> I am driving the ironic-ml2 integration and introduced baremetal type for 
> vmic_type. 

Basically that’s my plan. So in my current implementation
I use the baremetal vnic_type [1] and add a binding profile [2].

> You can very much use the same integration here - however, I am not 
> completely clear about your use case. 
> Do you want neutron/ML2 to plumb the network for Manila or do you want to 
> find out what VLAN (segmentation id) is used on the segment which connects 
> TOR to the storage device? 

Generally I want to find the best architecture for this feature :)
Introducing a neutron-agent that does the plumbing will mean this agent needs
to have a connection to the storage node itself. So we will end-up in a
storage agent with a driver model (or an agent for each storage device). On
the other side it follows the idea that neutron takes care about networking.

From a Manila perspective the easiest solution would be to have an interface to
get the segmentation id of the lowest bound segment.

> 
> You had this on the agenda of ML2 meeting for tomorrow and I was going to 
> discuss this with you in the meeting. But, I noticed that you removed it from 
> the agenda. Do you have what you need? If not, you may want to join us in the 
> ML2 meeting tomorrow and we can discuss this use case there. 

Yeah I am sorry - I have to move the topic +1 week due to an internal meeting :(
But we can have a chat on IRC (mkoderer on freenode).

Regards
Marc

[1]: https://review.openstack.org/#/c/283494/ 
<https://review.openstack.org/#/c/283494/>
[2]: https://review.openstack.org/#/c/284034/ 
<https://review.openstack.org/#/c/284034/>


> 
> -Sukhdev
> 
> 
> On Tue, Mar 1, 2016 at 1:08 AM, Koderer, Marc  <mailto:m...@koderer.com>> wrote:
> 
>> On 01 Mar 2016, at 06:22, Kevin Benton > <mailto:ke...@benton.pub>> wrote:
>> 
>> >This seems gross and backwards. It makes sense as a short term hack but 
>> >given that we have time to design this correctly I'd prefer to get this 
>> >information in a more straighforward way.
>> 
>> Well it depends on what is happening here. If Manilla is wiring up a 
>> specific VLAN for a port, that makes it part of the port binding process, in 
>> which case it should be an ML2 driver. Can you provide some more details 
>> about what Manilla is doing with this info?
> 
> The VLAN segment ID and IP address is used in the share driver to configure 
> the
> corresponding interface resources within the storage. Just to give some
> examples:
> 
>  - NetApp driver uses it to create a logical interface and assign it to a
>“storage virtual machine” [1]
>  - EMC driver does it in similar manner [2]
> 
> My idea was to use the same principle as ironic ml2 intregration is doing [3]
> by setting the vnic_type to “baremetal”.
> 
> In Manila's current implementation storage drivers are also responsible to
> setup the right networking setup. Would you suggest to move this part into the
> port binding phase?
> 
> Regards
> Marc
> 
> 
> [1]: 
> https://github.com/openstack/manila/blob/master/manila/share/drivers/netapp/dataontap/cluster_mode/lib_multi_svm.py#L272
>  
> <https://github.com/openstack/manila/blob/master/manila/share/drivers/netapp/dataontap/cluster_mode/lib_multi_svm.py#L272>
> [2]: 
> https://github.com/openstack/manila/blob/master/manila/share/drivers/emc/plugins/vnx/connection.py#L609
>  
> <https://github.com/openstack/manila/blob/master/manila/share/drivers/emc/plugins/vnx/connection.py#L609>
> [3]: 
> https://specs.openstack.org/openstack/ironic-specs/specs/not-implemented/ironic-ml2-integration.html
>  
> <https://specs.openstack.org/openstack/ironic-specs/specs/not-implemented/ironic-ml2-integration.html>
> 
> 
>> 
>> On Mon, Feb 29, 2016 at 5:29 PM, Ben Swartzlander > <mailto:b...@swartzlander.org>> wrote:
>> On 02/29/2016 04:38 PM, Kevin Benton wrote:
>> You're correct. Right now there is no way via the HTTP API to find which
>> segments a port is bound to.
>> This is something we can certainly consider adding, but it will need an
>> RFE so it wouldn't land until Newton at the earliest.
>> 
>> I believe Newton is the target for this work. This is feature freeze week 
>> after all.
>> 
>> Have you considered writing an ML2 driver that just notifies Manilla of
>> the port's segment info? All of this information is available to ML2
>> drivers in the PortContext object that is passed to them.
>> 
>> This seems gross and backwards. It makes sense as a short term hack but 
>> given that w

Re: [openstack-dev] [Neutron][ml2][Manila] API to query segments used during port binding

[Koderer, Marc]

> On 01 Mar 2016, at 06:22, Kevin Benton  wrote:
> 
> >This seems gross and backwards. It makes sense as a short term hack but 
> >given that we have time to design this correctly I'd prefer to get this 
> >information in a more straighforward way.
> 
> Well it depends on what is happening here. If Manilla is wiring up a specific 
> VLAN for a port, that makes it part of the port binding process, in which 
> case it should be an ML2 driver. Can you provide some more details about what 
> Manilla is doing with this info?

The VLAN segment ID and IP address is used in the share driver to configure the
corresponding interface resources within the storage. Just to give some
examples:

 - NetApp driver uses it to create a logical interface and assign it to a
   “storage virtual machine” [1]
 - EMC driver does it in similar manner [2]

My idea was to use the same principle as ironic ml2 intregration is doing [3]
by setting the vnic_type to “baremetal”.

In Manila's current implementation storage drivers are also responsible to
setup the right networking setup. Would you suggest to move this part into the
port binding phase?

Regards
Marc


[1]: 
https://github.com/openstack/manila/blob/master/manila/share/drivers/netapp/dataontap/cluster_mode/lib_multi_svm.py#L272
 

[2]: 
https://github.com/openstack/manila/blob/master/manila/share/drivers/emc/plugins/vnx/connection.py#L609
 

[3]: 
https://specs.openstack.org/openstack/ironic-specs/specs/not-implemented/ironic-ml2-integration.html
 



> 
> On Mon, Feb 29, 2016 at 5:29 PM, Ben Swartzlander  > wrote:
> On 02/29/2016 04:38 PM, Kevin Benton wrote:
> You're correct. Right now there is no way via the HTTP API to find which
> segments a port is bound to.
> This is something we can certainly consider adding, but it will need an
> RFE so it wouldn't land until Newton at the earliest.
> 
> I believe Newton is the target for this work. This is feature freeze week 
> after all.
> 
> Have you considered writing an ML2 driver that just notifies Manilla of
> the port's segment info? All of this information is available to ML2
> drivers in the PortContext object that is passed to them.
> 
> This seems gross and backwards. It makes sense as a short term hack but given 
> that we have time to design this correctly I'd prefer to get this information 
> in a more straighforward way.
> 
> -Ben Swartzlander
> 
> 
> On Mon, Feb 29, 2016 at 6:48 AM, Ihar Hrachyshka  
> >> wrote:
> 
> Fixed neutron tag in the subject.
> 
> Marc mailto:m...@koderer.com>  >> wrote:
> 
> Hi Neutron team,
> 
> I am currently working on a feature for hierarchical port
> binding support in
> Manila [1] [2]. Just to give some context: In the current
> implementation Manila
> creates a neutron port but let it unbound (state DOWN).
> Therefore Manila uses
> the port create only retrieve an IP address and segmentation ID
> (some drivers
> only support VLAN here).
> 
> My idea is to change this behavior and do an actual port binding
> action so that
> the configuration of VLAN isn’t a manual job any longer. And
> that multi-segment
> and HPB is supported on the long-run.
> 
> My current issue is: How can Manila retrieve the segment
> information for a
> bound port? Manila only is interested in the last (bottom)
> segmentation ID
> since I assume the storage is connected to a ToR switch.
> 
> Database-wise it’s possible to query it using
> ml2_port_binding_levels table.
> But AFAIK there is no API to query this. The only information
> that is exposed
> are all segments of a network. But this is not sufficient to
> identify which
> segments actually used for a port binding.
> 
> Regards
> Marc
> SAP SE
> 
> [1]:
> 
> https://wiki.openstack.org/wiki/Manila/design/manila-newton-hpb-support 
> 
> [2]: https://review.openstack.org/#/c/277731/ 
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe 
> 

[openstack-dev] [Neuton][ml2][Manila] API to query segments used during port binding

[Koderer, Marc]

Hi Neutron team,

I am currently working on a feature for hierarchical port binding support in
Manila [1] [2]. Just to give some context: In the current implementation Manila
creates a neutron port but let it unbound (state DOWN). Therefore Manila uses
the port create only retrieve an IP address and segmentation ID (some drivers
only support VLAN here).

My idea is to change this behavior and do an actual port binding action so that
the configuration of VLAN isn’t a manual job any longer. And that multi-segment
and HPB is supported on the long-run.

My current issue is: How can Manila retrieve the segment information for a
bound port? Manila only is interested in the last (bottom) segmentation ID
since I assume the storage is connected to a ToR switch.

Database-wise it’s possible to query it using ml2_port_binding_levels table.
But AFAIK there is no API to query this. The only information that is exposed
are all segments of a network. But this is not sufficient to identify which
segments actually used for a port binding.

Regards
Marc
SAP SE

[1]: https://wiki.openstack.org/wiki/Manila/design/manila-newton-hpb-support 

[2]: https://review.openstack.org/#/c/277731/ 
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [OpenStack-Dev][Manila] BP https://blueprints.launchpad.net/manila/+spec/access-groups

[Koderer, Marc]

Hi  Nidhi,

It’s might also a good idea to have a short discussion during the manila 
meeting [1].
You can simply add your topic to the agenda.

Regards
Marc

[1]: https://wiki.openstack.org/wiki/Manila/Meetings 


> On 26 Feb 2016, at 10:52, nidhi.h...@wipro.com wrote:
> 
> Hi Manila Team,
>  
> I am working on
> https://blueprints.launchpad.net/manila/+spec/access-groups 
> 
>  
> For this I have created initial document as attached with the mail.
> It contains DB CLI REST API related changes.
>  
> Could you please have a look and share your opinion.
>  
> Kindly let me know, if there is some understanding gap, 
> or something I have missed to document or 
> share your comments in general to make it better.
>  
> Thank you.
> Nidhi Mittal Hada
> Architect | PES / COE – Kolkata India
> Wipro Limited
> M +91 74 3910 9883 | O +91 33 3095 4767 | VOIP +91 33 3095 4767
>  
>  
>  
> The information contained in this electronic message and any attachments to 
> this message are intended for the exclusive use of the addressee(s) and may 
> contain proprietary, confidential or privileged information. If you are not 
> the intended recipient, you should not disseminate, distribute or copy this 
> e-mail. Please notify the sender immediately and destroy all copies of this 
> message and any attachments. WARNING: Computer viruses can be transmitted via 
> email. The recipient should check this email and any attachments for the 
> presence of viruses. The company accepts no liability for any damage caused 
> by any virus transmitted by this email. www.wipro.com 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org 
> ?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev 
> 
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [qa] deprecating Tempest stress framework

[Koderer, Marc]

I know that there are folks around that are using it.
+1 to move it to a separate repo.

Regards
Marc

> On 11 Feb 2016, at 13:59, Daniel Mellado  wrote:
> 
> +1 to that, it was my 2nd to-be-deprecated after javelin ;)
> 
> El 11/02/16 a las 12:47, Sean Dague escribió:
>> In order to keep Tempest healthy I feel like it's time to prune things
>> that are outside of the core mission, especially when there are other
>> options out there.
>> 
>> The stress test framework in tempest is one of those. It builds on other
>> things in Tempest, but isn't core to it.
>> 
>> I'd propose that becomes deprecated now, and removed in Newton. If there
>> are folks that would like to carry it on from there, I think we should
>> spin it into a dedicated repository and just have it require tempest.
>> 
>>  -Sean
>> 
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [QA] Proposed Changes to Tempest Core

[Koderer, Marc]

+1

> -Ursprüngliche Nachricht-
> Von: Matthew Treinish [mailto:mtrein...@kortar.org]
> Gesendet: Dienstag, 22. Juli 2014 00:34
> An: openstack-dev@lists.openstack.org
> Betreff: [openstack-dev] [QA] Proposed Changes to Tempest Core
> 
> 
> Hi Everyone,
> 
> I would like to propose 2 changes to the Tempest core team:
> 
> First, I'd like to nominate Andrea Fritolli to the Tempest core team. Over
> the past cycle Andrea has been steadily become more actively engaged in the
> Tempest community. Besides his code contributions around refactoring
> Tempest's authentication and credentials code, he has been providing reviews
> that have been of consistently high quality that show insight into both the
> project internals and it's future direction. In addition he has been active
> in the qa-specs repo both providing reviews and spec proposals, which has
> been very helpful as we've been adjusting to using the new process. Keeping
> in mind that becoming a member of the core team is about earning the trust
> from the members of the current core team through communication and quality
> reviews, not simply a matter of review numbers, I feel that Andrea will make
> an excellent addition to the team.
> 
> As per the usual, if the current Tempest core team members would please vote
> +1 or -1(veto) to the nomination when you get a chance. We'll keep the polls
> open for 5 days or until everyone has voted.
> 
> References:
> 
> https://review.openstack.org/#/q/reviewer:%22Andrea+Frittoli+%22,n,z
> 
> http://stackalytics.com/?user_id=andrea-frittoli&metric=marks&module=qa-
> group
> 
> 
> The second change that I'm proposing today is to remove Giulio Fidente from
> the core team. He asked to be removed from the core team a few weeks back
> because he is no longer able to dedicate the required time to Tempest
> reviews. So if there are no objections to this I will remove him from the
> core team in a few days.
> Sorry to see you leave the team Giulio...
> 
> 
> Thanks,
> 
> Matt Treinish

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [QA][Infra] Mid-Cycle Meet-up

[Koderer, Marc]

Hi all,

just some general information for your trip:

 - Darmstadt is quite close to Frankfurt am Main:
>From Frankfurt Airport you can take a bus shuttle that takes you directly to 
>the office campus (the trip takes around 25 minutes depending on your arrival 
>gate):
http://www.rmv.de/linkableblob/de/33548-59802/data/AirLiner_Direktbus_Darmstadt_Frankfurt_Flughafen_PDF.pdf

- If your trip gets approved please send me your name, email and your company 
name in order to create badges for you in advance

- I will create a wiki page with more details (like hotel information) and send 
it around

Regards
Marc

> -Ursprüngliche Nachricht-
> Von: Matthew Treinish [mailto:mtrein...@kortar.org]
> Gesendet: Donnerstag, 29. Mai 2014 18:07
> An: openstack-dev@lists.openstack.org; openstack-in...@lists.openstack.org
> Betreff: [openstack-dev] [QA][Infra] Mid-Cycle Meet-up
> 
> 
> Hi Everyone,
> 
> So we'd like to announce to everyone that we're going to be doing a
> combined Infra and QA program mid-cycle meet-up. It will be the week of
> July 14th in Darmstadt, Germany at Deutsche Telekom who has graciously
> offered to sponsor the event. The plan is to use the week as both a time
> for face to face collaboration for both programs respectively as well as
> having a couple days of bootstrapping for new users/contributors. The
> intent was that this would be useful for people who are interested in
> contributing to either Infra or QA, and those who are running third party
> CI systems.
> 
> The current break down for the week that we're looking at is:
> 
> July 14th: Infra
> July 15th: Infra
> July 16th: Bootstrapping for new users
> July 17th: More bootstrapping
> July 18th: QA
> 
> We still have to work out more details, and will follow up once we have
> them.
> But, we thought it would be better to announce the event earlier so people
> can start to plan travel if they need it.
> 
> 
> Thanks,
> 
> Matt Treinish
> Jim Blair

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [qa] QA Summit Meet-up Atlanta

[Koderer, Marc]

Hi,

7:30pm at Ted's Montana Grill. I called them and they put us on the list.
Again the location: 
https://plus.google.com/100773563660993493024/posts/P9YSgT8AVXh

It's quite near to all the hotels.

See you there!
Marc 
____
From: Koderer, Marc [m.kode...@telekom.de]
Sent: Wednesday, May 07, 2014 7:33 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [qa] QA Summit Meet-up Atlanta

Hi folks,

ok seems to me more complex than expected ;)
Monday is the OpenStack speakers dinner.

Let's do it on Sunday. Matthew already suggested a location:
 Ted's Montana Grill
 https://plus.google.com/100773563660993493024/posts/P9YSgT8AVXh

For all the people interested just send me your mobile number..

Regards
Marc

________
From: Koderer, Marc [m.kode...@telekom.de]
Sent: Monday, May 05, 2014 8:41 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [qa] QA Summit Meet-up Atlanta

All right, let’s meet on Monday ;)
We can discuss the details after the QA meeting this week.

Von: Frittoli, Andrea (HP Cloud) [mailto:fritt...@hp.com]
Gesendet: Donnerstag, 1. Mai 2014 18:42
An: OpenStack Development Mailing List (not for usage questions)
Betreff: Re: [openstack-dev] [qa] QA Summit Meet-up Atlanta

I will arrive Sunday late.
If you meet on Monday I’ll see you there ^_^

From: Miguel Lavalle [mailto:mig...@mlavalle.com]
Sent: 01 May 2014 17:28
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [qa] QA Summit Meet-up Atlanta

I arrive Sunday at 3:30pm. Either Sunday or Monday are fine with me. Lookging 
forward to it :-)

On Wed, Apr 30, 2014 at 5:11 AM, Koderer, Marc 
mailto:m.kode...@telekom.de>> wrote:
Hi folks,

last time we met one day before the Summit started for a short meet-up.
Should we do the same this time?

I will arrive Saturday to recover from the jet lag ;) So Sunday 11th would be 
fine for me.

Regards,
Marc
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org<mailto:OpenStack-dev@lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [qa] QA Summit Meet-up Atlanta

[Koderer, Marc]

Hi folks,

ok seems to me more complex than expected ;)
Monday is the OpenStack speakers dinner.

Let's do it on Sunday. Matthew already suggested a location: 
 Ted's Montana Grill
 https://plus.google.com/100773563660993493024/posts/P9YSgT8AVXh

For all the people interested just send me your mobile number..

Regards
Marc

____
From: Koderer, Marc [m.kode...@telekom.de]
Sent: Monday, May 05, 2014 8:41 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [qa] QA Summit Meet-up Atlanta

All right, let’s meet on Monday ;)
We can discuss the details after the QA meeting this week.

Von: Frittoli, Andrea (HP Cloud) [mailto:fritt...@hp.com]
Gesendet: Donnerstag, 1. Mai 2014 18:42
An: OpenStack Development Mailing List (not for usage questions)
Betreff: Re: [openstack-dev] [qa] QA Summit Meet-up Atlanta

I will arrive Sunday late.
If you meet on Monday I’ll see you there ^_^

From: Miguel Lavalle [mailto:mig...@mlavalle.com]
Sent: 01 May 2014 17:28
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [qa] QA Summit Meet-up Atlanta

I arrive Sunday at 3:30pm. Either Sunday or Monday are fine with me. Lookging 
forward to it :-)

On Wed, Apr 30, 2014 at 5:11 AM, Koderer, Marc 
mailto:m.kode...@telekom.de>> wrote:
Hi folks,

last time we met one day before the Summit started for a short meet-up.
Should we do the same this time?

I will arrive Saturday to recover from the jet lag ;) So Sunday 11th would be 
fine for me.

Regards,
Marc
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org<mailto:OpenStack-dev@lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [qa] QA Summit Meet-up Atlanta

[Koderer, Marc]

All right, let’s meet on Monday ;)
We can discuss the details after the QA meeting this week.

Von: Frittoli, Andrea (HP Cloud) [mailto:fritt...@hp.com]
Gesendet: Donnerstag, 1. Mai 2014 18:42
An: OpenStack Development Mailing List (not for usage questions)
Betreff: Re: [openstack-dev] [qa] QA Summit Meet-up Atlanta

I will arrive Sunday late.
If you meet on Monday I’ll see you there ^_^

From: Miguel Lavalle [mailto:mig...@mlavalle.com]
Sent: 01 May 2014 17:28
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [qa] QA Summit Meet-up Atlanta

I arrive Sunday at 3:30pm. Either Sunday or Monday are fine with me. Lookging 
forward to it :-)

On Wed, Apr 30, 2014 at 5:11 AM, Koderer, Marc 
mailto:m.kode...@telekom.de>> wrote:
Hi folks,

last time we met one day before the Summit started for a short meet-up.
Should we do the same this time?

I will arrive Saturday to recover from the jet lag ;) So Sunday 11th would be 
fine for me.

Regards,
Marc
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org<mailto:OpenStack-dev@lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [qa] QA Summit Meet-up Atlanta

[Koderer, Marc]

Hi folks,

last time we met one day before the Summit started for a short meet-up.
Should we do the same this time? 

I will arrive Saturday to recover from the jet lag ;) So Sunday 11th would be 
fine for me.

Regards,
Marc
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [qa] API schema location

[Koderer, Marc]

> From: Matthew Treinish [mtrein...@kortar.org]
> Sent: Tuesday, March 18, 2014 7:08 PM
> On Tue, Mar 18, 2014 at 12:34:57PM +0100, Koderer, Marc wrote:
> > On Tue, 18 Marc 2014 12:00:00 +0100
> > Christopher Yeoh [mailto:cbky...@gmail.com] wrote:
> > > On Tue, 18 Mar 2014 10:39:19 +0100
> > > "Koderer, Marc"  wrote:
> > > >
> > > > I just recognized that we have very similar interface definitions in
> > > > tempest/api_schema and etc/schema:
> > > >
> > > > https://github.com/openstack/tempest/tree/master/etc/schemas
> > > > https://github.com/openstack/tempest/tree/master/tempest/api_schema
> > > >
> > > > Any objections if I move them to a single location? I'd prefer to use
> > > > json as file format instead of .py. As final goal we should find a way
> > > > how to merge them competently but I feel like this is something for
> > > > the design summit ;)
> > > >
> > >
> > > Heh we just moved them but I'm open to other suggestions - they are are
> > > specific to API testing though aren't they? Long term the idea is that
> > > they should be generated by Nova rather than tempest.  I think to prevent
> > > unintentional changes we'd probably cache a copy in tempest though rather
> > > than dynamically query them.
> 
> The idea was never to dynamically query them; there should always be a copy in
> the tempest tree. Like you said to prevent unintentional changes which is the
> same reason we don't auto-discover api versions. The idea for querying nova to
> get the schemas was to enable a tool which could populate the schemas
> automatically so that we didn't have to manually generate them individually. 
> I'd
> say, to a certain extent, that this new round of validation patches could use
> the same kind of tool.
> 
> >
> > Sry that I didn't recognized this review.
> > Both definitions are coupled to API testing, yes.
> >
> > >
> > > My feeling at the moment is that they should  .py files.
> > > Because I think there's cases where we will want to have some schema
> > > definitions based on others or share common parts and use bits of python
> > > code to achieve this. For example availability zone list and detailed
> > > listing  have a lot in common (detailed listing just has a more
> > > parameters). I think there'll be similar cases for v2 and v3 versions as
> > > well.  While we're still manually generating them and keeping them up to
> > > date I think it's worth sharing as much as we can.
> >
> > Ok understood. We just converted the negative testing
> > definitions to json files due to review findings..
> 
> Well, when I left the review comment about it being a json file, I didn't 
> think
> about inheritance. Chris has a good point about reusing common bits and just
> extending it. That wasn't how you proposed the negative test schemas would be
> used which is why I suggested using a raw json file.
> > It's just very confusing for new people if they see
> > two separate folders with schema definitions.
> >
> > But unfortunately there isn't an easy way.
> 
> Am I missing something or are these schemas being added now just a subset of
> what is being used for negative testing? Why can't we either add the extra
> negative test info around the new test validation patches and get the double
> benefit. Or have the negative test schemas just extend these new schemas being
> added?

Yes, the api_schema files should theoretically be a
subsets of the negative test schemas.
But I don't think that extending them will be possible:

if you have a property definition like this:

"properties": {
"minRam": {  "type": "integer",}

how can you extend it to:

"properties": {
"minRam": {
"type": "integer",
"results": {
"gen_none": 400,
"gen_string": 400
}

This is the reason why I am unsure how inheritance can solve something here.

> >
> > >
> > > I agree there's a lot of commonality and we should long term just have one
> > > schema definition. There's quite a bit to discuss (eg level of strictness
> > > is currently different) in this area and a summit session about it would
> > > be very useful.
> > >
> >
> > +1
> >
> 
> I agree there is probably enough here to discuss during a summit session on
> where schema validation fits into tempest. As a part of that discussing how to
> store and manage schema definitions for both the negative test framework and
> validation tests.
> 
> 
> -Matt Treinish
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [qa] API schema location

[Koderer, Marc]

On Tue, 18 Marc 2014 12:00:00 +0100
Christopher Yeoh [mailto:cbky...@gmail.com] wrote:
> On Tue, 18 Mar 2014 10:39:19 +0100
> "Koderer, Marc"  wrote:
> >
> > I just recognized that we have very similar interface definitions in
> > tempest/api_schema and etc/schema:
> >
> > https://github.com/openstack/tempest/tree/master/etc/schemas
> > https://github.com/openstack/tempest/tree/master/tempest/api_schema
> >
> > Any objections if I move them to a single location? I'd prefer to use
> > json as file format instead of .py. As final goal we should find a way
> > how to merge them competently but I feel like this is something for
> > the design summit ;)
> >
> 
> Heh we just moved them but I'm open to other suggestions - they are are
> specific to API testing though aren't they? Long term the idea is that
> they should be generated by Nova rather than tempest.  I think to prevent
> unintentional changes we'd probably cache a copy in tempest though rather
> than dynamically query them.

Sry that I didn't recognized this review.
Both definitions are coupled to API testing, yes.

> 
> My feeling at the moment is that they should  .py files.
> Because I think there's cases where we will want to have some schema
> definitions based on others or share common parts and use bits of python
> code to achieve this. For example availability zone list and detailed
> listing  have a lot in common (detailed listing just has a more
> parameters). I think there'll be similar cases for v2 and v3 versions as
> well.  While we're still manually generating them and keeping them up to
> date I think it's worth sharing as much as we can.

Ok understood. We just converted the negative testing
definitions to json files due to review findings..
It's just very confusing for new people if they see
two separate folders with schema definitions.

But unfortunately there isn't an easy way.

> 
> I agree there's a lot of commonality and we should long term just have one
> schema definition. There's quite a bit to discuss (eg level of strictness
> is currently different) in this area and a summit session about it would
> be very useful.
> 

+1

> Regards,
> 
> Chris
> 
> > Regards,
> > Marc
> >
> >
> > DEUTSCHE TELEKOM AG
> > Digital Business Unit, Cloud Services (P&I) Marc Koderer Cloud
> > Technology Software Developer T-Online-Allee 1, 64211 Darmstadt
> > E-Mail: m.kode...@telekom.de
> > www.telekom.com
> >
> > LIFE IS FOR SHARING.
> >
> > DEUTSCHE TELEKOM AG
> > Supervisory Board: Prof. Dr. Ulrich Lehner (Chairman) Board of
> > Management: René Obermann (Chairman), Reinhard Clemens, Niek Jan van
> > Damme, Timotheus Höttges, Dr. Thomas Kremer, Claudia Nemat, Prof. Dr.
> > Marion Schick Commercial register: Amtsgericht Bonn HRB 6794
> > Registered office: Bonn
> >
> > BIG CHANGES START SMALL – CONSERVE RESOURCES BY NOT PRINTING EVERY
> > E-MAIL.
> >
> > ___
> > OpenStack-dev mailing list
> > OpenStack-dev@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [qa] API schema location

[Koderer, Marc]

Hi Chris, hi all,

I just recognized that we have very similar interface definitions in
tempest/api_schema and etc/schema:

https://github.com/openstack/tempest/tree/master/etc/schemas
https://github.com/openstack/tempest/tree/master/tempest/api_schema

Any objections if I move them to a single location? I'd prefer to use json as
file format instead of .py. As final goal we should find a way how to merge
them competently but I feel like this is something for the design summit ;)

Regards,
Marc


DEUTSCHE TELEKOM AG
Digital Business Unit, Cloud Services (P&I)
Marc Koderer
Cloud Technology Software Developer
T-Online-Allee 1, 64211 Darmstadt
E-Mail: m.kode...@telekom.de
www.telekom.com   

LIFE IS FOR SHARING. 

DEUTSCHE TELEKOM AG
Supervisory Board: Prof. Dr. Ulrich Lehner (Chairman)
Board of Management: René Obermann (Chairman),
Reinhard Clemens, Niek Jan van Damme, Timotheus Höttges,
Dr. Thomas Kremer, Claudia Nemat, Prof. Dr. Marion Schick
Commercial register: Amtsgericht Bonn HRB 6794
Registered office: Bonn

BIG CHANGES START SMALL – CONSERVE RESOURCES BY NOT PRINTING EVERY E-MAIL.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [qa] [Infra] pep8 issues in tempest gate / testscenarios lib

[Koderer, Marc]

Hi folks,

I can't make it to the QA meeting for today so I wanted to summarize the issue
that we have with the pep8 and tempest gate. An example for the issue you can
find here:
  https://review.openstack.org/#/c/79256/ 
  
http://logs.openstack.org/56/79256/1/gate/gate-tempest-pep8/088cc12/console.html

pep8 check shows an error but the check itself is marked as success.

For me this show two issues. First flake8 should return with an exit code !=0.
I will have a closer look into hacking and what went wrong here.

Second issue is the current implementation of the negative testing framework:
we are using the testscenarios lib with the "load_tests" variable interpreted
by the test runner. This forces us to build the scenario at import time and if
we want to have tempest configurations for this (like introduced in
https://review.openstack.org/#/c/73982/) the laziness for the config doesn't
work.

Although it seems like if I remove the inheritance of the xml class to the
json class 
(https://github.com/openstack/tempest/blob/master/tempest/api/compute/admin/test_flavors_negative_xml.py#L24)
that error doesn't appear any longer, I see a general problem with
the usage of "import-time" code and we may think about a better solution in 
general.

I'll try to address the missing pieces tomorrow.
Bug: https://bugs.launchpad.net/tempest/+bug/1291826

Regards,
Marc

DEUTSCHE TELEKOM AG
Digital Business Unit, Cloud Services (P&I)
Marc Koderer
Cloud Technology Software Developer
T-Online-Allee 1, 64211 Darmstadt
E-Mail: m.kode...@telekom.de
www.telekom.com   

LIFE IS FOR SHARING. 

DEUTSCHE TELEKOM AG
Supervisory Board: Prof. Dr. Ulrich Lehner (Chairman)
Board of Management: René Obermann (Chairman),
Reinhard Clemens, Niek Jan van Damme, Timotheus Höttges,
Dr. Thomas Kremer, Claudia Nemat, Prof. Dr. Marion Schick
Commercial register: Amtsgericht Bonn HRB 6794
Registered office: Bonn

BIG CHANGES START SMALL – CONSERVE RESOURCES BY NOT PRINTING EVERY E-MAIL.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Tempest - Stress Test][qa] : implement a full SSH connection on "ssh_floating.py" and improve it

[Koderer, Marc]

> Von: LELOUP Julien [mailto:julien.lel...@3ds.com]
> Gesendet: Montag, 3. März 2014 11:21
> An: OpenStack Development Mailing List (not for usage questions); Koderer,
> Marc
> Betreff: [Tempest - Stress Test][qa] : implement a full SSH connection on
> "ssh_floating.py" and improve it
>
> Hello,
>
> It tourns out that this stress test maybe not implemented in the right
> place.
>
> At ther moment I have put the SSH stress test in large_ops but Joe Gordon
> pointed (in https://review.openstack.org/#/c/74067/) that the large_ops
> gates are not meant to launch real servers.

Yep, this is a discussion that came up in the last summit and I might
undervalue the consequences. IMHO stress test's should both work with
fake drivers and with real environments. If they don't and we really
want to support such cases we need to flag such tests or make these
checks optional. If we do so these check possibly never run in the
gate. Especially for stress tests it's might the case that there
are designed for in-house usage and not for the gate.

>
> So where should I put this test ? I'm thinking of creating a new scenario
> file inheriting of the class specified in the large_ops file
> (TestLargeOpsScenario) in order to avoid duplicating the "nova_boot"
> method.
> Is it OK for you ?
>
> Is there a better place where I should put my test ?

I mean we still have the tempest/stress/actions/ and we could put something
like that in there. In general I would like to discuss this topic in the next
QA meeting..
@Julien: are you able to join the next meeting? It would be 22:00 UTC.


>
> Best Regards,
>
> Julien LELOUP
> julien.lel...@3ds.com
>

Regards,
Marc


>
> -Original Message-
> From: LELOUP Julien [mailto:julien.lel...@3ds.com]
> Sent: Monday, February 17, 2014 5:02 PM
> To: OpenStack Development Mailing List (not for usage questions); Koderer,
> Marc
> Subject: Re: [openstack-dev] [qa] [Tempest - Stress Test] : implement a
> full SSH connection on "ssh_floating.py" and improve it
>
> Hello Marc,
>
> I'm raising this subject again since I have pushed a patch set
> implementing this SSH stress test in Tempest.
> As you suggested, I wrote it as a scenario test : I'm currently using it
> to check the ability of my stack to launch x servers at the same time and
> having them fully available to use.
>
> As a reminder, here is the Blueprint :
> https://blueprints.launchpad.net/tempest/+spec/stress-test-ssh-floating-ip
>
> Plese feel free to check it and make feedback :)
>
> Best Regards,
>
> Julien LELOUP
> julien.lel...@3ds.com
>
> -Original Message-
> From: LELOUP Julien [mailto:julien.lel...@3ds.com]
> Sent: Monday, January 20, 2014 11:55 AM
> To: Koderer, Marc
> Cc: openstack-dev@lists.openstack.org
> Subject: Re: [openstack-dev] [qa] [Tempest - Stress Test] : implement a
> full SSH connection on "ssh_floating.py" and improve it
>
> Hello Marc,
>
> Thanks again for your help, your blog post is helpful.
>
> So I will start writing a new scenario test to get this full SSH stress
> test on newly created VM.
> I will put more details about it in the blueprint I created for this :
> https://blueprints.launchpad.net/tempest/+spec/stress-test-ssh-floating-ip
>
> Best Regards,
>
> Julien LELOUP
> julien.lel...@3ds.com
>
>
> -Original Message-
> From: Koderer, Marc [mailto:m.kode...@telekom.de]
> Sent: Saturday, January 18, 2014 10:11 AM
> To: LELOUP Julien
> Cc: openstack-dev@lists.openstack.org
> Subject: RE: [qa] [Tempest - Stress Test] : implement a full SSH
> connection on "ssh_floating.py" and improve it
>
> Hello Julien,
>
> maybe my blog post helps you with some more details:
>
> http://telekomcloud.github.io/2013/09/11/new-ways-of-tempest-stress-
> testing.html
>
> You can run single test if you add a new json file with the test function
> you want to test. Like:
> https://github.com/openstack/tempest/blob/master/tempest/stress/etc/sample
> -unit-test.json
>
> With that you can launch them with the parameters you already described.
>
> Regards,
> Marc
>
> 
> From: LELOUP Julien [julien.lel...@3ds.com]
> Sent: Friday, January 17, 2014 3:49 PM
> To: Koderer, Marc
> Cc: openstack-dev@lists.openstack.org
> Subject: RE: [Tempest - Stress Test] : implement a full SSH connection on
> "ssh_floating.py" and improve it
>
> Hi Marc,
>
> The Etherpad you provided was helpful to know the current state of the
> stress tests.
>
> I admit that I have some difficulties to understand how I can run a single
> test built with the 

Re: [openstack-dev] Cleaning OpenStack resources

[Koderer, Marc]

Hi Forent,

thanks for the link. Our current implementation of for the cleanup is a bit 
buggy. I will give your script a try.
Maybe we could use it an replace the existing module.

Regards
Marc 

From: Florent Flament [florent.flament-...@cloudwatt.com]
Sent: Thursday, February 20, 2014 10:30 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [qa] Cleaning OpenStack resources

Hi David,

I have been working on an OpenStack resources cleanup script that allows you to 
wipe out all resources in a given project, I guess you could use / adapt it for 
your own case. It is available on github there: 
https://github.com/cloudwatt/ospurge

You can also install it with pip (pip install ospurge).

As for the floating ips, you should be able to list and remove them, by using 
the neutron CLI:
* neutron floatingip-list
* neutron floatingip-delete

Forent Flament

- Original Message -
From: "David Kranz" 
To: "OpenStack Development Mailing List" 
Sent: Wednesday, February 19, 2014 10:15:12 PM
Subject: [openstack-dev] [qa] Cleaning OpenStack resources

I was looking at https://review.openstack.org/#/c/73274/1 which makes it
configurable whether a brute-force cleanup of resources is done after
success. This got my wondering how this should really be done. As admin,
there are some resources that can be cleaned and some that I don't  know
how. For example, as admin you can list all servers and delete them with
the --all-tenants flag. But for floating ips I don't see a way to list
all of them even as admin through the apis. Is there a way that an admin
can, through the api, locate all resources used by a particular tenant?

  -David

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [qa] [Tempest - Stress Test] : some improvements / issues on the stress test framework

[Koderer, Marc]

Hi Julien,

please don't forget the [qa] tag - otherwise your lost in the ML noise ;)

Ok thanks for the bug reports. I confirmed 1273245 and 1273254 but I am not 
totally sure with 1273186.
Could you give some more details how the CLI interface will look like? Or 
simply propose a patch.
It could end up in a quite confusing interface.. if you allow kwargs for such a 
generic case.

Are you already working on those bugs? If yes, could you assign them to you?

Regards,
Marc

> 
> From: LELOUP Julien [julien.lel...@3ds.com]
> Sent: Monday, January 27, 2014 4:01 PM
> To: openstack-dev@lists.openstack.org
> Subject: [openstack-dev] [Tempest - Stress Test] : some improvements / issues 
> on the stress test framework
>
> Hi everyone,
>
> I would like to discuss some ideas / behavior that seems broken in the stress 
> test part of Tempest.
>
> I opened some tickets in Launchpad and I would like to get the feedback of 
> the community on these ideas / issues :
>
> - Provide kwargs from UnitTest to stress test scenarios 
> (https://bugs.launchpad.net/tempest/+bug/1273186 )
>
> - Stress Test - tearDown() not called if an exception occurred  
> (https://bugs.launchpad.net/tempest/+bug/1273245 )
>
> - Stress Test - cleanUp() removing all test resources as an admin 
> (https://bugs.launchpad.net/tempest/+bug/1273254 )
>
> Best Regards,
>
> Julien LELOUP
> julien.lel...@3ds.com
>
>
> This email and any attachments are intended solely for the use of the 
> individual or entity to whom it is addressed and may be confidential and/or 
> privileged.
>
> If you are not one of the named recipients or have received this email in 
> error,
>
> (i) you should not read, disclose, or copy it,
>
> (ii) please notify sender of your receipt by reply email and delete this 
> email and all attachments,
>
> (iii) Dassault Systemes does not accept or assume any liability or 
> responsibility for any use of or reliance on this email.
>
> For other languages, go to http://www.3ds.com/terms/email-disclaimer
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [qa] [Tempest - Stress Test] : implement a full SSH connection on "ssh_floating.py" and improve it

[Koderer, Marc]

Hello Julien,

maybe my blog post helps you with some more details:

http://telekomcloud.github.io/2013/09/11/new-ways-of-tempest-stress-testing.html

You can run single test if you add a new json file with the test function you 
want to test. Like:
https://github.com/openstack/tempest/blob/master/tempest/stress/etc/sample-unit-test.json

With that you can launch them with the parameters you already described.

Regards,
Marc


From: LELOUP Julien [julien.lel...@3ds.com]
Sent: Friday, January 17, 2014 3:49 PM
To: Koderer, Marc
Cc: openstack-dev@lists.openstack.org
Subject: RE: [Tempest - Stress Test] : implement a full SSH connection on 
"ssh_floating.py" and improve it

Hi Marc,

The Etherpad you provided was helpful to know the current state of the stress 
tests.

I admit that I have some difficulties to understand how I can run a single test 
built with the @stresstest decorator (even not a beginner in Python, I still 
have things to learn on this technology and a lot more on OpenStack/Tempest :) 
).
I used to run my test using "./run_stress.py -t  -d ", which allowed me to run only one test 
with a dedicated configuration (number of threads, ...)

For what I understand now in Tempest, I only managed to run all tests, using 
"./run_tests.sh" and the only configuration I found related to stress tests was 
the [stress] section in tempest.conf.

For example : let say I ported my SSH stress test as a scenario test with the 
@stresstest decorator.
How can I launch this test (and only this one) and use a dedicated 
configuration file like ones we can found in "tempest/stress/etc" ?

Another question I have now : in the case that I have to use "run_test.sh" and 
not "run_stress.py" anymore, how do I get the test runs statistics I used to 
have, and where should I put some code to improve them ?

When I will have cleared my mind with all these kinds of practical details, 
maybe I should add some content on the Wiki about stress tests in Tempest.

Best Regards,

Julien LELOUP
julien.lel...@3ds.com

-Original Message-
From: Koderer, Marc [mailto:m.kode...@telekom.de]
Sent: Friday, January 17, 2014 3:07 PM
To: LELOUP Julien
Cc: openstack-dev@lists.openstack.org
Subject: RE: [qa] RE: [Tempest - Stress Test] : implement a full SSH connection 
on "ssh_floating.py" and improve it

Hi Julien,

most of the cases in tempest/stress are already covered by exiting tests in 
/api or /scenario. The only thing that is missing is the decorator on them.

BTW here is the Etherpad from the summit talk that we had:
https://etherpad.openstack.org/p/icehouse-summit-qa-stress-tests

It possible help to understand the state. I didn't managed to work on the 
action items that are left.

Your suggestions sound good. So I'd happy so see some patches :)

Regards
Marc

From: LELOUP Julien [julien.lel...@3ds.com]
Sent: Friday, January 17, 2014 11:52 AM
To: Koderer, Marc
Cc: openstack-dev@lists.openstack.org
Subject: RE: [qa] RE: [Tempest - Stress Test] : implement a full SSH connection 
on "ssh_floating.py" and improve it

Hello Marc,

Thanks for your answer.

At the moment I'm willing to spend some time on this kind of scenario so I will 
see how to use the stress decorator inside a scenario test.
Does this means that all stress tests available in "tempest/stress" should be 
ported as scenario tests with this decorator ?

I do have some ideas about features on stress test that I find useful for my 
own use case : like adding more statistics on stress test runs in order to use 
them as benchmarks.
I don't know if this kind of feature was already discussed in the OpenStack 
community but since stress tests are a bit deprecated now, maybe there is some 
room for this kind of improvement on "fresh" stress tests.

Best Regards,

Julien LELOUP

-Original Message-
From: Koderer, Marc [mailto:m.kode...@telekom.de]
Sent: Friday, January 17, 2014 9:45 AM
To: LELOUP Julien
Cc: openstack-dev@lists.openstack.org
Subject: [qa] RE: [Tempest - Stress Test] : implement a full SSH connection on 
"ssh_floating.py" and improve it

Hello Juilen,

I forwarded your mail to the correct mailing list. Please do not use the qa 
list any longer.

I am happy that you are interested in stress tests. All the tests in 
tempest/stress/actions are more or less deprecated. So what you should use 
instead is the stress decorator (e.g. 
https://github.com/openstack/tempest/blob/master/tempest/api/volume/test_volumes_actions.py#L55).
Unfortunately it's not yet used for scenarios like you describe. I'd suggest to 
build a scenario test in tempest/scenario and use this decorator on it.

Any patch like that is welcome on gerrit. If you are planning to work in that 
area for more than just a patch, a blueprint would be n

Re: [openstack-dev] [qa] RE: [Tempest - Stress Test] : implement a full SSH connection on "ssh_floating.py" and improve it

[Koderer, Marc]

Hi Julien,

most of the cases in tempest/stress are already covered by exiting tests in /api
or /scenario. The only thing that is missing is the decorator on them.

BTW here is the Etherpad from the summit talk that we had:
https://etherpad.openstack.org/p/icehouse-summit-qa-stress-tests

It possible help to understand the state. I didn't managed to work on the action
items that are left.

Your suggestions sound good. So I'd happy so see some patches :)

Regards
Marc

From: LELOUP Julien [julien.lel...@3ds.com]
Sent: Friday, January 17, 2014 11:52 AM
To: Koderer, Marc
Cc: openstack-dev@lists.openstack.org
Subject: RE: [qa] RE: [Tempest - Stress Test] : implement a full SSH connection 
on "ssh_floating.py" and improve it

Hello Marc,

Thanks for your answer.

At the moment I'm willing to spend some time on this kind of scenario so I will 
see how to use the stress decorator inside a scenario test.
Does this means that all stress tests available in "tempest/stress" should be 
ported as scenario tests with this decorator ?

I do have some ideas about features on stress test that I find useful for my 
own use case : like adding more statistics on stress test runs in order to use 
them as benchmarks.
I don't know if this kind of feature was already discussed in the OpenStack 
community but since stress tests are a bit deprecated now, maybe there is some 
room for this kind of improvement on "fresh" stress tests.

Best Regards,

Julien LELOUP

-Original Message-
From: Koderer, Marc [mailto:m.kode...@telekom.de]
Sent: Friday, January 17, 2014 9:45 AM
To: LELOUP Julien
Cc: openstack-dev@lists.openstack.org
Subject: [qa] RE: [Tempest - Stress Test] : implement a full SSH connection on 
"ssh_floating.py" and improve it

Hello Juilen,

I forwarded your mail to the correct mailing list. Please do not use the qa 
list any longer.

I am happy that you are interested in stress tests. All the tests in 
tempest/stress/actions are more or less deprecated. So what you should use 
instead is the stress decorator (e.g. 
https://github.com/openstack/tempest/blob/master/tempest/api/volume/test_volumes_actions.py#L55).
Unfortunately it's not yet used for scenarios like you describe. I'd suggest to 
build a scenario test in tempest/scenario and use this decorator on it.

Any patch like that is welcome on gerrit. If you are planning to work in that 
area for more than just a patch, a blueprint would be nice. A good way to 
coordinate your efforts is also in the QA meeting
(https://wiki.openstack.org/wiki/Meetings/QATeamMeeting)

Regards
Marc


From: LELOUP Julien [julien.lel...@3ds.com]
Sent: Wednesday, January 15, 2014 5:57 PM
To: openstack...@lists.openstack.org
Subject: [openstack-qa] [Tempest - Stress Test] : implement a full SSH 
connection on "ssh_floating.py" and improve it

Hi everyone,

I’m quite new on OpenStack / Tempest and I’m actually working on stress tests. 
I want to suggest a new feature in a currently available stress test.
Not sure if this email should be posted on the QA mailing list or the dev 
mailing list, but I give it a try here since it is about a Tempest stress test ☺

At the moment the “ssh_floating.py” stress test seems really interesting but I 
would like to improve it a bit.

By now this script is simulating an SSH connection by binding a TCP socket on 
the newly created instance. But this test don’t allow us to check if this 
instance is really available. I’m mostly thinking about the metadata service 
unable to provide the SSH key pair to the instance, but surely other scenarios 
can lead to an instance considered “ACTIVE” but actually unusable.

So I’m implementing a full SSH connection test using the “paramiko” SSH library 
and a key pair generated in the same way the other test resources are managed 
in this script : either one SSH key pair for every test runs or a new key pair 
for each run (depends on the JSON configuration file).
I don’t plan to remove the old test (TCP socket binding), rather move this one 
on a separate test function and put the full SSH connection test code instead.

Is this feature interesting for the OpenStack community ?
Should I create a blueprint on the Tempest project on Launchpad in order to 
provide my code through Gerrit ?

On a second time, I plan to overhaul improve this “ssh_floating.py” script by 
clean the code a little bit, and add more cleaning code in order to avoid 
leaving instances/security groups/floating IP behind : I do have this kind of 
behavior right now and I already improved the teardown() on this way.

Should I consider this code as a new functionality (thus create a blueprint) or 
should I create a defect and assign it to myself ?

Cordialement / Best Regards,

Julien LELOUP

R&D 3DExperience Platform IaaS Factory Technology Engineer





julien.lel...@3ds.com<mailto:j

[openstack-dev] [qa] negative test framework: ready for review

[Koderer, Marc]

Hi all,

first part of the negative test framework is ready for review:

https://review.openstack.org/#/c/64733/

Please have a look.

Regards
Marc and David

DEUTSCHE TELEKOM AG
Digital Business Unit, Cloud Services (P&I)
Marc Koderer
Cloud Technology Software Developer
T-Online-Allee 1, 64211 Darmstadt
www.telekom.com  

LIFE IS FOR SHARING.

DEUTSCHE TELEKOM AG
Supervisory Board: Prof. Dr. Ulrich Lehner (Chairman)
Board of Management: René Obermann (Chairman),
Reinhard Clemens, Niek Jan van Damme, Timotheus Höttges,
Dr. Thomas Kremer, Claudia Nemat, Prof. Dr. Marion Schick
Commercial register: Amtsgericht Bonn HRB 6794
Registered office: Bonn

BIG CHANGES START SMALL – CONSERVE RESOURCES BY NOT PRINTING EVERY E-MAIL.
 
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [qa] RE: [Tempest - Stress Test] : implement a full SSH connection on "ssh_floating.py" and improve it

[Koderer, Marc]

Hello Juilen,

I forwarded your mail to the correct mailing list. Please do not use the qa list
any longer.

I am happy that you are interested in stress tests. All the tests in
tempest/stress/actions are more or less deprecated. So what you should use
instead is the stress decorator (e.g. 
https://github.com/openstack/tempest/blob/master/tempest/api/volume/test_volumes_actions.py#L55).
Unfortunately it's not yet used for scenarios like you describe. I'd suggest to
build a scenario test in tempest/scenario and use this decorator on it.

Any patch like that is welcome on gerrit. If you are planning to work in that
area for more than just a patch, a blueprint would be nice. A good way to
coordinate your efforts is also in the QA meeting
(https://wiki.openstack.org/wiki/Meetings/QATeamMeeting)

Regards
Marc


From: LELOUP Julien [julien.lel...@3ds.com]
Sent: Wednesday, January 15, 2014 5:57 PM
To: openstack...@lists.openstack.org
Subject: [openstack-qa] [Tempest - Stress Test] : implement a full SSH 
connection on "ssh_floating.py" and improve it

Hi everyone,

I’m quite new on OpenStack / Tempest and I’m actually working on stress tests. 
I want to suggest a new feature in a currently available stress test.
Not sure if this email should be posted on the QA mailing list or the dev 
mailing list, but I give it a try here since it is about a Tempest stress test ☺

At the moment the “ssh_floating.py” stress test seems really interesting but I 
would like to improve it a bit.

By now this script is simulating an SSH connection by binding a TCP socket on 
the newly created instance. But this test don’t allow us to check if this 
instance is really available. I’m mostly thinking about the metadata service 
unable to provide the SSH key pair to the instance, but surely other scenarios 
can lead to an instance considered “ACTIVE” but actually unusable.

So I’m implementing a full SSH connection test using the “paramiko” SSH library 
and a key pair generated in the same way the other test resources are managed 
in this script : either one SSH key pair for every test runs or a new key pair 
for each run (depends on the JSON configuration file).
I don’t plan to remove the old test (TCP socket binding), rather move this one 
on a separate test function and put the full SSH connection test code instead.

Is this feature interesting for the OpenStack community ?
Should I create a blueprint on the Tempest project on Launchpad in order to 
provide my code through Gerrit ?

On a second time, I plan to overhaul improve this “ssh_floating.py” script by 
clean the code a little bit, and add more cleaning code in order to avoid 
leaving instances/security groups/floating IP behind : I do have this kind of 
behavior right now and I already improved the teardown() on this way.

Should I consider this code as a new functionality (thus create a blueprint) or 
should I create a defect and assign it to myself ?

Cordialement / Best Regards,

Julien LELOUP

R&D 3DExperience Platform IaaS Factory Technology Engineer





julien.lel...@3ds.com

[cid:image003.gif@01CF1216.D43ECE20]

3DS.COM


Dassault Systèmes | 10 rue Marcel Dassault, CS 40501 | 78946 
Vélizy-Villacoublay Cedex | France




This email and any attachments are intended solely for the use of the 
individual or entity to whom it is addressed and may be confidential and/or 
privileged.

If you are not one of the named recipients or have received this email in error,

(i) you should not read, disclose, or copy it,

(ii) please notify sender of your receipt by reply email and delete this email 
and all attachments,

(iii) Dassault Systemes does not accept or assume any liability or 
responsibility for any use of or reliance on this email.

For other languages, go to http://www.3ds.com/terms/email-disclaimer

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [qa] negative test generation and testscenarios

[Koderer, Marc]

Hi Robert,

https://review.openstack.org/#/c/64733 is updated and is using testscenarios
now. I was wondering if it would make sense to use a decorator instead of
a class attribute to mark a test case for a certain test scenario, like:

@TestCaseWithScenario(_scenarios1)
def test_demo(self):
...
@TestCaseWithScenario(_scenarios2)
def test_demo2(self):


I already implement something that seems to work with subunit:

http://paste.openstack.org/show/61175/

I think this would ease up the usage of testscenarios for the negative test
framework in tempest and looks quite cool.

Feedback is welcome!

Regards
Marc


From: Robert Collins [robe...@robertcollins.net]
Sent: Tuesday, January 07, 2014 3:59 AM
To: David Kranz
Cc: OpenStack Development Mailing List
Subject: Re: [openstack-dev] [qa] negative test generation and testscenarios

On 7 January 2014 07:41, David Kranz  wrote:
> Thanks to all who looked at https://review.openstack.org/#/c/64733/. There
> were a few minor issues I will address but the biggest one was the
> suggestion to run each variation as a separate test case using
> testscenarios. After looking into that I see a problem with this use case.
> Many of these negative tests need to allocate resources such as servers that
> are then referenced in the test cases. Our tests currently do stuff like
> that in setUpClass. But testscenarios, unless I misunderstand the code,
> replicates the whole class which means the resource setup would also be
> duplicated for each test. I think we need to share allocated resources
> across the potentially large number of negative variations, and see two
> options for that:

You don't understand the code :). testscenarios doesn't affect the
class of tests. As long as the resulting suite is wrapped in a
setUpClass compatible TestSuite object, setUpClass should keep
working.

> 1. Some magic similar to testscenarios but that creates test methods with
> varying parameters instead of test classes with varying attributes.
> Leave allocation in setUpClass.

-1 No different to testscenarios, just different to be different.

> 2. Using testscenarios as-is but move the allocation/cleanup to the module
> level (fixtures?)
> This would be a little complicated because a bunch of the setup stuff is
> defined as methods on the test base classes. Also, the classes would
> run in parallel.

Long term perhaps, not needed for this work.

-Rob

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [qa] api schema validation pattern changes

[Koderer, Marc]

> Von: David Kranz [mailto:dkr...@redhat.com]
> Gesendet: Montag, 13. Januar 2014 05:04
> An: openstack-dev@lists.openstack.org
> Betreff: Re: [openstack-dev] [nova] [qa] api schema validation pattern
> changes
> 
> On 01/12/2014 10:14 PM, Matthew Treinish wrote:
> > 
> >>> Last, is a question, is it possible to currently run the full API
> an
> >>> build a json schema for it all? Or to query these validating
> >>> schemas? We *really* want that over in tempest, so we can
> completely
> >>> drop the manual creation of negative testing, and just fuzz known
> bad against the schema definitions.
> >> Sorry, I'm not sure I understand this question correctly.
> >> We need to define schemas for each API with the separated schema
> patches.
> >> It is impossible to define one schema for all APIs.
> >>
> >>
> >> Hi  David, Marc,
> >>
> >> I guess the negative test generator of Tempest would need each API
> definition.
> >> Glance can provide API definitions through API with jsonschema
> >> format, but Nova does not have such feature.
> >> We need to port these API schema from Nova to Tempest, I guess.
> right?
> >>
> > As I understand things after reviewing the first versions of the
> > negative test generator patch is that we have to hard code the schema
> > into a file (right now it's in the test file, but eventually it'll be
> > an external input file). One of my issues with doing that is it's
> > highly manual process, essentially a copy and paste from the nova
> > tree. I think what we're looking for from this jsonschema validation
> > work is an API which we can query the API and get the jsonschema
> definitions; similar to what the glance API offers.
> I looked at the glance schema api and it seemed to just return the
> schema for the  json returned by the call, not for the arguments to the
> request. Am I wrong about that?
> Additionally, the schema for the request json in these patches is not
> enough for the negative test generator. The schema for negative
> generation also needs the http method type and a description of  the
> resources that are part of the url of the request. The negative test
> patch in the shared fork
> https://github.com/mkoderer/tempest/tree/feature/negative_tests
> defines such a schema. The tempest patch will be updated from this fork
> on Monday.
> >
> > I wouldn't advocate making the negative test generator be fully
> > dynamic for the same reason we don't autodetect which API versions
> and
> > extensions/features are enabled in tempest, but rather rely on the
> > config file. But, instead have an additional tool which could query
> > the schema for all the endpoints in nova and generate an input file
> > for the negative test generator. That way we'll still catch breaking
> > API changes in the gate, but it's not a manual process to update the
> > input file with the schema definitions in tempest when there is a
> > breaking API change. (Which hopefully should almost never happen)
> I agree, it would be ideal if there were a way for tempest to grab the
> appropriate schemas from somewhere else rather than hard-coding them in
> tempest itself. But as far as I can see,  most of the services don't
> even have json schemas defined.

I think at the end the hard coded json schema should be optional.
If the endpoint supports an export of the schema it simply takes it and
put the result automatically into the description of the negative test.
For me this is a second step since currently we already "duplicating"
somehow this in our manual negative tests. What we would need is
at least one interface that exposes the schema.

 Marc


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [qa] Moving the QA meeting time

[Koderer, Marc]

Hi all!

> -Ursprüngliche Nachricht-
> Von: Kenichi Oomichi [mailto:oomi...@mxs.nes.nec.co.jp]
> Gesendet: Donnerstag, 5. Dezember 2013 01:37
> An: OpenStack Development Mailing List (not for usage questions)
> Betreff: Re: [openstack-dev] [qa] Moving the QA meeting time
> 
> 
> Hi Matthew,
> 
> Thank you for picking this up.
> 
> > -Original Message-
> > From: Matthew Treinish [mailto:mtrein...@kortar.org]
> > Sent: Thursday, December 05, 2013 6:04 AM
> > To: openstack-dev@lists.openstack.org
> > Subject: [openstack-dev] [qa] Moving the QA meeting time
> >
> > Hi everyone,
> >
> > I'm looking at changing our weekly QA meeting time to make it more
> > globally attendable. Right now the current time of 17:00 UTC doesn't
> > really work for people who live in Asia Pacific timezones. (which
> > includes a third of the current core review team) There are 2
> approaches that I can see taking here:
> >
> >  1. We could either move the meeting time later so that it makes it
> easier for
> > people in the Asia Pacific region to attend.
> >
> >  2. Or we move to a alternating meeting time, where every other week
> the meeting
> > time changes. So we keep the current slot and alternate with
> something more
> > friendly for other regions.
> >
> > I think trying to stick to a single meeting time would be a better
> > call just for simplicity. But it gets difficult to appease everyone
> > that way which is where the appeal of the 2nd approach comes in.
> >
> > Looking at the available time slots here:
> > https://wiki.openstack.org/wiki/Meetings
> > there are plenty of open slots before 1500 UTC which would be early
> > for people in the US and late for people in the Asia Pacific region.
> > There are plenty of slots starting at 2300 UTC which is late for
> people in Europe.
> >
> > Would something like 2200 UTC on Wed. or Thurs work for everyone?
> >
> > What are people's opinions on this?
> 
> I am in JST.
> Is Chris in CST, and Marc in CET?

Yes, Giulio and I are in CET. And Attila too, right?

> 
> Here is timezone difference.
> 15:00 UTC -> 07:00 PST -> 01:30 CST -> 16:00 CET - 24:00 JST 22:00 UTC
> -> 14:00 PST -> 08:30 CST -> 23:00 CET - 07:00 JST 23:00 UTC -> 15:00
> PST -> 09:30 CST -> 24:00 CET - 08:00 JST
> 
> I feel 22:00 would be nice.

I'd prefer to have two slots since 22 UTC is quite late. But I am ok with it if 
all others are fine.


Regards
Marc

DEUTSCHE TELEKOM AG
Digital Business Unit, Cloud Services (P&I)
Marc Koderer
Cloud Technology Software Developer 
T-Online-Allee 1, 64211 Darmstadt
E-Mail: m.kode...@telekom.de 
www.telekom.com

LIFE IS FOR SHARING.  

DEUTSCHE TELEKOM AG
Supervisory Board: Prof. Dr. Ulrich Lehner (Chairman)
Board of Management: René Obermann (Chairman),
Reinhard Clemens, Niek Jan van Damme, Timotheus Höttges,
Dr. Thomas Kremer, Claudia Nemat, Prof. Dr. Marion Schick
Commercial register: Amtsgericht Bonn HRB 6794
Registered office: Bonn

BIG CHANGES START SMALL - CONSERVE RESOURCES BY NOT PRINTING EVERY E-MAIL.




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [qa] Proposals for Tempest core

[Koderer, Marc]

+1 for both!

From: Sean Dague [s...@dague.net]
Sent: Friday, November 15, 2013 2:38 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [qa] Proposals for Tempest core

It's post summit time, so time to evaluate our current core group for
Tempest. There are a few community members that I'd like to nominate for
Tempest core, as I've found their review feedback over the last few
months to be invaluable. Tempest core folks, please +1 or -1 as you feel
appropriate:

Masayuki Igawa

His review history is here -
https://review.openstack.org/#/q/reviewer:masayuki.igawa%2540gmail.com+project:openstack/tempest,n,z

Ken'ichi Ohmichi

His review history is here -
https://review.openstack.org/#/q/reviewer:ken1ohmichi%2540gmail.com+project:openstack/tempest,n,z

They have both been actively engaged in the Tempest community, and have
been actively contributing to both Tempest and OpenStack integrated
projects, working hard to both enhance test coverage, and fix the issues
found in the projects themselves. This has been hugely beneficial to
OpenStack as a whole.

At the same time, it's also time, I think, to remove Jay Pipes from
tempest-core. Jay's not had much time for reviews of late, and it's
important that the core review team is a working title about actively
reviewing code.

With this change Tempest core would end up no longer being majority
north american, or even majority english as first language (that kind of
excites me). Adjusting to both there will be another mailing list thread
about changing our weekly meeting time to make it more friendly to our
APAC contributors.

-Sean

--
Sean Dague
http://dague.net


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [qa] Tracking development of scenario tests

[Koderer, Marc]

Hi all,

I think we have quite the same issue with the neutron testing. I already put it 
on the agenda for the QA meeting for today.
Let's make it to a general topic.

Regards
Marc

From: Giulio Fidente [gfide...@redhat.com]
Sent: Thursday, November 14, 2013 6:23 AM
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [qa] Tracking development of scenario tests

On 11/14/2013 12:24 AM, David Kranz wrote:
> 1. Developer checks in the zeroth version of a scenario test as work in
> progress. It contains a description of the test, and possibly work
> items.  This will "claim" the area of the proposed scenario to avoid
> duplication and allow others to comment through gerrit.
> 2. The developer pushes new versions, removing work in progress if the
> code is in working state and a review is desired and/or others will be
> contributing to the scenario.
> 3. When finished, any process-oriented content such as progress tracking
> is removed and the test is ready for final review.

+1 , the description will eventually contribute to documenting the scenarios

yet the submitter (step 1) remains in charge of adding to the draft the
reviewers

how about we map at least one volunteer to each service (via the HACKING
file) and ask submitters to add such a person as reviewer of its drafts
when the tests touch the service? this should help avoid tests duplication.

I very much like the idea of using gerrit for this
--
Giulio Fidente
GPG KEY: 08D733BA | IRC: giulivo

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [qa] moratorium on new negative tests in Tempest

[Koderer, Marc]

Hi,

see below.

Regards
Marc

From: Kenichi Oomichi [oomi...@mxs.nes.nec.co.jp]
Sent: Wednesday, November 13, 2013 7:24 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [qa] moratorium on new negative tests in Tempest

>> -Original Message-
>> From: David Kranz [mailto:dkr...@redhat.com]
>> Sent: Wednesday, November 13, 2013 4:33 AM
>> To: openstack-dev@lists.openstack.org
>> Subject: Re: [openstack-dev] [qa] moratorium on new negative tests in Tempest

>> So that is the starting point. Comments and suggestions welcome! Marc
>> and I just started working on an etherpad
>> https://etherpad.openstack.org/p/bp_negative_tests but any one is
>> welcome to contribute there.

>Negative tests based on yaml would be nice because of cleaning the code up
>and making the tests more readable.
>just one question:
> On the etherpad, there are some "invaid_uuid"s.
> Does that mean invalid string (ex. utf-8 string, not ascii)?
> or invalid uuid format(ex. uuid.uuid4() + "foo")?

Great that you already had a look!
So my idea is that we have a battery of functions which can create erroneous 
input.
My intention for invalid_uuid was just something like uuid.uuid4() - but the 
name is a bit misleading.
We can use additional functions that create the input that you are suggesting. 
I think all of them make sense.

> IIUC, in negative test session, we discussed that tests passing utf-8 string
> as API parameter should be negative tests, and the server should return a
> BadRequest response.
> I guess we need to implement such API negative tests. After that, if finding
> an unfavorable behavior of some server, we need to implement API validation
> for the server.
> (unfavorable behavior ex. When a client send utf-8 request, the server returns
> a NotFound response, not a BadRequest one)

+1
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [qa] fuzzy testing of client functions

[Koderer, Marc]

Hello folks,

I remember that we had a quick chat about fuzzy testing in a QA meeting. IMHO 
we have much too many negative tests in tempest that aren't really complex.
So I tried to build up a little tool that discovers the parameters of a client 
functions and randomizes the input for it.

It works like this:

  ./run_fuzzy_test.py image_client update_image  -n 100
 2013-10-19 13:42:10.634 27748 ERROR ClientFuzzTest [-] Bad request
 2013-10-19 13:42:10.658 27748 ERROR ClientFuzzTest [-] Bad request
 2013-10-19 13:42:10.670 27748 ERROR ClientFuzzTest [-] must be string or 
buffer, not int
 2013-10-19 13:42:10.709 27748 ERROR ClientFuzzTest [-] Object not found
 2013-10-19 13:42:10.738 27748 ERROR ClientFuzzTest [-] Bad request

Could you give me some feedback about your thoughts: 
https://review.openstack.org/#/c/52768

It's maybe just a starting point for a discussion on the summit.

Regards,
Marc


DEUTSCHE TELEKOM AG
Digital Business Unit, Cloud Services (P&I)
Marc Koderer
Cloud Technology Software Developer
T-Online-Allee 1, 64211 Darmstadt
www.telekom.com   

LIFE IS FOR SHARING. 

DEUTSCHE TELEKOM AG
Supervisory Board: Prof. Dr. Ulrich Lehner (Chairman)
Board of Management: René Obermann (Chairman),
Reinhard Clemens, Niek Jan van Damme, Timotheus Höttges,
Dr. Thomas Kremer, Claudia Nemat, Prof. Dr. Marion Schick
Commercial register: Amtsgericht Bonn HRB 6794
Registered office: Bonn

BIG CHANGES START SMALL – CONSERVE RESOURCES BY NOT PRINTING EVERY E-MAIL.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [qa] What is the purpose of stress test in tempest?

[Koderer, Marc]

Hi all,

After some refactoring work in tempest/stress I would like to raise a general
question since I have the feeling we have different opinions about the purpose
of tempest stress test.
Giulio already put this topic on the agenda for the next QA meeting and I just
want that we use the time in between to think about the problem ;)

Please have a look to the discussions in:
  https://review.openstack.org/#/c/39752/
  https://review.openstack.org/#/c/38980/

IMHO a stress test is not a independent test area (like api-tests, scenario
test etc) it's just a way how tests are processed. So in theory any small
API-test could be used as a stress test and any scenario test could be used
too.

I see two use cases for stress tests:

  - As a developer I want to find bugs that occur under load (like raise 
conditions)
--> Leads to many small and concurrent api tests 
  - As OPS/QA I want to generate load that simulates real life load in a
production-like system
--> Leads to concurrent scenario test

Kind Regards
Marc


DEUTSCHE TELEKOM AG
Digital Business Unit, Cloud Services (P&I)
Marc Koderer
Cloud Technology Software Developer 
T-Online-Allee 1, 64211 Darmstadt
E-Mail: m.kode...@telekom.de 
www.telekom.com

LIFE IS FOR SHARING.  

DEUTSCHE TELEKOM AG
Supervisory Board: Prof. Dr. Ulrich Lehner (Chairman)
Board of Management: René Obermann (Chairman),
Reinhard Clemens, Niek Jan van Damme, Timotheus Höttges,
Dr. Thomas Kremer, Claudia Nemat, Prof. Dr. Marion Schick
Commercial register: Amtsgericht Bonn HRB 6794
Registered office: Bonn

BIG CHANGES START SMALL - CONSERVE RESOURCES BY NOT PRINTING EVERY E-MAIL.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev