Hello Julien,

maybe my blog post helps you with some more details:

http://telekomcloud.github.io/2013/09/11/new-ways-of-tempest-stress-testing.html

You can run single test if you add a new json file with the test function you 
want to test. Like:
https://github.com/openstack/tempest/blob/master/tempest/stress/etc/sample-unit-test.json

With that you can launch them with the parameters you already described.

Regards,
Marc

________________________________________
From: LELOUP Julien [julien.lel...@3ds.com]
Sent: Friday, January 17, 2014 3:49 PM
To: Koderer, Marc
Cc: openstack-dev@lists.openstack.org
Subject: RE: [Tempest - Stress Test] : implement a full SSH connection on 
"ssh_floating.py" and improve it

Hi Marc,

The Etherpad you provided was helpful to know the current state of the stress 
tests.

I admit that I have some difficulties to understand how I can run a single test 
built with the @stresstest decorator (even not a beginner in Python, I still 
have things to learn on this technology and a lot more on OpenStack/Tempest :) 
).
I used to run my test using "./run_stress.py -t <JSON configuration pointing at 
my action/.py script> -d <duration>", which allowed me to run only one test 
with a dedicated configuration (number of threads, ...)

For what I understand now in Tempest, I only managed to run all tests, using 
"./run_tests.sh" and the only configuration I found related to stress tests was 
the [stress] section in tempest.conf.

For example : let say I ported my SSH stress test as a scenario test with the 
@stresstest decorator.
How can I launch this test (and only this one) and use a dedicated 
configuration file like ones we can found in "tempest/stress/etc" ?

Another question I have now : in the case that I have to use "run_test.sh" and 
not "run_stress.py" anymore, how do I get the test runs statistics I used to 
have, and where should I put some code to improve them ?

When I will have cleared my mind with all these kinds of practical details, 
maybe I should add some content on the Wiki about stress tests in Tempest.

Best Regards,

Julien LELOUP
julien.lel...@3ds.com

-----Original Message-----
From: Koderer, Marc [mailto:m.kode...@telekom.de]
Sent: Friday, January 17, 2014 3:07 PM
To: LELOUP Julien
Cc: openstack-dev@lists.openstack.org
Subject: RE: [qa] RE: [Tempest - Stress Test] : implement a full SSH connection 
on "ssh_floating.py" and improve it

Hi Julien,

most of the cases in tempest/stress are already covered by exiting tests in 
/api or /scenario. The only thing that is missing is the decorator on them.

BTW here is the Etherpad from the summit talk that we had:
https://etherpad.openstack.org/p/icehouse-summit-qa-stress-tests

It possible help to understand the state. I didn't managed to work on the 
action items that are left.

Your suggestions sound good. So I'd happy so see some patches :)

Regards
Marc
________________________________________
From: LELOUP Julien [julien.lel...@3ds.com]
Sent: Friday, January 17, 2014 11:52 AM
To: Koderer, Marc
Cc: openstack-dev@lists.openstack.org
Subject: RE: [qa] RE: [Tempest - Stress Test] : implement a full SSH connection 
on "ssh_floating.py" and improve it

Hello Marc,

Thanks for your answer.

At the moment I'm willing to spend some time on this kind of scenario so I will 
see how to use the stress decorator inside a scenario test.
Does this means that all stress tests available in "tempest/stress" should be 
ported as scenario tests with this decorator ?

I do have some ideas about features on stress test that I find useful for my 
own use case : like adding more statistics on stress test runs in order to use 
them as benchmarks.
I don't know if this kind of feature was already discussed in the OpenStack 
community but since stress tests are a bit deprecated now, maybe there is some 
room for this kind of improvement on "fresh" stress tests.

Best Regards,

Julien LELOUP

-----Original Message-----
From: Koderer, Marc [mailto:m.kode...@telekom.de]
Sent: Friday, January 17, 2014 9:45 AM
To: LELOUP Julien
Cc: openstack-dev@lists.openstack.org
Subject: [qa] RE: [Tempest - Stress Test] : implement a full SSH connection on 
"ssh_floating.py" and improve it

Hello Juilen,

I forwarded your mail to the correct mailing list. Please do not use the qa 
list any longer.

I am happy that you are interested in stress tests. All the tests in 
tempest/stress/actions are more or less deprecated. So what you should use 
instead is the stress decorator (e.g. 
https://github.com/openstack/tempest/blob/master/tempest/api/volume/test_volumes_actions.py#L55).
Unfortunately it's not yet used for scenarios like you describe. I'd suggest to 
build a scenario test in tempest/scenario and use this decorator on it.

Any patch like that is welcome on gerrit. If you are planning to work in that 
area for more than just a patch, a blueprint would be nice. A good way to 
coordinate your efforts is also in the QA meeting
(https://wiki.openstack.org/wiki/Meetings/QATeamMeeting)

Regards
Marc

________________________________________
From: LELOUP Julien [julien.lel...@3ds.com]
Sent: Wednesday, January 15, 2014 5:57 PM
To: openstack...@lists.openstack.org
Subject: [openstack-qa] [Tempest - Stress Test] : implement a full SSH 
connection on "ssh_floating.py" and improve it

Hi everyone,

I’m quite new on OpenStack / Tempest and I’m actually working on stress tests. 
I want to suggest a new feature in a currently available stress test.
Not sure if this email should be posted on the QA mailing list or the dev 
mailing list, but I give it a try here since it is about a Tempest stress test ☺

At the moment the “ssh_floating.py” stress test seems really interesting but I 
would like to improve it a bit.

By now this script is simulating an SSH connection by binding a TCP socket on 
the newly created instance. But this test don’t allow us to check if this 
instance is really available. I’m mostly thinking about the metadata service 
unable to provide the SSH key pair to the instance, but surely other scenarios 
can lead to an instance considered “ACTIVE” but actually unusable.

So I’m implementing a full SSH connection test using the “paramiko” SSH library 
and a key pair generated in the same way the other test resources are managed 
in this script : either one SSH key pair for every test runs or a new key pair 
for each run (depends on the JSON configuration file).
I don’t plan to remove the old test (TCP socket binding), rather move this one 
on a separate test function and put the full SSH connection test code instead.

Is this feature interesting for the OpenStack community ?
Should I create a blueprint on the Tempest project on Launchpad in order to 
provide my code through Gerrit ?

On a second time, I plan to overhaul improve this “ssh_floating.py” script by 
clean the code a little bit, and add more cleaning code in order to avoid 
leaving instances/security groups/floating IP behind : I do have this kind of 
behavior right now and I already improved the teardown() on this way.

Should I consider this code as a new functionality (thus create a blueprint) or 
should I create a defect and assign it to myself ?

Cordialement / Best Regards,

Julien LELOUP

R&D 3DExperience Platform IaaS Factory Technology Engineer





julien.lel...@3ds.com<mailto:julien.lel...@3ds.com>

[cid:image003.gif@01CF1216.D43ECE20]

3DS.COM<http://www.3ds.com/>


Dassault Systèmes | 10 rue Marcel Dassault, CS 40501 | 78946 
Vélizy-Villacoublay Cedex | France




This email and any attachments are intended solely for the use of the 
individual or entity to whom it is addressed and may be confidential and/or 
privileged.

If you are not one of the named recipients or have received this email in error,

(i) you should not read, disclose, or copy it,

(ii) please notify sender of your receipt by reply email and delete this email 
and all attachments,

(iii) Dassault Systemes does not accept or assume any liability or 
responsibility for any use of or reliance on this email.

For other languages, go to http://www.3ds.com/terms/email-disclaimer

This email and any attachments are intended solely for the use of the 
individual or entity to whom it is addressed and may be confidential and/or 
privileged.

If you are not one of the named recipients or have received this email in error,

(i) you should not read, disclose, or copy it,

(ii) please notify sender of your receipt by reply email and delete this email 
and all attachments,

(iii) Dassault Systemes does not accept or assume any liability or 
responsibility for any use of or reliance on this email.

For other languages, go to http://www.3ds.com/terms/email-disclaimer
This email and any attachments are intended solely for the use of the 
individual or entity to whom it is addressed and may be confidential and/or 
privileged.

If you are not one of the named recipients or have received this email in error,

(i) you should not read, disclose, or copy it,

(ii) please notify sender of your receipt by reply email and delete this email 
and all attachments,

(iii) Dassault Systemes does not accept or assume any liability or 
responsibility for any use of or reliance on this email.

For other languages, go to http://www.3ds.com/terms/email-disclaimer
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Reply via email to