Re: [openstack-dev] [Tempest - Stress Test][qa] : implement a full SSH connection on "ssh_floating.py" and improve it
>>I mean we still have the tempest/stress/actions/ and we could put something >>like that in there. In general I would like to discuss this topic in the next >>QA meeting.. >>@Julien: are you able to join the next meeting? It would be 22:00 UTC. >@Marc : so next Thrusday (3/6/2014) ? Yes I can be there. I have a last minute hindrance preventing me to attend to the next QA meeting. I will see to attend the one at 1700 UTC Thursday 13 march. Best Regards, Julien LELOUP julien.lel...@3ds.com This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged. If you are not one of the named recipients or have received this email in error, (i) you should not read, disclose, or copy it, (ii) please notify sender of your receipt by reply email and delete this email and all attachments, (iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email. For other languages, go to http://www.3ds.com/terms/email-disclaimer ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Tempest - Stress Test][qa] : implement a full SSH connection on "ssh_floating.py" and improve it
From: Koderer, Marc [mailto:m.kode...@telekom.de] Sent: Tuesday, March 04, 2014 10:41 AM To: LELOUP Julien; OpenStack Development Mailing List (not for usage questions) Subject: AW: [Tempest - Stress Test][qa] : implement a full SSH connection on "ssh_floating.py" and improve it >> >> So where should I put this test ? I'm thinking of creating a new >> scenario file inheriting of the class specified in the large_ops file >> (TestLargeOpsScenario) in order to avoid duplicating the "nova_boot" >> method. >> Is it OK for you ? >> >> Is there a better place where I should put my test ? >I mean we still have the tempest/stress/actions/ and we could put something >like that in there. In general I would like to discuss this topic in the next >QA meeting.. >@Julien: are you able to join the next meeting? It would be 22:00 UTC. @Marc : so next Thrusday (3/6/2014) ? Yes I can be there. > > Best Regards, > > Julien LELOUP > julien.lel...@3ds.com > Regards, Marc > > -Original Message- > From: LELOUP Julien [mailto:julien.lel...@3ds.com] > Sent: Monday, February 17, 2014 5:02 PM > To: OpenStack Development Mailing List (not for usage questions); > Koderer, Marc > Subject: Re: [openstack-dev] [qa] [Tempest - Stress Test] : implement > a full SSH connection on "ssh_floating.py" and improve it > > Hello Marc, > > I'm raising this subject again since I have pushed a patch set > implementing this SSH stress test in Tempest. > As you suggested, I wrote it as a scenario test : I'm currently using > it to check the ability of my stack to launch x servers at the same > time and having them fully available to use. > > As a reminder, here is the Blueprint : > https://blueprints.launchpad.net/tempest/+spec/stress-test-ssh-floatin > g-ip > > Plese feel free to check it and make feedback :) > > Best Regards, > > Julien LELOUP > julien.lel...@3ds.com > > -Original Message- > From: LELOUP Julien [mailto:julien.lel...@3ds.com] > Sent: Monday, January 20, 2014 11:55 AM > To: Koderer, Marc > Cc: openstack-dev@lists.openstack.org > Subject: Re: [openstack-dev] [qa] [Tempest - Stress Test] : implement > a full SSH connection on "ssh_floating.py" and improve it > > Hello Marc, > > Thanks again for your help, your blog post is helpful. > > So I will start writing a new scenario test to get this full SSH > stress test on newly created VM. > I will put more details about it in the blueprint I created for this : > https://blueprints.launchpad.net/tempest/+spec/stress-test-ssh-floatin > g-ip > > Best Regards, > > Julien LELOUP > julien.lel...@3ds.com > > > -Original Message- > From: Koderer, Marc [mailto:m.kode...@telekom.de] > Sent: Saturday, January 18, 2014 10:11 AM > To: LELOUP Julien > Cc: openstack-dev@lists.openstack.org > Subject: RE: [qa] [Tempest - Stress Test] : implement a full SSH > connection on "ssh_floating.py" and improve it > > Hello Julien, > > maybe my blog post helps you with some more details: > > http://telekomcloud.github.io/2013/09/11/new-ways-of-tempest-stress- > testing.html > > You can run single test if you add a new json file with the test > function you want to test. Like: > https://github.com/openstack/tempest/blob/master/tempest/stress/etc/sa > mple > -unit-test.json > > With that you can launch them with the parameters you already described. > > Regards, > Marc > > > From: LELOUP Julien [julien.lel...@3ds.com] > Sent: Friday, January 17, 2014 3:49 PM > To: Koderer, Marc > Cc: openstack-dev@lists.openstack.org > Subject: RE: [Tempest - Stress Test] : implement a full SSH connection > on "ssh_floating.py" and improve it > > Hi Marc, > > The Etherpad you provided was helpful to know the current state of the > stress tests. > > I admit that I have some difficulties to understand how I can run a > single test built with the @stresstest decorator (even not a beginner > in Python, I still have things to learn on this technology and a lot > more on OpenStack/Tempest :) ). > I used to run my test using "./run_stress.py -t pointing at my action/.py script> -d ", which allowed me to > run only one test with a dedicated configuration (number of threads, > ...) > > For what I understand now in Tempest, I only managed to run all tests, > using "./run_tests.sh" and the only configuration I found related to > stress tests was the [stress] section in tempest.conf. > > For example : let say I ported my SSH stress test as a scenario test > with the @stresstest decorator. > How can I launch this test (and only this one) and use a dedicated > configuration file like ones we can found in "tempest/stress/etc" ? > > Another question I have now : in the case that I have to use "run_test.sh" > and not "run_stress.py" anymore, how do I get the test runs statistics > I used to have, and where should I put some code to improve them ? > > When I will have cleared my mind with all these kinds of practical > details, maybe I should add some content on the Wiki about str
Re: [openstack-dev] [Tempest - Stress Test][qa] : implement a full SSH connection on "ssh_floating.py" and improve it
> Von: LELOUP Julien [mailto:julien.lel...@3ds.com] > Gesendet: Montag, 3. März 2014 11:21 > An: OpenStack Development Mailing List (not for usage questions); Koderer, > Marc > Betreff: [Tempest - Stress Test][qa] : implement a full SSH connection on > "ssh_floating.py" and improve it > > Hello, > > It tourns out that this stress test maybe not implemented in the right > place. > > At ther moment I have put the SSH stress test in large_ops but Joe Gordon > pointed (in https://review.openstack.org/#/c/74067/) that the large_ops > gates are not meant to launch real servers. Yep, this is a discussion that came up in the last summit and I might undervalue the consequences. IMHO stress test's should both work with fake drivers and with real environments. If they don't and we really want to support such cases we need to flag such tests or make these checks optional. If we do so these check possibly never run in the gate. Especially for stress tests it's might the case that there are designed for in-house usage and not for the gate. > > So where should I put this test ? I'm thinking of creating a new scenario > file inheriting of the class specified in the large_ops file > (TestLargeOpsScenario) in order to avoid duplicating the "nova_boot" > method. > Is it OK for you ? > > Is there a better place where I should put my test ? I mean we still have the tempest/stress/actions/ and we could put something like that in there. In general I would like to discuss this topic in the next QA meeting.. @Julien: are you able to join the next meeting? It would be 22:00 UTC. > > Best Regards, > > Julien LELOUP > julien.lel...@3ds.com > Regards, Marc > > -Original Message- > From: LELOUP Julien [mailto:julien.lel...@3ds.com] > Sent: Monday, February 17, 2014 5:02 PM > To: OpenStack Development Mailing List (not for usage questions); Koderer, > Marc > Subject: Re: [openstack-dev] [qa] [Tempest - Stress Test] : implement a > full SSH connection on "ssh_floating.py" and improve it > > Hello Marc, > > I'm raising this subject again since I have pushed a patch set > implementing this SSH stress test in Tempest. > As you suggested, I wrote it as a scenario test : I'm currently using it > to check the ability of my stack to launch x servers at the same time and > having them fully available to use. > > As a reminder, here is the Blueprint : > https://blueprints.launchpad.net/tempest/+spec/stress-test-ssh-floating-ip > > Plese feel free to check it and make feedback :) > > Best Regards, > > Julien LELOUP > julien.lel...@3ds.com > > -Original Message- > From: LELOUP Julien [mailto:julien.lel...@3ds.com] > Sent: Monday, January 20, 2014 11:55 AM > To: Koderer, Marc > Cc: openstack-dev@lists.openstack.org > Subject: Re: [openstack-dev] [qa] [Tempest - Stress Test] : implement a > full SSH connection on "ssh_floating.py" and improve it > > Hello Marc, > > Thanks again for your help, your blog post is helpful. > > So I will start writing a new scenario test to get this full SSH stress > test on newly created VM. > I will put more details about it in the blueprint I created for this : > https://blueprints.launchpad.net/tempest/+spec/stress-test-ssh-floating-ip > > Best Regards, > > Julien LELOUP > julien.lel...@3ds.com > > > -Original Message- > From: Koderer, Marc [mailto:m.kode...@telekom.de] > Sent: Saturday, January 18, 2014 10:11 AM > To: LELOUP Julien > Cc: openstack-dev@lists.openstack.org > Subject: RE: [qa] [Tempest - Stress Test] : implement a full SSH > connection on "ssh_floating.py" and improve it > > Hello Julien, > > maybe my blog post helps you with some more details: > > http://telekomcloud.github.io/2013/09/11/new-ways-of-tempest-stress- > testing.html > > You can run single test if you add a new json file with the test function > you want to test. Like: > https://github.com/openstack/tempest/blob/master/tempest/stress/etc/sample > -unit-test.json > > With that you can launch them with the parameters you already described. > > Regards, > Marc > > > From: LELOUP Julien [julien.lel...@3ds.com] > Sent: Friday, January 17, 2014 3:49 PM > To: Koderer, Marc > Cc: openstack-dev@lists.openstack.org > Subject: RE: [Tempest - Stress Test] : implement a full SSH connection on > "ssh_floating.py" and improve it > > Hi Marc, > > The Etherpad you provided was helpful to know the current state of the > stress tests. > > I admit that I have some difficulties to understand how I can run a single > test built with the @stresstest decorator (even not a beginner in Python, > I still have things to learn on this technology and a lot more on > OpenStack/Tempest :) ). > I used to run my test using "./run_stress.py -t pointing at my action/.py script> -d ", which allowed me to run > only one test with a dedicated configuration (number of threads, ...) > > For what I understand now in Tempest, I only managed to run all tests, > using "./run_tests.sh" and the only configuration
[openstack-dev] [Tempest - Stress Test][qa] : implement a full SSH connection on "ssh_floating.py" and improve it
Hello, It tourns out that this stress test maybe not implemented in the right place. At ther moment I have put the SSH stress test in large_ops but Joe Gordon pointed (in https://review.openstack.org/#/c/74067/) that the large_ops gates are not meant to launch real servers. So where should I put this test ? I'm thinking of creating a new scenario file inheriting of the class specified in the large_ops file (TestLargeOpsScenario) in order to avoid duplicating the "nova_boot" method. Is it OK for you ? Is there a better place where I should put my test ? Best Regards, Julien LELOUP julien.lel...@3ds.com -Original Message- From: LELOUP Julien [mailto:julien.lel...@3ds.com] Sent: Monday, February 17, 2014 5:02 PM To: OpenStack Development Mailing List (not for usage questions); Koderer, Marc Subject: Re: [openstack-dev] [qa] [Tempest - Stress Test] : implement a full SSH connection on "ssh_floating.py" and improve it Hello Marc, I'm raising this subject again since I have pushed a patch set implementing this SSH stress test in Tempest. As you suggested, I wrote it as a scenario test : I'm currently using it to check the ability of my stack to launch x servers at the same time and having them fully available to use. As a reminder, here is the Blueprint : https://blueprints.launchpad.net/tempest/+spec/stress-test-ssh-floating-ip Plese feel free to check it and make feedback :) Best Regards, Julien LELOUP julien.lel...@3ds.com -Original Message- From: LELOUP Julien [mailto:julien.lel...@3ds.com] Sent: Monday, January 20, 2014 11:55 AM To: Koderer, Marc Cc: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [qa] [Tempest - Stress Test] : implement a full SSH connection on "ssh_floating.py" and improve it Hello Marc, Thanks again for your help, your blog post is helpful. So I will start writing a new scenario test to get this full SSH stress test on newly created VM. I will put more details about it in the blueprint I created for this : https://blueprints.launchpad.net/tempest/+spec/stress-test-ssh-floating-ip Best Regards, Julien LELOUP julien.lel...@3ds.com -Original Message- From: Koderer, Marc [mailto:m.kode...@telekom.de] Sent: Saturday, January 18, 2014 10:11 AM To: LELOUP Julien Cc: openstack-dev@lists.openstack.org Subject: RE: [qa] [Tempest - Stress Test] : implement a full SSH connection on "ssh_floating.py" and improve it Hello Julien, maybe my blog post helps you with some more details: http://telekomcloud.github.io/2013/09/11/new-ways-of-tempest-stress-testing.html You can run single test if you add a new json file with the test function you want to test. Like: https://github.com/openstack/tempest/blob/master/tempest/stress/etc/sample-unit-test.json With that you can launch them with the parameters you already described. Regards, Marc From: LELOUP Julien [julien.lel...@3ds.com] Sent: Friday, January 17, 2014 3:49 PM To: Koderer, Marc Cc: openstack-dev@lists.openstack.org Subject: RE: [Tempest - Stress Test] : implement a full SSH connection on "ssh_floating.py" and improve it Hi Marc, The Etherpad you provided was helpful to know the current state of the stress tests. I admit that I have some difficulties to understand how I can run a single test built with the @stresstest decorator (even not a beginner in Python, I still have things to learn on this technology and a lot more on OpenStack/Tempest :) ). I used to run my test using "./run_stress.py -t -d ", which allowed me to run only one test with a dedicated configuration (number of threads, ...) For what I understand now in Tempest, I only managed to run all tests, using "./run_tests.sh" and the only configuration I found related to stress tests was the [stress] section in tempest.conf. For example : let say I ported my SSH stress test as a scenario test with the @stresstest decorator. How can I launch this test (and only this one) and use a dedicated configuration file like ones we can found in "tempest/stress/etc" ? Another question I have now : in the case that I have to use "run_test.sh" and not "run_stress.py" anymore, how do I get the test runs statistics I used to have, and where should I put some code to improve them ? When I will have cleared my mind with all these kinds of practical details, maybe I should add some content on the Wiki about stress tests in Tempest. Best Regards, Julien LELOUP julien.lel...@3ds.com -Original Message- From: Koderer, Marc [mailto:m.kode...@telekom.de] Sent: Friday, January 17, 2014 3:07 PM To: LELOUP Julien Cc: openstack-dev@lists.openstack.org Subject: RE: [qa] RE: [Tempest - Stress Test] : implement a full SSH connection on "ssh_floating.py" and improve it Hi Julien, most of the cases in tempest/stress are already covered by exiting tests in /api or /scenario. The only thing that is missing is the decorator on them. BTW here is the Etherpad from the summ
