Re: [openstack-dev] Which program for Rally

2014-08-18 Thread Matthew Treinish
On Fri, Aug 15, 2014 at 01:57:29AM +0400, Boris Pavlovic wrote: Matt, One thing did just occur to me while writing this though it's probably worth investigating splitting out the stress test framework as an external tool/project after we start work on the tempest library. [3] I

Re: [openstack-dev] Which program for Rally

2014-08-15 Thread Jeremy Stanley
On 2014-08-13 19:28:48 -0700 (-0700), Joe Gordon wrote: We actually run out of nodes almost every day now (except weekends), we have about 800 nodes, and hit that quota most days [...] It's worth noting that a lot of the recent exhaustion has been due to OpenStack bugs impacting the providers

Re: [openstack-dev] Which program for Rally

2014-08-14 Thread Boris Pavlovic
Matt, One thing did just occur to me while writing this though it's probably worth investigating splitting out the stress test framework as an external tool/project after we start work on the tempest library. [3] I fully agree with the fact that stress testing doesn't belong to Tempest.

Re: [openstack-dev] Which program for Rally

2014-08-13 Thread Boris Pavlovic
Matt, On Mon, Aug 11, 2014 at 07:06:11PM -0400, Zane Bitter wrote: On 11/08/14 16:21, Matthew Treinish wrote: I'm sorry, but the fact that the docs in the rally tree has a section for user testimonials [4] I feel speaks a lot about the intent of the project. Yes, you are absolutely

Re: [openstack-dev] Which program for Rally

2014-08-13 Thread Matthew Treinish
On Tue, Aug 12, 2014 at 01:45:17AM +0400, Boris Pavlovic wrote: Hi stackers, I would like to put some more details on current situation. The issue is with what Rally is in it's current form. It's scope is too large and monolithic, and it duplicates much of the functionality we

Re: [openstack-dev] Which program for Rally

2014-08-13 Thread Duncan Thomas
On 13 August 2014 13:57, Matthew Treinish mtrein...@kortar.org wrote: On Tue, Aug 12, 2014 at 01:45:17AM +0400, Boris Pavlovic wrote: Keystone, Glance, Cinder, Neutron and Heat are running rally performance jobs, that can be used for performance testing, benchmarking, regression testing

Re: [openstack-dev] Which program for Rally

2014-08-13 Thread Angus Salkeld
On Wed, 2014-08-13 at 15:48 -0600, Duncan Thomas wrote: On 13 August 2014 13:57, Matthew Treinish mtrein...@kortar.org wrote: On Tue, Aug 12, 2014 at 01:45:17AM +0400, Boris Pavlovic wrote: Keystone, Glance, Cinder, Neutron and Heat are running rally performance jobs, that can be used for

Re: [openstack-dev] Which program for Rally

2014-08-13 Thread Robert Collins
On 14 August 2014 09:48, Duncan Thomas duncan.tho...@gmail.com wrote: On 13 August 2014 13:57, Matthew Treinish mtrein...@kortar.org wrote: So this is actually the communication problem I mentioned before. Singling out individual projects and getting them to add a rally job is not cross

Re: [openstack-dev] Which program for Rally

2014-08-13 Thread Joe Gordon
On Wed, Aug 13, 2014 at 2:48 PM, Duncan Thomas duncan.tho...@gmail.com wrote: On 13 August 2014 13:57, Matthew Treinish mtrein...@kortar.org wrote: On Tue, Aug 12, 2014 at 01:45:17AM +0400, Boris Pavlovic wrote: Keystone, Glance, Cinder, Neutron and Heat are running rally performance jobs,

Re: [openstack-dev] Which program for Rally

2014-08-13 Thread Tom Fifield
On 13/08/14 19:55, Boris Pavlovic wrote: Matt, On Mon, Aug 11, 2014 at 07:06:11PM -0400, Zane Bitter wrote: On 11/08/14 16:21, Matthew Treinish wrote: I'm sorry, but the fact that the docs in the rally tree has a section for user testimonials [4] I feel speaks a

Re: [openstack-dev] Which program for Rally

2014-08-12 Thread Doug Hellmann
On Aug 11, 2014, at 12:00 PM, David Kranz dkr...@redhat.com wrote: On 08/06/2014 05:48 PM, John Griffith wrote: I have to agree with Duncan here. I also don't know if I fully understand the limit in options. Stress test seems like it could/should be different (again overlap isn't a

Re: [openstack-dev] Which program for Rally

2014-08-12 Thread Matthew Treinish
On Mon, Aug 11, 2014 at 07:06:11PM -0400, Zane Bitter wrote: On 11/08/14 16:21, Matthew Treinish wrote: I'm sorry, but the fact that the docs in the rally tree has a section for user testimonials [4] I feel speaks a lot about the intent of the project. What... does that even mean? Yeah,

Re: [openstack-dev] Which program for Rally

2014-08-11 Thread David Kranz
On 08/06/2014 05:48 PM, John Griffith wrote: I have to agree with Duncan here. I also don't know if I fully understand the limit in options. Stress test seems like it could/should be different (again overlap isn't a horrible thing) and I don't see it as siphoning off resources so not sure of

Re: [openstack-dev] Which program for Rally

2014-08-11 Thread Zane Bitter
On 08/08/14 10:41, Anne Gentle wrote: - Would have to ensure Rally is what we want first as getting to be PTL since you are first to propose seems to be the model. I know that at one time it was popular in the trade/gutter press to cast aspersions on new projects by saying that someone

Re: [openstack-dev] Which program for Rally

2014-08-11 Thread Matthew Treinish
I apologize for the delay in my response to this thread, between travelling and having a stuck 'a' key on my laptop this is the earliest I could respond. I opted for a separate branch on this thread to summarize my views and I'll respond inline later on some of the previous discussion. On Wed,

Re: [openstack-dev] Which program for Rally

2014-08-11 Thread David Kranz
On 08/11/2014 04:21 PM, Matthew Treinish wrote: I apologize for the delay in my response to this thread, between travelling and having a stuck 'a' key on my laptop this is the earliest I could respond. I opted for a separate branch on this thread to summarize my views and I'll respond

Re: [openstack-dev] Which program for Rally

2014-08-11 Thread Boris Pavlovic
Hi stackers, I would like to put some more details on current situation. The issue is with what Rally is in it's current form. It's scope is too large and monolithic, and it duplicates much of the functionality we either already have or need in current QA or Infra projects. But, nothing

Re: [openstack-dev] Which program for Rally

2014-08-11 Thread Zane Bitter
On 11/08/14 16:21, Matthew Treinish wrote: I'm sorry, but the fact that the docs in the rally tree has a section for user testimonials [4] I feel speaks a lot about the intent of the project. What... does that even mean? They seem like just the type of guys that would help Keystone with

Re: [openstack-dev] Which program for Rally

2014-08-08 Thread Neependra Kumar Khare
- Original Message - From: Thierry Carrez thie...@openstack.org To: OpenStack Development Mailing List openstack-dev@lists.openstack.org Sent: Wednesday, August 6, 2014 4:00:35 PM Subject: [openstack-dev] Which program for Rally 1. Rally as an essential QA tool Performance testing

Re: [openstack-dev] Which program for Rally

2014-08-08 Thread Anne Gentle
On Wed, Aug 6, 2014 at 5:30 AM, Thierry Carrez thie...@openstack.org wrote: Hi everyone, At the TC meeting yesterday we discussed Rally program request and incubation request. We quickly dismissed the incubation request, as Rally appears to be able to live happily on top of OpenStack and

Re: [openstack-dev] Which program for Rally

2014-08-07 Thread marc
Hi John, see below. Zitat von John Griffith john.griff...@solidfire.com: I have to agree with Duncan here. I also don't know if I fully understand the limit in options. Stress test seems like it could/should be different This is correct, Rally and Tempest Stress test have a different

[openstack-dev] Which program for Rally

2014-08-07 Thread Rohan Kanade
Date: Wed, 06 Aug 2014 09:44:12 -0400 From: Sean Dague s...@dague.net To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] Which program for Rally Message-ID: 53e2312c.8000...@dague.net Content-Type: text/plain; charset=utf-8 Like the fact that right now the rally team

Re: [openstack-dev] Which program for Rally

2014-08-07 Thread Angus Salkeld
On Wed, 2014-08-06 at 15:48 -0600, John Griffith wrote: I have to agree with Duncan here. I also don't know if I fully understand the limit in options. Stress test seems like it could/should be different (again overlap isn't a horrible thing) and I don't see it as siphoning off resources so

Re: [openstack-dev] Which program for Rally

2014-08-07 Thread Sean Dague
On 08/06/2014 05:48 PM, John Griffith wrote: I have to agree with Duncan here. I also don't know if I fully understand the limit in options. Stress test seems like it could/should be different (again overlap isn't a horrible thing) and I don't see it as siphoning off resources so not sure of

Re: [openstack-dev] Which program for Rally

2014-08-07 Thread Sean Dague
On 08/07/2014 07:58 AM, Angus Salkeld wrote: On Wed, 2014-08-06 at 15:48 -0600, John Griffith wrote: I have to agree with Duncan here. I also don't know if I fully understand the limit in options. Stress test seems like it could/should be different (again overlap isn't a horrible thing) and

Re: [openstack-dev] Which program for Rally

2014-08-07 Thread John Griffith
On Thu, Aug 7, 2014 at 6:20 AM, Sean Dague s...@dague.net wrote: On 08/07/2014 07:58 AM, Angus Salkeld wrote: On Wed, 2014-08-06 at 15:48 -0600, John Griffith wrote: I have to agree with Duncan here. I also don't know if I fully understand the limit in options. Stress test seems like it

Re: [openstack-dev] Which program for Rally

2014-08-07 Thread John Griffith
On Thu, Aug 7, 2014 at 9:02 AM, John Griffith john.griff...@solidfire.com wrote: On Thu, Aug 7, 2014 at 6:20 AM, Sean Dague s...@dague.net wrote: On 08/07/2014 07:58 AM, Angus Salkeld wrote: On Wed, 2014-08-06 at 15:48 -0600, John Griffith wrote: I have to agree with Duncan here. I

Re: [openstack-dev] Which program for Rally

2014-08-06 Thread Russell Bryant
On 08/06/2014 06:30 AM, Thierry Carrez wrote: Hi everyone, At the TC meeting yesterday we discussed Rally program request and incubation request. We quickly dismissed the incubation request, as Rally appears to be able to live happily on top of OpenStack and would benefit from having a

Re: [openstack-dev] Which program for Rally

2014-08-06 Thread Sean Dague
On 08/06/2014 09:11 AM, Russell Bryant wrote: On 08/06/2014 06:30 AM, Thierry Carrez wrote: Hi everyone, At the TC meeting yesterday we discussed Rally program request and incubation request. We quickly dismissed the incubation request, as Rally appears to be able to live happily on top of

Re: [openstack-dev] Which program for Rally

2014-08-06 Thread Russell Bryant
On 08/06/2014 09:44 AM, Sean Dague wrote: Something that we need to figure out is given where we are in the release cycle do we want to ask the QA team to go off and do Rally deep dive now to try to pull it apart into the parts that make sense for other programs to take in. There are always

Re: [openstack-dev] Which program for Rally

2014-08-06 Thread Duncan Thomas
I'm not following here - you complain about rally being monolithic, then suggest that parts of it should be baked into tempest - a tool that is already huge and difficult to get into. I'd rather see tools that do one thing well and some overlap than one tool to rule them all. On 6 August 2014

Re: [openstack-dev] Which program for Rally

2014-08-06 Thread John Griffith
I have to agree with Duncan here. I also don't know if I fully understand the limit in options. Stress test seems like it could/should be different (again overlap isn't a horrible thing) and I don't see it as siphoning off resources so not sure of the issue. We've become quite wrapped up in

Re: [openstack-dev] Which program for Rally

2014-08-06 Thread Yingjun Li
From a user’s aspect i do think Rally is more suitable for a product-ready cloud, and seems like it is where it focused on. It’s very easy to evaluate that if the performance of the cloud is better after we adjust some configs or some other tuning. It also provides SLA which maybe not so

Re: [openstack-dev] Which program for Rally

2014-08-06 Thread Swapnil Kulkarni
I agree with Duncan and John here, I am not as old contributor in OpneStack as most of the people commenting here are, but I see we have done this right throughout the OpenStack lifecycle, at the start we only had Nova and we could have always said hey lets have everything in Nova but we went