Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-12-28 Thread Boris Pavlovic
21:02 > *To:* OpenStack Development Mailing List (not for usage questions) > *Cc:* Ali Beddah; Tim Bell > > *Subject:* Re: [openstack-dev] Announce of Rally - benchmarking system > for OpenStack > > > > Ali Gamal, > > > > ? > > > > > > > &g

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-12-28 Thread Tim Bell
From: bo...@pavlovic.ru [mailto:bo...@pavlovic.ru] On Behalf Of Boris Pavlovic Sent: 28 December 2013 21:02 To: OpenStack Development Mailing List (not for usage questions) Cc: Ali Beddah; Tim Bell Subject: Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack Ali Gamal, ?

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-12-28 Thread Boris Pavlovic
ulated ? > > > > Tim > > > > > > *From:* Ali Gamal [mailto:aga...@itsyn.com] > *Sent:* 28 December 2013 20:31 > > *To:* OpenStack Development Mailing List > *Cc:* Ali Beddah > *Subject:* Re: [openstack-dev] Announce of Rally - benchmarking system > for OpenStack

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-12-28 Thread Tim Bell
be accumulated ? Tim From: Ali Gamal [mailto:aga...@itsyn.com] Sent: 28 December 2013 20:31 To: OpenStack Development Mailing List Cc: Ali Beddah Subject: Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack On Oct 17, 2013 12:45 AM, "Boris Pavlovic" mai

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-12-28 Thread Ali Gamal
ali.bed...@gmail.com On Oct 17, 2013 12:45 AM, "Boris Pavlovic" wrote: > Hi Stackers, > > > We are thrilled to present to you Rally, the benchmarking system for > OpenStack. > > > It is not a secret that we have performance & scaling issues and that > OpenStack won’t scale out of box. It is also

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-12-28 Thread Ali Gamal
On Oct 17, 2013 12:45 AM, "Boris Pavlovic" wrote: > Hi Stackers, > > > We are thrilled to present to you Rally, the benchmarking system for > OpenStack. > > > It is not a secret that we have performance & scaling issues and that > OpenStack won’t scale out of box. It is also well known that if yo

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-21 Thread Tim Bell
It is not just the development effort but also those users who rely on tempest to probe their production environments. If there is a second project, they'd have to configure the endpoints, user accounts etc. in both systems. In my view, this cannot be done at the gate, it would take too long bu

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-21 Thread Sean Dague
On 10/20/2013 02:36 PM, Alex Gaynor wrote: There's several issues involved in doing automated regression checking for benchmarks: - You need a platform which is stable. Right now all our CI runs on virtualized instances, and I don't think there's any particular guarantee it'll be the same underl

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-21 Thread Joe Gordon
6m 09s (non-voting) best, Joe > > Tim > > > -Original Message- > > From: Robert Collins [mailto:robe...@robertcollins.net] > > Sent: 20 October 2013 21:03 > > To: OpenStack Development Mailing List > > Subject: Re: [openstack-dev] Announce of Rally - benchm

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-20 Thread David Kranz
On 10/20/2013 03:03 PM, Robert Collins wrote: On 21 October 2013 07:36, Alex Gaynor wrote: There's several issues involved in doing automated regression checking for benchmarks: - You need a platform which is stable. Right now all our CI runs on virtualized instances, and I don't think there's

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-20 Thread Tim Bell
ollins [mailto:robe...@robertcollins.net] > Sent: 20 October 2013 21:03 > To: OpenStack Development Mailing List > Subject: Re: [openstack-dev] Announce of Rally - benchmarking system for > OpenStack > > On 21 October 2013 07:36, Alex Gaynor wrote: > > There's several

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-20 Thread Robert Collins
On 21 October 2013 07:36, Alex Gaynor wrote: > There's several issues involved in doing automated regression checking for > benchmarks: > > - You need a platform which is stable. Right now all our CI runs on > virtualized instances, and I don't think there's any particular guarantee > it'll be the

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-20 Thread Alex Gaynor
sure changes which impact performance > are picked up early ? > > ** ** > > Tim > > ** ** > > ** ** > > *From:* Boris Pavlovic [mailto:bpavlo...@mirantis.com] > *Sent:* 20 October 2013 19:34 > > *To:* OpenStack Development Mai

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-20 Thread Tim Bell
extended to cover benchmarking and ensure changes which impact performance are picked up early ? Tim From: Boris Pavlovic [mailto:bpavlo...@mirantis.com] Sent: 20 October 2013 19:34 To: OpenStack Development Mailing List Subject: Re: [openstack-dev] Announce of Rally - benchmarking system for Op

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-20 Thread Boris Pavlovic
Hi Sean, >> Honestly, there has been so much work done in Tempest with the stress tests and the scenario tests in this release, and a large growing community around those, that it doesn't make any sense to me that you put item #3 as a non Tempest thing. I really don't like to copy paste function

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-19 Thread Sean Dague
On 10/18/2013 12:17 PM, Boris Pavlovic wrote: John, Actually seems like a pretty good suggestion IMO, at least something worth some investigation and consideration before quickly discounting it. Rather than "that's not what tempest is", maybe it's something tempest "could do". Don't know, not

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-19 Thread Boris Pavlovic
David, Yes, that is the goal. We should be able to get information about "how" it works, not only that it works. And get such interesting results: https://wiki.openstack.org/wiki/Rally#How_influence_amqp_rpc_single_reply_queue_on_performance Best regards, Boris Pavlovic --- Mirantis Inc. O

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-18 Thread David Kranz
On 10/18/2013 12:17 PM, Boris Pavlovic wrote: John, Actually seems like a pretty good suggestion IMO, at least something worth some investigation and consideration before quickly discounting it. Rather than "that's not what tempest is", maybe it's something tempest "could do". Don't know, n

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-18 Thread John Griffith
On Fri, Oct 18, 2013 at 10:17 AM, Boris Pavlovic wrote: > John, > > Actually seems like a pretty good suggestion IMO, at least something worth > some investigation and consideration before quickly discounting it. Rather > than "that's not what tempest is", maybe it's something tempest "could do"

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-18 Thread Boris Pavlovic
John, Actually seems like a pretty good suggestion IMO, at least something worth some investigation and consideration before quickly discounting it. Rather than "that's not what tempest is", maybe it's something tempest "could do". Don't know, not saying one way or the other, just wondering if i

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-17 Thread John Griffith
On Thu, Oct 17, 2013 at 1:44 PM, Jay Pipes wrote: > On 10/17/2013 03:32 PM, Boris Pavlovic wrote: > >> Jay, >> >> >> Or, alternately, just have Rally as part of Tempest. >> >> >> Actually, tempest is used only to verify that cloud works properly. >> And verification is only small part of the Rall

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-17 Thread Jay Pipes
On 10/17/2013 03:32 PM, Boris Pavlovic wrote: Jay, Or, alternately, just have Rally as part of Tempest. Actually, tempest is used only to verify that cloud works properly. And verification is only small part of the Rally. At this moment we are using fuel-ostf-tests, but we are going to use t

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-17 Thread Boris Pavlovic
t;> > > Or, alternately, just have Rally as part of Tempest. > > -jay > > From: Boris Pavlovic > <mailto:bpavlo...@mirantis.com**>> >> >> Reply-To: OpenStack Development Mailing List >> >> <mailto:openstack-dev@lists.**openstac

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-17 Thread Jay Pipes
Mailing List mailto:openstack-dev@lists.openstack.org>> Subject: [openstack-dev] Announce of Rally - benchmarking system for OpenStack Hi Stackers, We are thrilled to present to you Rally, the benchmarking system for OpenStack. It is not a secret that we have performance & scaling issues an

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-17 Thread Joshua Harlow
mailto:bpavlo...@mirantis.com>> Reply-To: OpenStack Development Mailing List mailto:openstack-dev@lists.openstack.org>> Date: Wednesday, October 16, 2013 3:41 PM To: OpenStack Development Mailing List mailto:openstack-dev@lists.openstack.org>> Subject: [openstack-dev] Announce of Rally - b

[openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-16 Thread Boris Pavlovic
Hi Stackers, We are thrilled to present to you Rally, the benchmarking system for OpenStack. It is not a secret that we have performance & scaling issues and that OpenStack won’t scale out of box. It is also well known that if you get your super big DC (5k-15k servers) you are able to find & fi