Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-12-28 Thread Ali Gamal
On Oct 17, 2013 12:45 AM, Boris Pavlovic bpavlo...@mirantis.com wrote: Hi Stackers, We are thrilled to present to you Rally, the benchmarking system for OpenStack. It is not a secret that we have performance scaling issues and that OpenStack won’t scale out of box. It is also well known

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-12-28 Thread Ali Gamal
ali.bed...@gmail.com On Oct 17, 2013 12:45 AM, Boris Pavlovic bpavlo...@mirantis.com wrote: Hi Stackers, We are thrilled to present to you Rally, the benchmarking system for OpenStack. It is not a secret that we have performance scaling issues and that OpenStack won’t scale out of box.

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-12-28 Thread Tim Bell
be accumulated ? Tim From: Ali Gamal [mailto:aga...@itsyn.com] Sent: 28 December 2013 20:31 To: OpenStack Development Mailing List Cc: Ali Beddah Subject: Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack On Oct 17, 2013 12:45 AM, Boris Pavlovic bpavlo

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-12-28 Thread Boris Pavlovic
...@itsyn.com] *Sent:* 28 December 2013 20:31 *To:* OpenStack Development Mailing List *Cc:* Ali Beddah *Subject:* Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack On Oct 17, 2013 12:45 AM, Boris Pavlovic bpavlo...@mirantis.com wrote: Hi Stackers, We are thrilled

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-12-28 Thread Tim Bell
: bo...@pavlovic.ru [mailto:bo...@pavlovic.ru] On Behalf Of Boris Pavlovic Sent: 28 December 2013 21:02 To: OpenStack Development Mailing List (not for usage questions) Cc: Ali Beddah; Tim Bell Subject: Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack Ali Gamal, ? Tim

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-12-28 Thread Boris Pavlovic
) *Cc:* Ali Beddah; Tim Bell *Subject:* Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack Ali Gamal, ? Tim, Yes it fits. There are couple of use cases that should be covered by Rally: 1) Easy way to find fix bottlenecks/scale issues improve

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-21 Thread Joe Gordon
-Original Message- From: Robert Collins [mailto:robe...@robertcollins.net] Sent: 20 October 2013 21:03 To: OpenStack Development Mailing List Subject: Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack On 21 October 2013 07:36, Alex Gaynor alex.gay...@gmail.com

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-21 Thread Sean Dague
On 10/20/2013 02:36 PM, Alex Gaynor wrote: There's several issues involved in doing automated regression checking for benchmarks: - You need a platform which is stable. Right now all our CI runs on virtualized instances, and I don't think there's any particular guarantee it'll be the same

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-21 Thread Tim Bell
It is not just the development effort but also those users who rely on tempest to probe their production environments. If there is a second project, they'd have to configure the endpoints, user accounts etc. in both systems. In my view, this cannot be done at the gate, it would take too long

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-20 Thread Boris Pavlovic
Hi Sean, Honestly, there has been so much work done in Tempest with the stress tests and the scenario tests in this release, and a large growing community around those, that it doesn't make any sense to me that you put item #3 as a non Tempest thing. I really don't like to copy paste

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-20 Thread Tim Bell
to cover benchmarking and ensure changes which impact performance are picked up early ? Tim From: Boris Pavlovic [mailto:bpavlo...@mirantis.com] Sent: 20 October 2013 19:34 To: OpenStack Development Mailing List Subject: Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack Hi

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-20 Thread Alex Gaynor
** ** ** ** *From:* Boris Pavlovic [mailto:bpavlo...@mirantis.com] *Sent:* 20 October 2013 19:34 *To:* OpenStack Development Mailing List *Subject:* Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack ** ** Hi Sean, ** ** ** ** Honestly, there has been so much

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-20 Thread Robert Collins
On 21 October 2013 07:36, Alex Gaynor alex.gay...@gmail.com wrote: There's several issues involved in doing automated regression checking for benchmarks: - You need a platform which is stable. Right now all our CI runs on virtualized instances, and I don't think there's any particular

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-20 Thread Tim Bell
[mailto:robe...@robertcollins.net] Sent: 20 October 2013 21:03 To: OpenStack Development Mailing List Subject: Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack On 21 October 2013 07:36, Alex Gaynor alex.gay...@gmail.com wrote: There's several issues involved in doing

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-20 Thread David Kranz
On 10/20/2013 03:03 PM, Robert Collins wrote: On 21 October 2013 07:36, Alex Gaynor alex.gay...@gmail.com wrote: There's several issues involved in doing automated regression checking for benchmarks: - You need a platform which is stable. Right now all our CI runs on virtualized instances, and

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-19 Thread Boris Pavlovic
David, Yes, that is the goal. We should be able to get information about how it works, not only that it works. And get such interesting results: https://wiki.openstack.org/wiki/Rally#How_influence_amqp_rpc_single_reply_queue_on_performance Best regards, Boris Pavlovic --- Mirantis Inc. On

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-19 Thread Sean Dague
On 10/18/2013 12:17 PM, Boris Pavlovic wrote: John, Actually seems like a pretty good suggestion IMO, at least something worth some investigation and consideration before quickly discounting it. Rather than that's not what tempest is, maybe it's something tempest could do. Don't know, not

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-18 Thread Boris Pavlovic
John, Actually seems like a pretty good suggestion IMO, at least something worth some investigation and consideration before quickly discounting it. Rather than that's not what tempest is, maybe it's something tempest could do. Don't know, not saying one way or the other, just wondering if it's

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-18 Thread John Griffith
On Fri, Oct 18, 2013 at 10:17 AM, Boris Pavlovic bo...@pavlovic.me wrote: John, Actually seems like a pretty good suggestion IMO, at least something worth some investigation and consideration before quickly discounting it. Rather than that's not what tempest is, maybe it's something tempest

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-18 Thread David Kranz
On 10/18/2013 12:17 PM, Boris Pavlovic wrote: John, Actually seems like a pretty good suggestion IMO, at least something worth some investigation and consideration before quickly discounting it. Rather than that's not what tempest is, maybe it's something tempest could do. Don't know, not

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-17 Thread Joshua Harlow
@lists.openstack.org Subject: [openstack-dev] Announce of Rally - benchmarking system for OpenStack Hi Stackers, We are thrilled to present to you Rally, the benchmarking system for OpenStack. It is not a secret that we have performance scaling issues and that OpenStack won’t scale out of box

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-17 Thread Jay Pipes
:41 PM To: OpenStack Development Mailing List openstack-dev@lists.openstack.org mailto:openstack-dev@lists.openstack.org Subject: [openstack-dev] Announce of Rally - benchmarking system for OpenStack Hi Stackers, We are thrilled to present to you Rally, the benchmarking system for OpenStack

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-17 Thread Boris Pavlovic
@lists.openstack.org mailto:openstack-dev@lists.**openstack.orgopenstack-dev@lists.openstack.org Subject: [openstack-dev] Announce of Rally - benchmarking system for OpenStack Hi Stackers, We are thrilled to present to you Rally, the benchmarking system for OpenStack. It is not a secret

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-17 Thread Jay Pipes
On 10/17/2013 03:32 PM, Boris Pavlovic wrote: Jay, Or, alternately, just have Rally as part of Tempest. Actually, tempest is used only to verify that cloud works properly. And verification is only small part of the Rally. At this moment we are using fuel-ostf-tests, but we are going to use

Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-17 Thread John Griffith
On Thu, Oct 17, 2013 at 1:44 PM, Jay Pipes jaypi...@gmail.com wrote: On 10/17/2013 03:32 PM, Boris Pavlovic wrote: Jay, Or, alternately, just have Rally as part of Tempest. Actually, tempest is used only to verify that cloud works properly. And verification is only small part of the

[openstack-dev] Announce of Rally - benchmarking system for OpenStack

2013-10-16 Thread Boris Pavlovic
Hi Stackers, We are thrilled to present to you Rally, the benchmarking system for OpenStack. It is not a secret that we have performance scaling issues and that OpenStack won’t scale out of box. It is also well known that if you get your super big DC (5k-15k servers) you are able to find fix