Re: [openstack-dev] [FEMDC][MassivelyDistributed] Strawman proposal for message bus analysis

[Ken Giusti]

Thanks for kicking this off.  I've left some comments in the review.

-K

On Tue, Aug 8, 2017 at 10:07 AM, Matthieu Simonin <matthieu.simo...@inria.fr
> wrote:

> Hello,
>
> As discussed in the last meeting, I started to formalize the content of
> the etherpad in the performance WG documentation :
>
> https://review.openstack.org/#/c/491818/
>
> I've set some co-authorship according to what I saw in the etherpad. I
> guess this list can be shrunk/expanded on demand :)
>
> Best,
>
> Matt
>
>
> - Mail original -
> > De: "Matthieu Simonin" <matthieu.simo...@inria.fr>
> > À: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> > Envoyé: Jeudi 6 Juillet 2017 16:31:46
> > Objet: Re: [openstack-dev] [FEMDC][MassivelyDistributed] Strawman
> proposal for message bus analysis
> >
> > - Mail original -
> > > De: "Paul-Andre Raymond" <paul-andre.raym...@nexius.com>
> > > À: "OpenStack Development Mailing List (not for usage questions)"
> > > <openstack-dev@lists.openstack.org>
> > > Envoyé: Mercredi 5 Juillet 2017 21:48:29
> > > Objet: Re: [openstack-dev] [FEMDC][MassivelyDistributed] Strawman
> proposal
> > > for message bus analysis
> > >
> > > Thank you Matt,
> > >
> > > This is very insightful. It helps.
> > >
> > > The second link did not work for me.
> >
> > Oh yeah that's probably because of the on-going doc migration[1].
> >
> > [1]:
> > http://specs.openstack.org/openstack/docs-specs/specs/
> pike/os-manuals-migration.html
> >
> > >
> > > In the presentation, it mentioned that the load consisted “Boot and
> List”
> > > operations through Rally.
> > > Did I understand well?
> >
> > Yes.
> >
> > > Were those hitting the Openstack UI?
> >
> > Rally benchmarks put loads the various APIs and gather some metrics about
> > the execution (time, failures...)
> >
> > > Was keystone involved? Was it using Fernet or another sort of token?
> >
> > Keystone is indeed involved and the token was at that time UUID token.
> >
> > >
> > > Intuitively, I expected
> > > - the big driver for performance on mariadb would be authentication
> tokens.
> > > And fernet would allow to control that.
> > > - The big driver for performance on rabbitmq would be ceilometer, and
> it is
> > > not clear from your presentation that any telemetry data hit the
> message
> > > queue.
> >
> > Telemetry wasn't set up, I guess this would have killed Rabbit earlier
> in the
> > tests.
> > The split between notification and RPC messaging is interesting in this
> area.
> >
> > Bye,
> >
> > Matt
> >
> > >
> > > Regards,
> > >
> > > Paul-Andre
> > >
> > >
> > >
> > > -Original Message-
> > > From: Matthieu Simonin <matthieu.simo...@inria.fr>
> > > Reply-To: "OpenStack Development Mailing List (not for usage
> questions)"
> > > <openstack-dev@lists.openstack.org>
> > > Date: Saturday, July 1, 2017 at 4:42 AM
> > > To: "OpenStack Development Mailing List (not for usage questions)"
> > > <openstack-dev@lists.openstack.org>
> > > Subject: Re: [openstack-dev] [FEMDC][MassivelyDistributed] Strawman
> > > proposal
> > > for message bus analysis
> > >
> > > Hi Paul-André,
> > >
> > > This was without ceilometer. Nova + Neutron were consuming a lot of
> > > connections.
> > > Some charts are available in the Barcelona presentation[1] and the
> > > performance docs[2].
> > > In the latter you'll find some telemetry related tests.
> > >
> > > [1]:
> > > https://www.openstack.org/assets/presentation-media/
> Chasing-1000-nodes-scale.pdf
> > > [2]: https://docs.openstack.org/developer/performance-docs/
> > >
> > > Best,
> > >
> > > Matt
> > >
> > > - Mail original -
> > > > De: "Paul-Andre Raymond" <paul-andre.raym...@nexius.com>
> > > > À: "OpenStack Development Mailing List (not for usage questions)"
> > > > <openstack-dev@lists.openstack.org>
> > > > Envoyé: Vendredi 30 Juin 2017 18:42:04
> > > > Objet: Re: [openstack-de

Re: [openstack-dev] [FEMDC][MassivelyDistributed] Strawman proposal for message bus analysis

[Matthieu Simonin]

Hello,

As discussed in the last meeting, I started to formalize the content of the 
etherpad in the performance WG documentation :

https://review.openstack.org/#/c/491818/

I've set some co-authorship according to what I saw in the etherpad. I guess 
this list can be shrunk/expanded on demand :)

Best,

Matt


- Mail original -
> De: "Matthieu Simonin" <matthieu.simo...@inria.fr>
> À: "OpenStack Development Mailing List (not for usage questions)" 
> <openstack-dev@lists.openstack.org>
> Envoyé: Jeudi 6 Juillet 2017 16:31:46
> Objet: Re: [openstack-dev] [FEMDC][MassivelyDistributed] Strawman proposal 
> for message bus analysis
> 
> - Mail original -
> > De: "Paul-Andre Raymond" <paul-andre.raym...@nexius.com>
> > À: "OpenStack Development Mailing List (not for usage questions)"
> > <openstack-dev@lists.openstack.org>
> > Envoyé: Mercredi 5 Juillet 2017 21:48:29
> > Objet: Re: [openstack-dev] [FEMDC][MassivelyDistributed] Strawman proposal
> > for message bus analysis
> > 
> > Thank you Matt,
> > 
> > This is very insightful. It helps.
> > 
> > The second link did not work for me.
> 
> Oh yeah that's probably because of the on-going doc migration[1].
> 
> [1]:
> http://specs.openstack.org/openstack/docs-specs/specs/pike/os-manuals-migration.html
> 
> > 
> > In the presentation, it mentioned that the load consisted “Boot and List”
> > operations through Rally.
> > Did I understand well?
> 
> Yes.
> 
> > Were those hitting the Openstack UI?
> 
> Rally benchmarks put loads the various APIs and gather some metrics about
> the execution (time, failures...)
> 
> > Was keystone involved? Was it using Fernet or another sort of token?
> 
> Keystone is indeed involved and the token was at that time UUID token.
> 
> > 
> > Intuitively, I expected
> > - the big driver for performance on mariadb would be authentication tokens.
> > And fernet would allow to control that.
> > - The big driver for performance on rabbitmq would be ceilometer, and it is
> > not clear from your presentation that any telemetry data hit the message
> > queue.
> 
> Telemetry wasn't set up, I guess this would have killed Rabbit earlier in the
> tests.
> The split between notification and RPC messaging is interesting in this area.
> 
> Bye,
> 
> Matt
> 
> > 
> > Regards,
> > 
> > Paul-Andre
> > 
> > 
> > 
> > -Original Message-----
> > From: Matthieu Simonin <matthieu.simo...@inria.fr>
> > Reply-To: "OpenStack Development Mailing List (not for usage questions)"
> > <openstack-dev@lists.openstack.org>
> > Date: Saturday, July 1, 2017 at 4:42 AM
> > To: "OpenStack Development Mailing List (not for usage questions)"
> > <openstack-dev@lists.openstack.org>
> > Subject: Re: [openstack-dev] [FEMDC][MassivelyDistributed] Strawman
> > proposal
> > for message bus analysis
> > 
> > Hi Paul-André,
> > 
> > This was without ceilometer. Nova + Neutron were consuming a lot of
> > connections.
> > Some charts are available in the Barcelona presentation[1] and the
> > performance docs[2].
> > In the latter you'll find some telemetry related tests.
> > 
> > [1]:
> >     
> > https://www.openstack.org/assets/presentation-media/Chasing-1000-nodes-scale.pdf
> > [2]: https://docs.openstack.org/developer/performance-docs/
> > 
> > Best,
> > 
> > Matt
> > 
> > - Mail original -
> > > De: "Paul-Andre Raymond" <paul-andre.raym...@nexius.com>
> > > À: "OpenStack Development Mailing List (not for usage questions)"
> > > <openstack-dev@lists.openstack.org>
> > > Envoyé: Vendredi 30 Juin 2017 18:42:04
> > > Objet: Re: [openstack-dev] [FEMDC][MassivelyDistributed] Strawman
> > > proposal for message bus analysis
> > > 
> > > Hi Matthieu,
> > > 
> > > You mentioned 15000 connections with 1000 compute nodes.
> > > Was that mostly Nova? Was ceilometer involved?
> > > I would be curious to know how much AMQP traffic is Control
> > > related
> > > (e.g. spinning up VMs) vs how much is telemetry related in a
> > >     typical
> > > openstack deployment.
> > > Do we know that?
> >   

Re: [openstack-dev] [FEMDC][MassivelyDistributed] Strawman proposal for message bus analysis

[Matthieu Simonin]

- Mail original -
> De: "Paul-Andre Raymond" <paul-andre.raym...@nexius.com>
> À: "OpenStack Development Mailing List (not for usage questions)" 
> <openstack-dev@lists.openstack.org>
> Envoyé: Mercredi 5 Juillet 2017 21:48:29
> Objet: Re: [openstack-dev] [FEMDC][MassivelyDistributed] Strawman proposal 
> for message bus analysis
> 
> Thank you Matt,
> 
> This is very insightful. It helps.
> 
> The second link did not work for me.

Oh yeah that's probably because of the on-going doc migration[1].

[1]: 
http://specs.openstack.org/openstack/docs-specs/specs/pike/os-manuals-migration.html

> 
> In the presentation, it mentioned that the load consisted “Boot and List”
> operations through Rally.
> Did I understand well?

Yes.

> Were those hitting the Openstack UI?

Rally benchmarks put loads the various APIs and gather some metrics about 
the execution (time, failures...)

> Was keystone involved? Was it using Fernet or another sort of token?

Keystone is indeed involved and the token was at that time UUID token.

> 
> Intuitively, I expected
> - the big driver for performance on mariadb would be authentication tokens.
> And fernet would allow to control that.
> - The big driver for performance on rabbitmq would be ceilometer, and it is
> not clear from your presentation that any telemetry data hit the message
> queue.

Telemetry wasn't set up, I guess this would have killed Rabbit earlier in the 
tests.
The split between notification and RPC messaging is interesting in this area.

Bye,

Matt

> 
> Regards,
> 
> Paul-Andre
> 
> 
> 
> -Original Message-
> From: Matthieu Simonin <matthieu.simo...@inria.fr>
> Reply-To: "OpenStack Development Mailing List (not for usage questions)"
> <openstack-dev@lists.openstack.org>
> Date: Saturday, July 1, 2017 at 4:42 AM
> To: "OpenStack Development Mailing List (not for usage questions)"
> <openstack-dev@lists.openstack.org>
> Subject: Re: [openstack-dev] [FEMDC][MassivelyDistributed] Strawman proposal
> for message bus analysis
> 
> Hi Paul-André,
> 
> This was without ceilometer. Nova + Neutron were consuming a lot of
> connections.
> Some charts are available in the Barcelona presentation[1] and the
> performance docs[2].
> In the latter you'll find some telemetry related tests.
> 
> [1]:
> 
> https://www.openstack.org/assets/presentation-media/Chasing-1000-nodes-scale.pdf
> [2]: https://docs.openstack.org/developer/performance-docs/
> 
> Best,
> 
> Matt
> 
> - Mail original -
> > De: "Paul-Andre Raymond" <paul-andre.raym...@nexius.com>
> > À: "OpenStack Development Mailing List (not for usage questions)"
> > <openstack-dev@lists.openstack.org>
> > Envoyé: Vendredi 30 Juin 2017 18:42:04
> > Objet: Re: [openstack-dev] [FEMDC][MassivelyDistributed] Strawman
> > proposal for message bus analysis
> > 
> > Hi Matthieu,
> > 
> > You mentioned 15000 connections with 1000 compute nodes.
> > Was that mostly Nova? Was ceilometer involved?
> > I would be curious to know how much AMQP traffic is Control
> > related
> > (e.g. spinning up VMs) vs how much is telemetry related in a
> > typical
> > openstack deployment.
> > Do we know that?
> > 
> > I have also left some comments in the doc.
> > 
> > Paul-Andre
> > 
> > 
> > -Original Message-
> > From: Matthieu Simonin <matthieu.simo...@inria.fr>
>     > Reply-To: "OpenStack Development Mailing List (not for usage
> > questions)"
> > <openstack-dev@lists.openstack.org>
> > Date: Wednesday, June 21, 2017 at 6:54 PM
> > To: "OpenStack Development Mailing List (not for usage questions)"
> > <openstack-dev@lists.openstack.org>
> > Subject: Re: [openstack-dev] [FEMDC][MassivelyDistributed] Strawman
> > proposal formessage bus analysis
> > 
> > Hi Ken,
> > 
> > Thanks for starting this !
> >     I've made a first pass on the epad and left some notes and
> >     questions
> > there.
> > 
> > Best,
> > 
> > Matthieu
> > - Mail original 

Re: [openstack-dev] [FEMDC][MassivelyDistributed] Strawman proposal for message bus analysis

[Paul-Andre Raymond]

Thank you Matt,

This is very insightful. It helps.

The second link did not work for me.

In the presentation, it mentioned that the load consisted “Boot and List” 
operations through Rally. 
Did I understand well?
Were those hitting the Openstack UI?
Was keystone involved? Was it using Fernet or another sort of token?

Intuitively, I expected 
- the big driver for performance on mariadb would be authentication tokens. And 
fernet would allow to control that.
- The big driver for performance on rabbitmq would be ceilometer, and it is not 
clear from your presentation that any telemetry data hit the message queue.

Regards,

Paul-Andre



-Original Message-
From: Matthieu Simonin <matthieu.simo...@inria.fr>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org>
Date: Saturday, July 1, 2017 at 4:42 AM
To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org>
Subject: Re: [openstack-dev] [FEMDC][MassivelyDistributed] Strawman proposal 
for message bus analysis

Hi Paul-André,

This was without ceilometer. Nova + Neutron were consuming a lot of 
connections.
Some charts are available in the Barcelona presentation[1] and the 
performance docs[2].
In the latter you'll find some telemetry related tests.

[1]: 
https://www.openstack.org/assets/presentation-media/Chasing-1000-nodes-scale.pdf
[2]: https://docs.openstack.org/developer/performance-docs/

Best,

Matt

- Mail original -
> De: "Paul-Andre Raymond" <paul-andre.raym...@nexius.com>
> À: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org>
    > Envoyé: Vendredi 30 Juin 2017 18:42:04
    > Objet: Re: [openstack-dev] [FEMDC][MassivelyDistributed] Strawman 
proposal for message bus analysis
> 
> Hi Matthieu,
> 
> You mentioned 15000 connections with 1000 compute nodes.
> Was that mostly Nova? Was ceilometer involved?
> I would be curious to know how much AMQP traffic is Control 
related
> (e.g. spinning up VMs) vs how much is telemetry related in a 
typical
> openstack deployment.
> Do we know that?
> 
> I have also left some comments in the doc.
> 
> Paul-Andre
> 
> 
> -Original Message-
> From: Matthieu Simonin <matthieu.simo...@inria.fr>
> Reply-To: "OpenStack Development Mailing List (not for usage 
questions)"
> <openstack-dev@lists.openstack.org>
> Date: Wednesday, June 21, 2017 at 6:54 PM
> To: "OpenStack Development Mailing List (not for usage questions)"
> <openstack-dev@lists.openstack.org>
> Subject: Re: [openstack-dev] [FEMDC][MassivelyDistributed] Strawman
> proposal for  message bus analysis
> 
> Hi Ken,
> 
> Thanks for starting this !
> I've made a first pass on the epad and left some notes and 
questions
> there.
> 
> Best,
> 
> Matthieu
> - Mail original -
> > De: "Ken Giusti" <kgiu...@gmail.com>
>     > À: "OpenStack Development Mailing List (not for usage 
questions)"
> > <openstack-dev@lists.openstack.org>
> > Envoyé: Mercredi 21 Juin 2017 15:23:26
> > Objet: [openstack-dev] [FEMDC][MassivelyDistributed] Strawman
> > proposal formessage bus analysis
> > 
> > Hi All,
> > 
> > Andy and I have taken a stab at defining some test scenarios for
> > anal the
> > different message bus technologies:
> > 
> > https://etherpad.openstack.org/p/1BGhFHDIoi
> > 
> > We've started with tests for just the oslo.messaging layer to
> > analyze
> > throughput and latency as the number of message bus clients - 
and
> > the bus
> > itself - scale out.
> > 
> > The next step will be to define messaging oriented test 
scenarios
> > for an
> > openstack deployment.  We've started by enumerating a few of the
> > tools,
> > topologies, and fault conditions that need to be covered.
> > 
> > Let's use this epad as a starting poin

Re: [openstack-dev] [FEMDC][MassivelyDistributed] Strawman proposal for message bus analysis

[Matthieu Simonin]

Hi Paul-André,

This was without ceilometer. Nova + Neutron were consuming a lot of connections.
Some charts are available in the Barcelona presentation[1] and the performance 
docs[2].
In the latter you'll find some telemetry related tests.

[1]: 
https://www.openstack.org/assets/presentation-media/Chasing-1000-nodes-scale.pdf
[2]: https://docs.openstack.org/developer/performance-docs/

Best,

Matt

- Mail original -
> De: "Paul-Andre Raymond" <paul-andre.raym...@nexius.com>
> À: "OpenStack Development Mailing List (not for usage questions)" 
> <openstack-dev@lists.openstack.org>
> Envoyé: Vendredi 30 Juin 2017 18:42:04
> Objet: Re: [openstack-dev] [FEMDC][MassivelyDistributed] Strawman proposal 
> for message bus analysis
> 
> Hi Matthieu,
> 
> You mentioned 15000 connections with 1000 compute nodes.
> Was that mostly Nova? Was ceilometer involved?
> I would be curious to know how much AMQP traffic is Control related
> (e.g. spinning up VMs) vs how much is telemetry related in a typical
> openstack deployment.
> Do we know that?
> 
> I have also left some comments in the doc.
> 
> Paul-Andre
> 
> 
> -Original Message-
> From: Matthieu Simonin <matthieu.simo...@inria.fr>
> Reply-To: "OpenStack Development Mailing List (not for usage questions)"
> <openstack-dev@lists.openstack.org>
> Date: Wednesday, June 21, 2017 at 6:54 PM
> To: "OpenStack Development Mailing List (not for usage questions)"
>     <openstack-dev@lists.openstack.org>
> Subject: Re: [openstack-dev] [FEMDC][MassivelyDistributed] Strawman
> proposal for  message bus analysis
> 
> Hi Ken,
> 
> Thanks for starting this !
> I've made a first pass on the epad and left some notes and questions
> there.
> 
> Best,
> 
> Matthieu
> - Mail original -
> > De: "Ken Giusti" <kgiu...@gmail.com>
> > À: "OpenStack Development Mailing List (not for usage questions)"
> > <openstack-dev@lists.openstack.org>
> > Envoyé: Mercredi 21 Juin 2017 15:23:26
> > Objet: [openstack-dev] [FEMDC][MassivelyDistributed] Strawman
> > proposal formessage bus analysis
> > 
> > Hi All,
> > 
> > Andy and I have taken a stab at defining some test scenarios for
> > anal the
> > different message bus technologies:
> > 
> > https://etherpad.openstack.org/p/1BGhFHDIoi
> > 
> > We've started with tests for just the oslo.messaging layer to
> > analyze
> > throughput and latency as the number of message bus clients - and
> > the bus
> > itself - scale out.
> > 
> > The next step will be to define messaging oriented test scenarios
> > for an
> > openstack deployment.  We've started by enumerating a few of the
> > tools,
> > topologies, and fault conditions that need to be covered.
> > 
> > Let's use this epad as a starting point for analyzing messaging -
> > please
> > feel free to contribute, question, and criticize :)
> > 
> > thanks,
> > 
> > 
> > 
> > --
> > Ken Giusti  (kgiu...@gmail.com)
> > 
> > 
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > 
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [FEMDC][MassivelyDistributed] Strawman proposal for message bus analysis

[Paul-Andre Raymond]

Hi Matthieu,

You mentioned 15000 connections with 1000 compute nodes.
Was that mostly Nova? Was ceilometer involved?
I would be curious to know how much AMQP traffic is Control related 
(e.g. spinning up VMs) vs how much is telemetry related in a typical openstack 
deployment.
Do we know that?

I have also left some comments in the doc.

Paul-Andre


-Original Message-
From: Matthieu Simonin <matthieu.simo...@inria.fr>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org>
Date: Wednesday, June 21, 2017 at 6:54 PM
To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org>
    Subject: Re: [openstack-dev] [FEMDC][MassivelyDistributed] Strawman 
proposal formessage bus analysis

Hi Ken,

Thanks for starting this !
I've made a first pass on the epad and left some notes and questions 
there.

Best,

Matthieu
- Mail original -
> De: "Ken Giusti" <kgiu...@gmail.com>
> À: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org>
        > Envoyé: Mercredi 21 Juin 2017 15:23:26
    > Objet: [openstack-dev] [FEMDC][MassivelyDistributed] Strawman 
proposal formessage bus analysis
> 
> Hi All,
> 
> Andy and I have taken a stab at defining some test scenarios for anal 
the
> different message bus technologies:
> 
> https://etherpad.openstack.org/p/1BGhFHDIoi
> 
> We've started with tests for just the oslo.messaging layer to analyze
> throughput and latency as the number of message bus clients - and the 
bus
> itself - scale out.
> 
> The next step will be to define messaging oriented test scenarios for 
an
> openstack deployment.  We've started by enumerating a few of the 
tools,
> topologies, and fault conditions that need to be covered.
> 
> Let's use this epad as a starting point for analyzing messaging - 
please
> feel free to contribute, question, and criticize :)
> 
> thanks,
> 
> 
> 
> --
> Ken Giusti  (kgiu...@gmail.com)
> 
> 
__
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [FEMDC][MassivelyDistributed] Strawman proposal for message bus analysis

[Ken Giusti]

On Wed, Jun 21, 2017 at 10:13 AM, Ilya Shakhat  wrote:

> Hi Ken,
>
> Please check scenarios and reports that exist in Performance Docs. In
> particular you may be interested in:
>  * O.M.Simulator - https://github.com/openstack/o
> slo.messaging/blob/master/tools/simulator.py
>  * MQ  performance scenario - https://docs.openstack.org/dev
> eloper/performance-docs/test_plans/mq/plan.html#message-queue-performance
>  * One of RabbitMQ reports - https://docs.openstack.org/dev
> eloper/performance-docs/test_results/mq/rabbitmq/cmsm/index.html
>  * MQ HA scenario - https://docs.openstack.org/dev
> eloper/performance-docs/test_plans/mq_ha/plan.html
>  * One of RabbitMQ HA reports - https://docs.openstack.org/dev
> eloper/performance-docs/test_results/mq_ha/rabbitmq-ha-
> queues/cs1ss2-ks2-ha/omsimulator-ha-call-cs1ss2-ks2-ha/index.html
>
>
Thank you Ilya - these tests you reference are indeed valuable.

But, IIUC, those tests benchmark queue throughput, using a single (highly
threaded) client->single server traffic flow.   If that is the case, I
think the tests we're trying to define might be a bit more specific to the
FEMDC [0] use cases:  multiple servers consuming from different topics
while many clients distributed across the message bus are connecting,
generating traffic, failing over, etc.

The goal of these tests would be to quantify the behavior of the message
bus as a whole under different messaging loads, failure conditions, etc.

[0] https://wiki.openstack.org/wiki/Fog_Edge_Massively_Distributed_Clouds




>
> Thanks,
> Ilya
>
> 2017-06-21 15:23 GMT+02:00 Ken Giusti :
>
>> Hi All,
>>
>> Andy and I have taken a stab at defining some test scenarios for anal the
>> different message bus technologies:
>>
>> https://etherpad.openstack.org/p/1BGhFHDIoi
>>
>> We've started with tests for just the oslo.messaging layer to analyze
>> throughput and latency as the number of message bus clients - and the bus
>> itself - scale out.
>>
>> The next step will be to define messaging oriented test scenarios for an
>> openstack deployment.  We've started by enumerating a few of the tools,
>> topologies, and fault conditions that need to be covered.
>>
>> Let's use this epad as a starting point for analyzing messaging - please
>> feel free to contribute, question, and criticize :)
>>
>> thanks,
>>
>>
>>
>> --
>> Ken Giusti  (kgiu...@gmail.com)
>>
>> 
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscrib
>> e
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Ken Giusti  (kgiu...@gmail.com)
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [FEMDC][MassivelyDistributed] Strawman proposal for message bus analysis

[Matthieu Simonin]

Hi Ken,

Thanks for starting this !
I've made a first pass on the epad and left some notes and questions there.

Best,

Matthieu
- Mail original -
> De: "Ken Giusti" <kgiu...@gmail.com>
> À: "OpenStack Development Mailing List (not for usage questions)" 
> <openstack-dev@lists.openstack.org>
> Envoyé: Mercredi 21 Juin 2017 15:23:26
> Objet: [openstack-dev] [FEMDC][MassivelyDistributed] Strawman proposal for
> message bus analysis
> 
> Hi All,
> 
> Andy and I have taken a stab at defining some test scenarios for anal the
> different message bus technologies:
> 
> https://etherpad.openstack.org/p/1BGhFHDIoi
> 
> We've started with tests for just the oslo.messaging layer to analyze
> throughput and latency as the number of message bus clients - and the bus
> itself - scale out.
> 
> The next step will be to define messaging oriented test scenarios for an
> openstack deployment.  We've started by enumerating a few of the tools,
> topologies, and fault conditions that need to be covered.
> 
> Let's use this epad as a starting point for analyzing messaging - please
> feel free to contribute, question, and criticize :)
> 
> thanks,
> 
> 
> 
> --
> Ken Giusti  (kgiu...@gmail.com)
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [FEMDC][MassivelyDistributed] Strawman proposal for message bus analysis

[Jay Pipes]

On 06/21/2017 11:31 AM, Ken Giusti wrote:
On Wed, Jun 21, 2017 at 11:24 AM, Jay Pipes > wrote:


On 06/21/2017 09:23 AM, Ken Giusti wrote:

Andy and I have taken a stab at defining some test scenarios for
anal the different message bus...


That was a particularly unfortunatey choice of words.

Ugh. Sorry - most unfortunate fat-finger...or Freudian slip...


LOL, no worries. It made my morning so far.

Best,
-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [FEMDC][MassivelyDistributed] Strawman proposal for message bus analysis

[Ken Giusti]

On Wed, Jun 21, 2017 at 11:24 AM, Jay Pipes  wrote:

> On 06/21/2017 09:23 AM, Ken Giusti wrote:
>
>> Andy and I have taken a stab at defining some test scenarios for anal the
>> different message bus...
>>
>
> That was a particularly unfortunatey choice of words.
>
>
Ugh. Sorry - most unfortunate fat-finger...or Freudian slip...


>
> Best,
> -jay
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Ken Giusti  (kgiu...@gmail.com)
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [FEMDC][MassivelyDistributed] Strawman proposal for message bus analysis

[Jay Pipes]

On 06/21/2017 09:23 AM, Ken Giusti wrote:
Andy and I have taken a stab at defining some test scenarios for anal 
the different message bus...


That was a particularly unfortunate choice of words.

Best,
-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [FEMDC][MassivelyDistributed] Strawman proposal for message bus analysis

[Jay Pipes]

On 06/21/2017 09:23 AM, Ken Giusti wrote:
Andy and I have taken a stab at defining some test scenarios for anal 
the different message bus...


That was a particularly unfortunatey choice of words.

Best,
-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [FEMDC][MassivelyDistributed] Strawman proposal for message bus analysis

[Ilya Shakhat]

Hi Ken,

Please check scenarios and reports that exist in Performance Docs. In
particular you may be interested in:
 * O.M.Simulator -
https://github.com/openstack/oslo.messaging/blob/master/tools/simulator.py
 * MQ  performance scenario -
https://docs.openstack.org/developer/performance-docs/test_plans/mq/plan.html#message-queue-performance
 * One of RabbitMQ reports -
https://docs.openstack.org/developer/performance-docs/test_results/mq/rabbitmq/cmsm/index.html
 * MQ HA scenario -
https://docs.openstack.org/developer/performance-docs/test_plans/mq_ha/plan.html
 * One of RabbitMQ HA reports -
https://docs.openstack.org/developer/performance-docs/test_results/mq_ha/rabbitmq-ha-queues/cs1ss2-ks2-ha/omsimulator-ha-call-cs1ss2-ks2-ha/index.html


Thanks,
Ilya

2017-06-21 15:23 GMT+02:00 Ken Giusti :

> Hi All,
>
> Andy and I have taken a stab at defining some test scenarios for anal the
> different message bus technologies:
>
> https://etherpad.openstack.org/p/1BGhFHDIoi
>
> We've started with tests for just the oslo.messaging layer to analyze
> throughput and latency as the number of message bus clients - and the bus
> itself - scale out.
>
> The next step will be to define messaging oriented test scenarios for an
> openstack deployment.  We've started by enumerating a few of the tools,
> topologies, and fault conditions that need to be covered.
>
> Let's use this epad as a starting point for analyzing messaging - please
> feel free to contribute, question, and criticize :)
>
> thanks,
>
>
>
> --
> Ken Giusti  (kgiu...@gmail.com)
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [FEMDC][MassivelyDistributed] Strawman proposal for message bus analysis

[Ken Giusti]

Hi All,

Andy and I have taken a stab at defining some test scenarios for anal the
different message bus technologies:

https://etherpad.openstack.org/p/1BGhFHDIoi

We've started with tests for just the oslo.messaging layer to analyze
throughput and latency as the number of message bus clients - and the bus
itself - scale out.

The next step will be to define messaging oriented test scenarios for an
openstack deployment.  We've started by enumerating a few of the tools,
topologies, and fault conditions that need to be covered.

Let's use this epad as a starting point for analyzing messaging - please
feel free to contribute, question, and criticize :)

thanks,



-- 
Ken Giusti  (kgiu...@gmail.com)
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev