Re: [openstack-dev] [oslo][messaging] Further improvements and refactoring

2014-07-02 Thread Ihar Hrachyshka
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 01/07/14 17:14, Alexei Kornienko wrote: Hi, Please see some minor comments inline. Do you think we can schedule some time to discuss this topic on one of the upcoming meetings? We can come out with some kind of the summary and actions plan

Re: [openstack-dev] [oslo][messaging] Further improvements and refactoring

2014-07-02 Thread Gordon Sim
On 07/01/2014 03:52 PM, Ihar Hrachyshka wrote: On 01/07/14 15:55, Alexei Kornienko wrote: And in addition I've provided some links to existing implementation with places that IMHO cause bottlenecks. From my point of view that code is doing obviously stupid things (like closing/opening sockets

Re: [openstack-dev] [oslo][messaging] Further improvements and refactoring

2014-07-02 Thread Gordon Sim
On 07/01/2014 04:14 PM, Alexei Kornienko wrote: A lot of driver details leak outside the API and it makes it hard to improve driver without changing the API. I agree that some aspects of specific driver implementations leak into the public API for the messaging library as a whole. There are

Re: [openstack-dev] [oslo][messaging] Further improvements and refactoring

2014-07-02 Thread Doug Hellmann
On Wed, Jul 2, 2014 at 5:46 AM, Ihar Hrachyshka ihrac...@redhat.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 01/07/14 17:14, Alexei Kornienko wrote: Hi, Please see some minor comments inline. Do you think we can schedule some time to discuss this topic on one of the

Re: [openstack-dev] [oslo][messaging] Further improvements and refactoring

2014-07-01 Thread Ihar Hrachyshka
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 30/06/14 21:34, Alexei Kornienko wrote: Hello, My understanding is that your analysis is mostly based on running a profiler against the code. Network operations can be bottlenecked in other places. You compare 'simple script using

Re: [openstack-dev] [oslo][messaging] Further improvements and refactoring

2014-07-01 Thread Alexei Kornienko
Hi, Thanks for detailed answer. Please see my comments inline. Regards, On 07/01/2014 04:28 PM, Ihar Hrachyshka wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 30/06/14 21:34, Alexei Kornienko wrote: Hello, My understanding is that your analysis is mostly based on running a

Re: [openstack-dev] [oslo][messaging] Further improvements and refactoring

2014-07-01 Thread Ihar Hrachyshka
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 01/07/14 15:55, Alexei Kornienko wrote: Hi, Thanks for detailed answer. Please see my comments inline. Regards, On 07/01/2014 04:28 PM, Ihar Hrachyshka wrote: On 30/06/14 21:34, Alexei Kornienko wrote: Hello, My understanding is

Re: [openstack-dev] [oslo][messaging] Further improvements and refactoring

2014-07-01 Thread Alexei Kornienko
Hi, Please see some minor comments inline. Do you think we can schedule some time to discuss this topic on one of the upcoming meetings? We can come out with some kind of the summary and actions plan to start working on. Regards, On 07/01/2014 05:52 PM, Ihar Hrachyshka wrote: -BEGIN

Re: [openstack-dev] [oslo][messaging] Further improvements and refactoring

2014-06-30 Thread Ihar Hrachyshka
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 06/27/2014 04:04 PM, Ihar Hrachyshka wrote: On 26/06/14 22:38, Alexei Kornienko wrote: Hello Jay, Benchmark for oslo.messaging is really simple: You create a client that sends messages infinitively and a server that processes them. After

Re: [openstack-dev] [oslo][messaging] Further improvements and refactoring

2014-06-30 Thread Gordon Sim
On 06/30/2014 12:22 PM, Ihar Hrachyshka wrote: Alexei Kornienko wrote: Some performance tests may be introduced but they would be more like functional tests since they require setup of actual messaging server (rabbit, etc.). Yes. I think we already have some. F.e.

Re: [openstack-dev] [oslo][messaging] Further improvements and refactoring

2014-06-30 Thread Ben Nemec
I'm far from an oslo.messaging expert, but a few general thoughts below. On 06/30/2014 02:34 PM, Alexei Kornienko wrote: Hello, My understanding is that your analysis is mostly based on running a profiler against the code. Network operations can be bottlenecked in other places. You

Re: [openstack-dev] [oslo][messaging] Further improvements and refactoring

2014-06-27 Thread Ihar Hrachyshka
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 26/06/14 22:38, Alexei Kornienko wrote: Hello Jay, Benchmark for oslo.messaging is really simple: You create a client that sends messages infinitively and a server that processes them. After you can use rabbitmq management plugin to see

Re: [openstack-dev] [oslo][messaging] Further improvements and refactoring

2014-06-27 Thread Alexei Kornienko
Hello, Please find some answers inline. Regards, Alexei On 06/27/2014 04:04 PM, Ihar Hrachyshka wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 26/06/14 22:38, Alexei Kornienko wrote: Hello Jay, Benchmark for oslo.messaging is really simple: You create a client that sends

Re: [openstack-dev] [oslo][messaging] Further improvements and refactoring

2014-06-27 Thread Sandy Walsh
Something to consider is the create the queue in advance feature is done for notifications, so we don't drop important messages on the floor by having an Exchange with no associated Queue. For RPC operations, this may not be required (we assume the service is available). If this check is truly

Re: [openstack-dev] [oslo][messaging] Further improvements and refactoring

2014-06-27 Thread Alexei Kornienko
Hi, Why should we create queue in advance? Let's consider following use cases: 1) * listener starts and creates a queue * publishers connect to exchange and start publishing No need to create a queue in advance here since listener does it when it starts 2) * publishers create a queue in

Re: [openstack-dev] [oslo][messaging] Further improvements and refactoring

2014-06-27 Thread Gordon Sim
On 06/26/2014 09:38 PM, Alexei Kornienko wrote: Benchmark for oslo.messaging is really simple: You create a client that sends messages infinitively and a server that processes them. After you can use rabbitmq management plugin to see average throughput in the queue. Simple example can be found

Re: [openstack-dev] [oslo][messaging] Further improvements and refactoring

2014-06-27 Thread Sandy Walsh
On 6/27/2014 11:27 AM, Alexei Kornienko wrote: Hi, Why should we create queue in advance? Notifications are used for communicating with downstream systems (which may or may not be online at the time). This includes dashboards, monitoring systems, billing systems, etc. They can't afford to lose

Re: [openstack-dev] [oslo][messaging] Further improvements and refactoring

2014-06-26 Thread Alexei Kornienko
Hello, Returning to performance issues of oslo.messaging. I've found 2 biggest issue in existing implementation of rabbit (kombu) driver: 1) For almost every message sent/received a new object of Consumer/Publisher class is created. And each object of this class tries to declare it's queue

Re: [openstack-dev] [oslo][messaging] Further improvements and refactoring

2014-06-16 Thread Gordon Sim
On 06/13/2014 02:06 PM, Ihar Hrachyshka wrote: On 10/06/14 15:40, Alexei Kornienko wrote: On 06/10/2014 03:59 PM, Gordon Sim wrote: I think there could be a lot of work required to significantly improve that driver, and I wonder if that would be better spent on e.g. the AMQP 1.0 driver which

Re: [openstack-dev] [oslo][messaging] Further improvements and refactoring

2014-06-13 Thread Ihar Hrachyshka
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 10/06/14 15:40, Alexei Kornienko wrote: On 06/10/2014 03:59 PM, Gordon Sim wrote: On 06/10/2014 12:03 PM, Dina Belova wrote: Hello, stackers! Oslo.messaging is future of how different OpenStack components communicate with each other,

Re: [openstack-dev] [oslo][messaging] Further improvements and refactoring

2014-06-11 Thread Mark McLoughlin
Hi, On Tue, 2014-06-10 at 15:47 +0400, Dina Belova wrote: Dims, No problem with creating the specs, we just want to understand if the community is OK with our suggestions in general :) If so, I'll create the appropriate specs and we'll discuss them :) Personally, I find it difficult to

[openstack-dev] [oslo][messaging] Further improvements and refactoring

2014-06-10 Thread Dina Belova
Hello, stackers! Oslo.messaging is future of how different OpenStack components communicate with each other, and really I’d love to start discussion about how we can make this library even better then it’s now and how can we refactor it make more production-ready. As we all remember,

Re: [openstack-dev] [oslo][messaging] Further improvements and refactoring

2014-06-10 Thread Davanum Srinivas
Dina, Alexey, Do you mind filing some spec(s) please? http://markmail.org/message/yqhndsr3zrqcfwq4 http://markmail.org/message/kpk35uikcnodq3jb thanks, dims On Tue, Jun 10, 2014 at 7:03 AM, Dina Belova dbel...@mirantis.com wrote: Hello, stackers! Oslo.messaging is future of how different

Re: [openstack-dev] [oslo][messaging] Further improvements and refactoring

2014-06-10 Thread Dina Belova
Dims, No problem with creating the specs, we just want to understand if the community is OK with our suggestions in general :) If so, I'll create the appropriate specs and we'll discuss them :) Thanks -- Dina On Tue, Jun 10, 2014 at 3:31 PM, Davanum Srinivas dava...@gmail.com wrote: Dina,

Re: [openstack-dev] [oslo][messaging] Further improvements and refactoring

2014-06-10 Thread Flavio Percoco
On 10/06/14 15:03 +0400, Dina Belova wrote: Hello, stackers! Oslo.messaging is future of how different OpenStack components communicate with each other, and really I’d love to start discussion about how we can make this library even better then it’s now and how can we refactor it make more

Re: [openstack-dev] [oslo][messaging] Further improvements and refactoring

2014-06-10 Thread Alexei Kornienko
Hi, Please find some answers inline. Regards, Alexei On 06/10/2014 03:06 PM, Flavio Percoco wrote: On 10/06/14 15:03 +0400, Dina Belova wrote: Hello, stackers! Oslo.messaging is future of how different OpenStack components communicate with each other, and really I'd love to start

Re: [openstack-dev] [oslo][messaging] Further improvements and refactoring

2014-06-10 Thread Gordon Sim
On 06/10/2014 12:03 PM, Dina Belova wrote: Hello, stackers! Oslo.messaging is future of how different OpenStack components communicate with each other, and really I’d love to start discussion about how we can make this library even better then it’s now and how can we refactor it make more

Re: [openstack-dev] [oslo][messaging] Further improvements and refactoring

2014-06-10 Thread Alexei Kornienko
On 06/10/2014 03:59 PM, Gordon Sim wrote: On 06/10/2014 12:03 PM, Dina Belova wrote: Hello, stackers! Oslo.messaging is future of how different OpenStack components communicate with each other, and really I’d love to start discussion about how we can make this library even better then it’s