Hi,
Why should we create queue in advance?
Let's consider following use cases:
1)
* listener starts and creates a queue
* publishers connect to exchange and start publishing
No need to create a queue in advance here since listener does it when it
starts
2)
* publishers create a queue in advance and start publishing
....
Creation is not correct since there is no guarantee that someone would
ever use this queue...
IMHO listener should create a queue and publishers should not care about
it at all.
What do you think?
On 06/27/2014 05:16 PM, Sandy Walsh wrote:
Something to consider is the "create the queue in advance" feature is
done for notifications, so we don't drop important messages on the
floor by having an Exchange with no associated Queue.
For RPC operations, this may not be required (we assume the service is
available). If this check is truly a time-sink we could ignore that
check for rpc calls.
-S
On 6/10/2014 9:31 AM, Alexei Kornienko wrote:
Hi,
Please find some answers inline.
Regards,
Alexei
On 06/10/2014 03:06 PM, Flavio Percoco wrote:
On 10/06/14 15:03 +0400, Dina Belova wrote:
Hello, stackers!
Oslo.messaging is future of how different OpenStack components
communicate with
each other, and really I'd love to start discussion about how we
can make this
library even better then it's now and how can we refactor it make more
production-ready.
As we all remember, oslo.messaging was initially inspired to be
created as a
logical continuation of nova.rpc - as a separated library, with
lots of
transports supported, etc. That's why oslo.messaging inherited not
only
advantages of now did the nova.rpc work (and it were lots of them),
but also
some architectural decisions that currently sometimes lead to the
performance
issues (we met some of them while Ceilometer performance testing
[1] during the
Icehouse).
For instance, simple testing messaging server (with connection pool
and
eventlet) can process 700 messages per second. The same functionality
implemented using plain kombu (without connection pool and
eventlet) driver is
processing ten times more - 7000-8000 messages per second.
So we have the following suggestions about how we may make this
process better
and quicker (and really I'd love to collect your feedback, folks):
1) Currently we have main loop running in the Executor class, and I
guess it'll
be much better to move it to the Server class, as it'll make
relationship
between the classes easier and will leave Executor only one task -
process the
message and that's it (in blocking or eventlet mode). Moreover,
this will make
further refactoring much easier.
To some extent, the executors are part of the server class since the
later is the one actually controlling them. If I understood your
proposal, the server class would implement the event loop, which means
we would have an EventletServer / BlockingServer, right?
If what I said is what you meant, then I disagree. Executors keep the
eventloop isolated from other parts of the library and this is really
important for us. One of the reason is to easily support multiple
python versions - by having different event loops.
Is my assumption correct? Could you elaborate more?
No It's not how we plan it. Server will do the loop and pass received
message to dispatcher and executor. It means that we would still have
blocking executor and eventlet executor in the same server class. We
would just change the implementation part to make it more consistent
and easier to control.
2) Some of the drivers implementations (such as impl_rabbit and
impl_qpid, for
instance) are full of useless separated classes that in reality
might be
included to other ones. There are already some changes making the
whole
structure easier [2], and after the 1st issue will be solved
Dispatcher and
Listener also will be able to be refactored.
This was done on purpose. The idea was to focus on backwards
compatibility rather than cleaning up/improving the drivers. That
said, sounds like those drivers could user some clean up. However, I
think we should first extend the test suite a bit more before hacking
the existing drivers.
3) If we'll separate RPC functionality and messaging functionality
it'll make
code base clean and easily reused.
What do you mean with this?
We mean that current drivers are written with RPC code hardcoded
inside (ReplyWaiter, etc.). Thats not how messaging library is
supposed to work. We can move RPC to a separate layer and this would
be beneficial for both rpc (code will become more clean and less
error-prone) and core messaging part (we'll be able to implement
messaging in way that will work much faster).
4) Connection pool can be refactored to implement more efficient
connection
reusage.
Please, elaborate. What changes do you envision?
Currently there is a class that is called ConnectionContext that is
used to manage pool. Additionaly it can be accessed/configured in
several other places. If we refactor it a little bit it would be much
easier to use connections from the pool.
As Dims suggested, I think filing some specs for this (and keeping the
proposals separate) would help a lot in understanding what the exact
plan is.
Glad to know you're looking forward to help improving oslo.messaging.
Thanks,
Flavio
Folks, are you ok with such a plan? Alexey Kornienko already
started some of
this work [2], but really we want to be sure that we chose the
correct vector
of development here.
Thanks!
[1] https://docs.google.com/document/d/
1ARpKiYW2WN94JloG0prNcLjMeom-ySVhe8fvjXG_uRU/edit?usp=sharing
[2] https://review.openstack.org/#/q/
status:open+owner:akornienko+project:openstack/oslo.messaging,n,z
Best regards,
Dina Belova
Software Engineer
Mirantis Inc.
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev