Hi stackers!
In Mitaka new oslo.messaging driver for RabbitMQ is released (pika driver).
I would like to share information about current state of pika driver and plans
for improvements
during next release cycle
Now we have driver implementation which:
1) passes all tempest tests
2) implements heartbeat mechanism properly (uses pika implementation instead of
implementing by own, like kombu driver does)
3) implements fast detection of connection problem (tcp_user_timeout usage for
sending, heartbeats for listening)
and reconnection to next available connection node (works better with
RabbitMQ cluster then kombu)
4) implements pure at-most-once message processing for RPC if retry=0 is set
for message sending
(kombu driver does not guarantee at-most-once processing even with retry=0
because uses acknowledges)
5) is about 50% faster then kombu (at least in my simple test with simulator.py
- 1 rpc server process and 1 rpc client process, each client runs 5 threads):
results for rabbit: 330.2 msg/sec
results for pika: 497.6 msg/sec
6) eats RabbitMQ a bit more then kombu (especially mandatory flag for rpc to
fail fast if nobody listen target).
Therefore in performance testing (17 rpc server processes and 17 rpc client
processes, each client runs 5 threads),
when RabbitMQ cluster is overloaded, pika driver works about 10% slower in
rpc call. My results:
results for rabbit: 3097 msg/sec
results for pika: 2825 msg/sec
but casts work faster about 17% then kombu because it is more lightweight
and RabbitMQ is not so heavy loaded in my tests:
results for rabbit: 5687 msg/sec
results for pika: 6697 msg/sec
7) implements separately notifications and rpc messaging (using different
exchanges, etc) which allows to use different configuration
for different use cases (for example durable notification messaging and not
durable rpc messaging)
Plans for future development:
1) Implement configurable message serialisation (json - current implementation,
msgpack)
2) Implement configurable connection factory (pool of connection - current
implementation, single connection)
3) Now it is impossible to perform rolling update from kombu to pika, we need
to implements some solution for cross driver rolling update
4) Polishing, bug fixing, profiling etc., as usual
P.S. Thank everyone who have been helping to develop pika driver!
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [email protected]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
