Brian Bouterse created QPID-5799:
------------------------------------
Summary: Allow qpid.messaging.endpoints.session to have all
associated receivers mointored via epoll mechanism
Key: QPID-5799
URL: https://issues.apache.org/jira/browse/QPID-5799
Project: Qpid
Issue Type: Improvement
Components: Python Client
Reporter: Brian Bouterse
Kombu [0] is a python producer/consumer software that I've written a qpid
transport[1] for. This qpid transport for Kombu, uses qpid.messaging, and has
all consuming receivers associated with a single session endpoint.
Celery[2] uses Kombu, and has an asynchronous I/O event loop, and would like to
monitor a single file descriptor for reading to know when there is any message
available on any receiver associated with the session endpoint.
Currently as a workaround, I create a pipe with one read and one write file
descriptors. I register the read file descriptor with Celery to monitor. I then
start a separate thread (Thread1) that calls next_receiver() on the session in
an infinite loop. Each time next_receiver() returns instead of fetching the
message, Thread1 writes a symbol to the file descriptor Celery is monitoring.
Celery then calls the on_readable() callback from the MainThread, which removes
a symbol from the pipe Celery is monitoring (clearing the epoll readable
attribute), and then calls next_receiver() again only this time from
MainThread, to acquire the receiver, and fetch() the message and process it.
This architecture is not ideal in two ways.
1. There is a redundant call to next_receiver() that happens with every read.
2. A separate thread to "monitor" qpid messaging should not be required, when
Celery is already capable of monitoring a single file descriptor, and can call
the read method which would use next_receiver() to drain from all receivers
associated with the session.
[0]: https://github.com/celery/kombu
[1]:
https://github.com/pulp/kombu/blob/pulp-dep-3.0.15-with-qpid/kombu/transport/qpid.py
[2]: https://github.com/celery/celery
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
