On Wed, May 6, 2015 at 10:59 AM, Alan Conway <acon...@redhat.com> wrote:
> On Thu, 2015-04-30 at 21:51 -0400, Rafael Schloming wrote: > > What sort of work/connections are you talking about here? Are you talking > > about processing AMQP messages in a thread pool or are you talking about > > writing some sort of multithreaded I/O handler? > > I'm talking about running multiple proton instances in a thread pool, > like the Qpid C++ broker does. It uses a thread safe poller (based on > epoll or somesuch) to dispatch data from many connections to a small > thread pool. The poller serializes work per connection. There is a > proton engine per connection so work to each engine is serialized but > different engines run concurrently. The worker threads feed the proton > transport, process events, and move messages to and from thread-safe > queues (which are potentially shared among all connections) > > I'm trying to understand if the C reactor can help with that. I have the > impression that the reactor requires all of the engines and their > handlers to run in a single thread. From there I could put messages into > a thread pool to process them concurrently. However the engines share no > data so there is no reason not to run them in parallel as well. > Serializing everything thru the reactor doesn't seem to buy me anything, > but it reduces concurrency and makes the reactor thread a bottleneck. > Your assumption that the engines share no state is not valid in general. Given that the application can attach arbitrary state to connections, sessions, links, deliveries, etc, it is entirely possible (and quite common) that a whole group of connections might actually be fairly intimately intertwined via shared application state. The reactor permits you to code such an application without having to worry about concurrency. On the other hand, using a reactor per connection doesn't seem to give > me anything over just using an engine per connection. > In this case it still provides a common event dispatch framework. If you write your code against the engine directly, then your code is basically going to be one big event handler for every possible event that might occur on a connection. This is fine until you start needing different behavior on different links, say you want to apply one flow control policy on one link and a different one on another link. You now either need to turn your one big event handler into spaghetti, or you need a way to associate different handlers with different links. You can of course do all this in an ad-hoc way rather than using the reactor, but the advantage of using the reactor is that by being consistent about how we do this, we can build up a library of useful handlers that can work well together. --Rafael
