On Sat, 2009-04-25 at 13:24 -0400, gregory james marsh wrote: > > Andrew, > > Thanks for the response to my post. > > What would be the consequences of me not registering the FD of a > Rdma::AsynchIO with the Poller? > > Instead I would have Rdma::AsynchIO internally poll it's own completion > queue (CQ) for the first message and then call the > "Rdma::AsynchIO.DataEvent -> Rdma::AsynchIO.ProcessCompletion -> > QueuePair.GetNextEvent" chain that the Dispatcher/Poller would call. When > that's complete Rdma::AsynchIO would go back to polling for another first > message and so on. > > Since each Rdma::AsynchIO would be handling its own polling, message flow > would be serialized via the CQ.
I think this appraoch would work. A concern is that in a broker that has multiple transports active busy waiting such as you are describing would wastefully steal resources away from other transports and activities. If you only need RDMA this might not be a concern, but it doesn't sound like a good approach for a more general broker that has to support TCP, etc. > You mention that the Poller "...serializes all events occurring on a given > registered Handle." Out of curiosity is this being done by the scoped > locks on the Mutex DispatchHandle.stateLock? (For example in > DispatchHandle.processEvent). No, the guarantee is, in the Linux case, given by the semantics of the way epoll is used. In this case we set it up so that once an event has been triggered for an fd, events are disabled until we re-enable them again explicitly. If you busy loop poll the cq, I think this is likely to give the same semantics just so long as each cq has it's own busy loop, and they are not shared between threads. Andrew --------------------------------------------------------------------- Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:[email protected]
