----- Original Message ----- > From: "aconway" <[email protected]> > To: [email protected] > Sent: Monday, October 26, 2015 12:23:01 PM > Subject: Re: The un-reactor > > On Thu, 2015-10-22 at 22:03 -0700, Cliff Jansen wrote: > > I'm not sure if this exactly fits your question (especially the > > multi-threaded part), but the python tornado loop examples run a > > regular reactor set of events and collectors but driven by the > > external loop. > > > > pn_iohandler is replaced with the TornadoLoophandler which knows how > > to insert Proton io selectables into the external loop and provide > > callbacks to do work when the external loop schedules activity on the > > descriptor. > > > > Aside from program configuration on startup, the user code (e.g. > > hello > > world) knows nothing about the change and runs just as if under > > Proton's own selector - the events it sees are the same. > > > > Something similar could be done with libuv in Proton-C, certainly > > with > > Posix, perhaps with some issues under Windows. > > I understand that this can be done, but why? Proton is an AMQP protocol > engine it is not a general purpose IO and event handling framework. I > see the value of proton providing a simple IO framework for isolated > proton projects when there is nothing else available. But if you > already use tornado, epoll, ACE, boost IO, the Go net package, C++11 > std::async, the proactor pattern etc. etc. then why would you want to > write code to integrate that with a single-threaded, non-standard, C > reactor? What is the reactor bringing to the party? > > You certainly want proton to handle AMQP for you, but you want to > integrate it into your existing framework, not the other way around. > The transport provides a simple bytes in/bytes out interface that is > exactly what you need, and we can make it a little more user friendly. >
Not sure if this is pertinent, but pygnus was designed with very little assumption about how the I/O (and timer handling) is achieved. Just as an example, pyngus 1) provides a container-level method for finding all Pyngus connections in the container that need I/O or have pending timers 2) The pyngus Connection wraps both the proton transport and the connection together. It provides APIs to read from and write to the connection, as well as a tick processing routine. And that's about it. It assumes the application will handle the polling of the container and the actual transfer of the I/O. The application must also handle mapping between the pyngus Connection and the particular I/O mechanism the connection uses (e.g. the proper socket). The oslo.messaging driver, for example, has a main loop that calls into the container to get all the connections needing processing, maps them to their corresponding socket and passes them to the main blocking select call. If the container returned a list of connections pending a timeout, the next timeout is used to calculate the select's timeout. For convenience pyngus includes two utility methods for use with sockets - one for reading from the socket, another for writing to the socket. Both take a pyngus connection and a socket-like object as arguments, and handles transferring data between the two. The oslo.messaging driver uses these on return from select, but pyngus proper doesn't require these methods. > The reactor integration points create more work and more overhead to > integrate with another IO framework, and if your framework is not > select-like (e.g. the go net package) then the reactor integration > points make no sense at all. > > > > > > Cliff > > > > On Thu, Oct 22, 2015 at 2:33 PM, aconway <[email protected]> wrote: > > > The proton reactor provides a complete solution for integrating > > > foreign > > > IO into a single threaded proton event loop. This is useful in > > > situations where proton is being used in isolation, there is no > > > other > > > IO handling framework available and everything is single threaded. > > > > > > However often that is not going to be the case. There are many > > > existing C and C++ libraries for managing polling of file > > > descriptors > > > and dispatching of work, and most server applications that might > > > want > > > to embed proton already have their own frameworks. > > > > > > So I'm thinking about how we can make integrating proton into an > > > existing (possibly multi-threaded) framework easier. I think all it > > > requires is a few simple functions to pull data out of a file > > > descriptor as events and push data back into a file descriptor, and > > > make this easier than directly using the collector and transport. > > > I've > > > done it in Go already but I think it could be captured in C and C++ > > > as > > > well. > > > > > > Anyone already done something like this? > > > > > > Cheers, > > > Alan. > > > > > > ------------------------------------------------------------------- > > > -- > > > To unsubscribe, e-mail: [email protected] > > > For additional commands, e-mail: [email protected] > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > -- -K --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
