[jira] [Comment Edited] (PROTON-1071) EventInjector hangs on Windows
[ https://issues.apache.org/jira/browse/PROTON-1071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15046607#comment-15046607 ] Gordon Sim edited comment on PROTON-1071 at 12/8/15 10:02 AM: -- There is no concept in Windows of a general select/poll that works on generic file descriptors/handles. You have a separate synchronous wait model for each file type. On top of that, you have a fancy proactor model (IO completion ports, IOCP) for asynchronous IO on a few file types: sockets and named pipes, but not console IO and anonymous pipes. The IOCP model is the preferred scalable IO mechanism for one or many threads. If using pn_selectable_set_fd(handle_xx) with IOCP, it cannot work for any handle_xx that is not IOCP aware. Well not without adding an extra thread that synchronously checks it (something that may have to be done for console IO). PROTON-668 was an attempt to find a useful subset of Posix and Windows capabilities (and other platforms, if identified) for performant IO (mostly looking at the dispatch router model). Threading consequences were one consideration, but so was mere achievability. The network (socket) IO plus pipe (interrupt mainly) IO were the prime candidates. Hence an "fd" in pn_selectable_set_fd is actually a pn_socket_t, not an int. An external socket may be used in this case, but also a pn_pipe object. In Posix, the pn_pipe is not an OS socket. In Windows, it currently is but may become something else, perhaps even an internal construct in future. Another difference to note, a Posix fd can move between different collections fds for poll/select. A Windows handle can be registered exactly once with a collection. It can never be moved to another selector. It is associated with a single IOCP port until it is closed. was (Author: cliffjansen): There is no concept in Windows of a general select/poll that works on generic file descriptors/handles. You have a separate synchronous wait model for each file type. On top of that, you have a fancy proactor model (IO completion ports, IOCP) for asynchronous IO on a few file types: sockets and named pipes, but not console IO and anonymous pipes. The IOCP model is the preferred scalable IO mechanism for one or many threads. If using pn_selectable_set_fd(handle_xx) with IOCP, it cannot work for any handle_xx that is not IOCP aware. Well not without adding an extra thread that synchronously checks it (something that may have to be done for console IO). PROTON-688 was an attempt to find a useful subset of Posix and Windows capabilities (and other platforms, if identified) for performant IO (mostly looking at the dispatch router model). Threading consequences were one consideration, but so was mere achievability. The network (socket) IO plus pipe (interrupt mainly) IO were the prime candidates. Hence an "fd" in pn_selectable_set_fd is actually a pn_socket_t, not an int. An external socket may be used in this case, but also a pn_pipe object. In Posix, the pn_pipe is not an OS socket. In Windows, it currently is but may become something else, perhaps even an internal construct in future. Another difference to note, a Posix fd can move between different collections fds for poll/select. A Windows handle can be registered exactly once with a collection. It can never be moved to another selector. It is associated with a single IOCP port until it is closed. > EventInjector hangs on Windows > -- > > Key: PROTON-1071 > URL: https://issues.apache.org/jira/browse/PROTON-1071 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c, python-binding >Affects Versions: 0.11 > Environment: Windows >Reporter: Ken Giusti >Assignee: Cliff Jansen > Fix For: 0.12.0 > > > I added a new reactor test that exercises the python-proton ApplicationEvent > and EventInjector classes: > proton_tests.reactor.ApplicationEventTest.test_application_events > See tests/python/proton_tests/reactor.py > This test passes on linux, but hangs when run on Windows. > Poking around a bit, I suspect the problem may be in the Windows selector > code. Description: > The EventInjector/ApplicationEvent classes provide a way to trigger events > from threads external to the reactor main loop. See > proton-c/bindings/python/proton/reactor.py. A pipe is used to wake up the > reactor when a new event is sent to the reactor (see reactor.py in the python > bindings). The EventInjector's trigger method puts the event on a queue and > writes to a pipe to wake up the reactor. The on_selectable_readable callback > in the EventInjector is called on the reactor thread to get the event off the > queue and clear the pipe. > On windows it appears as if the EventInjector selectable is made "readable"
[jira] [Comment Edited] (PROTON-1071) EventInjector hangs on Windows
[ https://issues.apache.org/jira/browse/PROTON-1071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15045851#comment-15045851 ] Gordon Sim edited comment on PROTON-1071 at 12/8/15 10:03 AM: -- This JIRA cuts across a number of Proton IO and threading issues. EventInjector as coded violates Proton's io model: the pipe needs to come from pn_pipe() and the reading and writing via pn_read() and pn_write(), all on the reactor's pn_io_t object. It works by fortuitous accident on Posix. Coding EventInjector to work this way for Posix is presumably simple. For Windows, it further requires tackling deferred thread safety, at least for pipes and the associated pn_io_t->selector (but that may pull in everything). As noted in PROTON-640, there is no thread safety in Proton io on Windows other than a weak guarantee that two separate threads may independently work on separate pn_io_t objects. PROTON-668 attempted to document the stronger but still limited thread-safety available in the Posix implementation of Proton io. This is the model of concurrency used by Qpid Dispatch Router and perhaps the model that the Windows implementation should strive to. There is an opposing point of view that pushing thread safety into Proton is counterproductive: applications know what threading/io model works best for their workload. Hence the rising interest in connection_engine. EventInjector seems like a pretty important use case for coordinating threads. Alternatively, a more limited (or dedicated) api extension, perhaps pn_reactor_inject() which does platform-specific thread-safe coordination of something to be collected and the pn_io_t->selector might be more clear in its intention (as opposed to knowing that pn_write/read/select work a certain way together). The Windows io.c and related code could be made to work at the same level of thread safety as Posix, but with obvious locking overhead. If the assumption is that an application needing high performance IO will just use the connection_engine and manage the IO itself, perhaps there is no need to care about the locking penalty. See also PROTON-889 regarding general thread safety outages in Proton on all platforms for pn_io_error() and pn_io_wouldblock(). was (Author: cliffjansen): This JIRA cuts across a number of Proton IO and threading issues. EventInjector as coded violates Proton's io model: the pipe needs to come from pn_pipe() and the reading and writing via pn_read() and pn_write(), all on the reactor's pn_io_t object. It works by fortuitous accident on Posix. Coding EventInjector to work this way for Posix is presumably simple. For Windows, it further requires tackling deferred thread safety, at least for pipes and the associated pn_io_t->selector (but that may pull in everything). As noted in PROTON-640, there is no thread safety in Proton io on Windows other than a weak guarantee that two separate threads may independently work on separate pn_io_t objects. PROTON-688 attempted to document the stronger but still limited thread-safety available in the Posix implementation of Proton io. This is the model of concurrency used by Qpid Dispatch Router and perhaps the model that the Windows implementation should strive to. There is an opposing point of view that pushing thread safety into Proton is counterproductive: applications know what threading/io model works best for their workload. Hence the rising interest in connection_engine. EventInjector seems like a pretty important use case for coordinating threads. Alternatively, a more limited (or dedicated) api extension, perhaps pn_reactor_inject() which does platform-specific thread-safe coordination of something to be collected and the pn_io_t->selector might be more clear in its intention (as opposed to knowing that pn_write/read/select work a certain way together). The Windows io.c and related code could be made to work at the same level of thread safety as Posix, but with obvious locking overhead. If the assumption is that an application needing high performance IO will just use the connection_engine and manage the IO itself, perhaps there is no need to care about the locking penalty. See also PROTON-889 regarding general thread safety outages in Proton on all platforms for pn_io_error() and pn_io_wouldblock(). > EventInjector hangs on Windows > -- > > Key: PROTON-1071 > URL: https://issues.apache.org/jira/browse/PROTON-1071 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c, python-binding >Affects Versions: 0.11 > Environment: Windows >Reporter: Ken Giusti >Assignee: Cliff Jansen > Fix For: 0.12.0 > > > I added a new reactor test that exercises the python-proton ApplicationEvent > and EventInjector
