[jira] [Commented] (PROTON-1071) EventInjector hangs on Windows
[ https://issues.apache.org/jira/browse/PROTON-1071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15046607#comment-15046607 ] Cliff Jansen commented on PROTON-1071: -- There is no concept in Windows of a general select/poll that works on generic file descriptors/handles. You have a separate synchronous wait model for each file type. On top of that, you have a fancy proactor model (IO completion ports, IOCP) for asynchronous IO on a few file types: sockets and named pipes, but not console IO and anonymous pipes. The IOCP model is the preferred scalable IO mechanism for one or many threads. If using pn_selectable_set_fd(handle_xx) with IOCP, it cannot work for any handle_xx that is not IOCP aware. Well not without adding an extra thread that synchronously checks it (something that may have to be done for console IO). PROTON-688 was an attempt to find a useful subset of Posix and Windows capabilities (and other platforms, if identified) for performant IO (mostly looking at the dispatch router model). Threading consequences were one consideration, but so was mere achievability. The network (socket) IO plus pipe (interrupt mainly) IO were the prime candidates. Hence an "fd" in pn_selectable_set_fd is actually a pn_socket_t, not an int. An external socket may be used in this case, but also a pn_pipe object. In Posix, the pn_pipe is not an OS socket. In Windows, it currently is but may become something else, perhaps even an internal construct in future. Another difference to note, a Posix fd can move between different collections fds for poll/select. A Windows handle can be registered exactly once with a collection. It can never be moved to another selector. It is associated with a single IOCP port until it is closed. > EventInjector hangs on Windows > -- > > Key: PROTON-1071 > URL: https://issues.apache.org/jira/browse/PROTON-1071 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c, python-binding >Affects Versions: 0.11 > Environment: Windows >Reporter: Ken Giusti >Assignee: Cliff Jansen > Fix For: 0.12.0 > > > I added a new reactor test that exercises the python-proton ApplicationEvent > and EventInjector classes: > proton_tests.reactor.ApplicationEventTest.test_application_events > See tests/python/proton_tests/reactor.py > This test passes on linux, but hangs when run on Windows. > Poking around a bit, I suspect the problem may be in the Windows selector > code. Description: > The EventInjector/ApplicationEvent classes provide a way to trigger events > from threads external to the reactor main loop. See > proton-c/bindings/python/proton/reactor.py. A pipe is used to wake up the > reactor when a new event is sent to the reactor (see reactor.py in the python > bindings). The EventInjector's trigger method puts the event on a queue and > writes to a pipe to wake up the reactor. The on_selectable_readable callback > in the EventInjector is called on the reactor thread to get the event off the > queue and clear the pipe. > On windows it appears as if the EventInjector selectable is made "readable" > even though nothing has been written to the pipe. This causes the os.read() > call in the on_selectable_readable() callback to hang. > Best I can tell the windows selector code doesn't work properly with a pipe. > The pn_selector_next() function is returning a read event on the pipe's read > descriptor even though the pipe is empty. But I'm not familiar with the > window's selector implementation, so this is a best guess. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-1071) EventInjector hangs on Windows
[ https://issues.apache.org/jira/browse/PROTON-1071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15046671#comment-15046671 ] Gordon Sim commented on PROTON-1071: In examples/python/reactor/cat.py a selectable is configured to use a normal file that is opened for reading. This is preceded by the comment: '# We can configure a selectable with any file descriptor we want.' I don't doubt that you are correct in all your points about the windows implementation, but I don't agree that EventInjector violates any clearly defined model or that it works on posix only by fortuitous accident. It seems to me that the design as originally conceived did not take windows into account. If the implication is that the reactor - proton's event loop - can only be used with proton objects and operations, that to me seems like a serious limitation. All that said, focusing solely on the functionality provided at present by EventInjector in python, what would be your suggestion for a solution that works on both windows and posix?. Preserving the API in python would be highly desirable but not mandatory. > EventInjector hangs on Windows > -- > > Key: PROTON-1071 > URL: https://issues.apache.org/jira/browse/PROTON-1071 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c, python-binding >Affects Versions: 0.11 > Environment: Windows >Reporter: Ken Giusti >Assignee: Cliff Jansen > Fix For: 0.12.0 > > > I added a new reactor test that exercises the python-proton ApplicationEvent > and EventInjector classes: > proton_tests.reactor.ApplicationEventTest.test_application_events > See tests/python/proton_tests/reactor.py > This test passes on linux, but hangs when run on Windows. > Poking around a bit, I suspect the problem may be in the Windows selector > code. Description: > The EventInjector/ApplicationEvent classes provide a way to trigger events > from threads external to the reactor main loop. See > proton-c/bindings/python/proton/reactor.py. A pipe is used to wake up the > reactor when a new event is sent to the reactor (see reactor.py in the python > bindings). The EventInjector's trigger method puts the event on a queue and > writes to a pipe to wake up the reactor. The on_selectable_readable callback > in the EventInjector is called on the reactor thread to get the event off the > queue and clear the pipe. > On windows it appears as if the EventInjector selectable is made "readable" > even though nothing has been written to the pipe. This causes the os.read() > call in the on_selectable_readable() callback to hang. > Best I can tell the windows selector code doesn't work properly with a pipe. > The pn_selector_next() function is returning a read event on the pipe's read > descriptor even though the pipe is empty. But I'm not familiar with the > window's selector implementation, so this is a best guess. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (PROTON-1071) EventInjector hangs on Windows
[ https://issues.apache.org/jira/browse/PROTON-1071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15046607#comment-15046607 ] Gordon Sim edited comment on PROTON-1071 at 12/8/15 10:02 AM: -- There is no concept in Windows of a general select/poll that works on generic file descriptors/handles. You have a separate synchronous wait model for each file type. On top of that, you have a fancy proactor model (IO completion ports, IOCP) for asynchronous IO on a few file types: sockets and named pipes, but not console IO and anonymous pipes. The IOCP model is the preferred scalable IO mechanism for one or many threads. If using pn_selectable_set_fd(handle_xx) with IOCP, it cannot work for any handle_xx that is not IOCP aware. Well not without adding an extra thread that synchronously checks it (something that may have to be done for console IO). PROTON-668 was an attempt to find a useful subset of Posix and Windows capabilities (and other platforms, if identified) for performant IO (mostly looking at the dispatch router model). Threading consequences were one consideration, but so was mere achievability. The network (socket) IO plus pipe (interrupt mainly) IO were the prime candidates. Hence an "fd" in pn_selectable_set_fd is actually a pn_socket_t, not an int. An external socket may be used in this case, but also a pn_pipe object. In Posix, the pn_pipe is not an OS socket. In Windows, it currently is but may become something else, perhaps even an internal construct in future. Another difference to note, a Posix fd can move between different collections fds for poll/select. A Windows handle can be registered exactly once with a collection. It can never be moved to another selector. It is associated with a single IOCP port until it is closed. was (Author: cliffjansen): There is no concept in Windows of a general select/poll that works on generic file descriptors/handles. You have a separate synchronous wait model for each file type. On top of that, you have a fancy proactor model (IO completion ports, IOCP) for asynchronous IO on a few file types: sockets and named pipes, but not console IO and anonymous pipes. The IOCP model is the preferred scalable IO mechanism for one or many threads. If using pn_selectable_set_fd(handle_xx) with IOCP, it cannot work for any handle_xx that is not IOCP aware. Well not without adding an extra thread that synchronously checks it (something that may have to be done for console IO). PROTON-688 was an attempt to find a useful subset of Posix and Windows capabilities (and other platforms, if identified) for performant IO (mostly looking at the dispatch router model). Threading consequences were one consideration, but so was mere achievability. The network (socket) IO plus pipe (interrupt mainly) IO were the prime candidates. Hence an "fd" in pn_selectable_set_fd is actually a pn_socket_t, not an int. An external socket may be used in this case, but also a pn_pipe object. In Posix, the pn_pipe is not an OS socket. In Windows, it currently is but may become something else, perhaps even an internal construct in future. Another difference to note, a Posix fd can move between different collections fds for poll/select. A Windows handle can be registered exactly once with a collection. It can never be moved to another selector. It is associated with a single IOCP port until it is closed. > EventInjector hangs on Windows > -- > > Key: PROTON-1071 > URL: https://issues.apache.org/jira/browse/PROTON-1071 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c, python-binding >Affects Versions: 0.11 > Environment: Windows >Reporter: Ken Giusti >Assignee: Cliff Jansen > Fix For: 0.12.0 > > > I added a new reactor test that exercises the python-proton ApplicationEvent > and EventInjector classes: > proton_tests.reactor.ApplicationEventTest.test_application_events > See tests/python/proton_tests/reactor.py > This test passes on linux, but hangs when run on Windows. > Poking around a bit, I suspect the problem may be in the Windows selector > code. Description: > The EventInjector/ApplicationEvent classes provide a way to trigger events > from threads external to the reactor main loop. See > proton-c/bindings/python/proton/reactor.py. A pipe is used to wake up the > reactor when a new event is sent to the reactor (see reactor.py in the python > bindings). The EventInjector's trigger method puts the event on a queue and > writes to a pipe to wake up the reactor. The on_selectable_readable callback > in the EventInjector is called on the reactor thread to get the event off the > queue and clear the pipe. > On windows it appears as if the EventInjector selectable is made "readable"
[jira] [Comment Edited] (PROTON-1071) EventInjector hangs on Windows
[ https://issues.apache.org/jira/browse/PROTON-1071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15045851#comment-15045851 ] Gordon Sim edited comment on PROTON-1071 at 12/8/15 10:03 AM: -- This JIRA cuts across a number of Proton IO and threading issues. EventInjector as coded violates Proton's io model: the pipe needs to come from pn_pipe() and the reading and writing via pn_read() and pn_write(), all on the reactor's pn_io_t object. It works by fortuitous accident on Posix. Coding EventInjector to work this way for Posix is presumably simple. For Windows, it further requires tackling deferred thread safety, at least for pipes and the associated pn_io_t->selector (but that may pull in everything). As noted in PROTON-640, there is no thread safety in Proton io on Windows other than a weak guarantee that two separate threads may independently work on separate pn_io_t objects. PROTON-668 attempted to document the stronger but still limited thread-safety available in the Posix implementation of Proton io. This is the model of concurrency used by Qpid Dispatch Router and perhaps the model that the Windows implementation should strive to. There is an opposing point of view that pushing thread safety into Proton is counterproductive: applications know what threading/io model works best for their workload. Hence the rising interest in connection_engine. EventInjector seems like a pretty important use case for coordinating threads. Alternatively, a more limited (or dedicated) api extension, perhaps pn_reactor_inject() which does platform-specific thread-safe coordination of something to be collected and the pn_io_t->selector might be more clear in its intention (as opposed to knowing that pn_write/read/select work a certain way together). The Windows io.c and related code could be made to work at the same level of thread safety as Posix, but with obvious locking overhead. If the assumption is that an application needing high performance IO will just use the connection_engine and manage the IO itself, perhaps there is no need to care about the locking penalty. See also PROTON-889 regarding general thread safety outages in Proton on all platforms for pn_io_error() and pn_io_wouldblock(). was (Author: cliffjansen): This JIRA cuts across a number of Proton IO and threading issues. EventInjector as coded violates Proton's io model: the pipe needs to come from pn_pipe() and the reading and writing via pn_read() and pn_write(), all on the reactor's pn_io_t object. It works by fortuitous accident on Posix. Coding EventInjector to work this way for Posix is presumably simple. For Windows, it further requires tackling deferred thread safety, at least for pipes and the associated pn_io_t->selector (but that may pull in everything). As noted in PROTON-640, there is no thread safety in Proton io on Windows other than a weak guarantee that two separate threads may independently work on separate pn_io_t objects. PROTON-688 attempted to document the stronger but still limited thread-safety available in the Posix implementation of Proton io. This is the model of concurrency used by Qpid Dispatch Router and perhaps the model that the Windows implementation should strive to. There is an opposing point of view that pushing thread safety into Proton is counterproductive: applications know what threading/io model works best for their workload. Hence the rising interest in connection_engine. EventInjector seems like a pretty important use case for coordinating threads. Alternatively, a more limited (or dedicated) api extension, perhaps pn_reactor_inject() which does platform-specific thread-safe coordination of something to be collected and the pn_io_t->selector might be more clear in its intention (as opposed to knowing that pn_write/read/select work a certain way together). The Windows io.c and related code could be made to work at the same level of thread safety as Posix, but with obvious locking overhead. If the assumption is that an application needing high performance IO will just use the connection_engine and manage the IO itself, perhaps there is no need to care about the locking penalty. See also PROTON-889 regarding general thread safety outages in Proton on all platforms for pn_io_error() and pn_io_wouldblock(). > EventInjector hangs on Windows > -- > > Key: PROTON-1071 > URL: https://issues.apache.org/jira/browse/PROTON-1071 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c, python-binding >Affects Versions: 0.11 > Environment: Windows >Reporter: Ken Giusti >Assignee: Cliff Jansen > Fix For: 0.12.0 > > > I added a new reactor test that exercises the python-proton ApplicationEvent > and EventInjector
[jira] [Commented] (PROTON-1071) EventInjector hangs on Windows
[ https://issues.apache.org/jira/browse/PROTON-1071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15047264#comment-15047264 ] ASF subversion and git services commented on PROTON-1071: - Commit 19a30357297f9d637f1d9f4e67d4de6191907771 in qpid-proton's branch refs/heads/master from Clifford Jansen [ https://git-wip-us.apache.org/repos/asf?p=qpid-proton.git;h=19a3035 ] PROTON-1071: disable ApplicationEventTest temporarily on Windows until fixed. > EventInjector hangs on Windows > -- > > Key: PROTON-1071 > URL: https://issues.apache.org/jira/browse/PROTON-1071 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c, python-binding >Affects Versions: 0.11 > Environment: Windows >Reporter: Ken Giusti >Assignee: Cliff Jansen > Fix For: 0.12.0 > > > I added a new reactor test that exercises the python-proton ApplicationEvent > and EventInjector classes: > proton_tests.reactor.ApplicationEventTest.test_application_events > See tests/python/proton_tests/reactor.py > This test passes on linux, but hangs when run on Windows. > Poking around a bit, I suspect the problem may be in the Windows selector > code. Description: > The EventInjector/ApplicationEvent classes provide a way to trigger events > from threads external to the reactor main loop. See > proton-c/bindings/python/proton/reactor.py. A pipe is used to wake up the > reactor when a new event is sent to the reactor (see reactor.py in the python > bindings). The EventInjector's trigger method puts the event on a queue and > writes to a pipe to wake up the reactor. The on_selectable_readable callback > in the EventInjector is called on the reactor thread to get the event off the > queue and clear the pipe. > On windows it appears as if the EventInjector selectable is made "readable" > even though nothing has been written to the pipe. This causes the os.read() > call in the on_selectable_readable() callback to hang. > Best I can tell the windows selector code doesn't work properly with a pipe. > The pn_selector_next() function is returning a read event on the pipe's read > descriptor even though the pipe is empty. But I'm not familiar with the > window's selector implementation, so this is a best guess. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-1071) EventInjector hangs on Windows
[ https://issues.apache.org/jira/browse/PROTON-1071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15047552#comment-15047552 ] Cliff Jansen commented on PROTON-1071: -- I would suggest the following possibilities: 1. C API extension - Assert no IO thread safety in Proton, i.e. weaker than dispatch (except next line) - new pn_io_iterrupt(pn_io_t *io, handler); <- may be called on any thread - add interrupt event - rework EventInjector to use interrupt event 2. Make Windows pn_pipe thread safe on write side (formalize self-pipe trick) - Assert no IO thread safety in Proton, i.e. weaker than dispatch (with single exception) - pn_write on the write-side fd/handle of a pn_pipe may be called on any thread - rework EventInjector to use pn_pipe and pn_read/pn_write - Assume multi-threaded Proton applications will use their own IO and connection_engine 3. Strive for Posix level of thread safety - Make same thread safety guarantee on Windows io.c and selector.c as Posix via locking - rework EventInjector to use pn_pipe and pn_read/pn_write - Assume high performance applications will use their own IO and connection_engine - Fix API for pn_io_error and pn_io_wouldblock While not related to the issue of event injection, it is worth noting that the Windows IO could be broadened in each of the above scenarios to include filesystem handles (easy) and stdin (medium easy, but requiring an extra Proton-owned thread) and probably anything else (less easy). External unbound sockets are already supported with the caveat that once selected, they are forever bound to the Proton selector's completion port and cannot be used within IOCP elsewhere (another Proton pn_io_t or external IOCP-based selector). > EventInjector hangs on Windows > -- > > Key: PROTON-1071 > URL: https://issues.apache.org/jira/browse/PROTON-1071 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c, python-binding >Affects Versions: 0.11 > Environment: Windows >Reporter: Ken Giusti >Assignee: Cliff Jansen > Fix For: 0.12.0 > > > I added a new reactor test that exercises the python-proton ApplicationEvent > and EventInjector classes: > proton_tests.reactor.ApplicationEventTest.test_application_events > See tests/python/proton_tests/reactor.py > This test passes on linux, but hangs when run on Windows. > Poking around a bit, I suspect the problem may be in the Windows selector > code. Description: > The EventInjector/ApplicationEvent classes provide a way to trigger events > from threads external to the reactor main loop. See > proton-c/bindings/python/proton/reactor.py. A pipe is used to wake up the > reactor when a new event is sent to the reactor (see reactor.py in the python > bindings). The EventInjector's trigger method puts the event on a queue and > writes to a pipe to wake up the reactor. The on_selectable_readable callback > in the EventInjector is called on the reactor thread to get the event off the > queue and clear the pipe. > On windows it appears as if the EventInjector selectable is made "readable" > even though nothing has been written to the pipe. This causes the os.read() > call in the on_selectable_readable() callback to hang. > Best I can tell the windows selector code doesn't work properly with a pipe. > The pn_selector_next() function is returning a read event on the pipe's read > descriptor even though the pipe is empty. But I'm not familiar with the > window's selector implementation, so this is a best guess. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
