Hi, Thank you Victor.
I've a development in few weeks on Windows, I'll use that with Qt for rendering. Regards. Ludovic Gasc (GMLudo) On 27 Jan 2015 18:10, "Victor Stinner" <[email protected]> wrote: > Hi, > > The ProactorEventLoop still lacks some features: UDP and signal handling. > https://docs.python.org/dev/library/asyncio-eventloops.html#windows > > add_reader() and add_writer() are not supported, but I'm not sure that > it's possible to implement them. They don't fit into IOCP design > (being notified when a read operation is done, not when you can > starting reading without blocking). > > I have pending (and working!) patches for signal handling, I should update > them. > > Victor > > 2015-01-27 17:22 GMT+01:00 Guido van Rossum <[email protected]>: > > Thanks for your work Victor! I occasionally get questions about asyncio's > > stability on Windows. I am now much more confident. > > > > On Mon, Jan 26, 2015 at 3:38 PM, Victor Stinner < > [email protected]> > > wrote: > >> > >> Hi, > >> > >> I spent last weeks on fixing issues specific to the Windows > >> ProactorEventLoop. Even if the code "was working" in most cases, > >> sometimes, I noticed strange warnings, bugs or crashs. Good news: all > >> known issues are now fixed, and the test suite now pass and is stable! > >> > >> Please test ProactorEventLoop as much as possible! My changes are > >> merged in the development versions of Tulip, Trollius, Python 3.4 and > >> Python 3.5. > >> > >> I added new tests. It should reduce the risk of regression. > >> > >> By the way, ProactorEventLoop now supports SSL on Python 3.5 and newer! > >> > >> -- > >> > >> I'm writing this email to try to keep a trace of the changes that I > >> made to fix all these issues. > >> > >> Major changes: > >> > >> (1) IocpProactor.connect_pipe() was implemented using a thread which > >> could not be interrupted. There were hacks in IocpProactor to > >> workaround issues related to this. I rewrote the code using an > >> explicit polling with an increasing delay between 1 ms and 100 ms. > >> > >> (2) I fixed IocpProactor.accept_pipe(). The function now uses the > >> result of ConnectNamedPipe() to decide if we should register the > >> overlapped operation to wait for its completion, or it is already > >> done. I made a simiar change for IocpProactor.recv() (ReadFile() now > >> raises an exception on broken pipe error). > >> > >> (3) I fixed the cancellation of the IocpProactor.wait_for_handle() > future. > >> > >> -- > >> > >> I spent most of my time to try to fix the latest issue, the > >> cancellation of wait_for_handle(). This issue was annoying because it > >> emited unexpected completion. For example, a process was seen a > >> terminated, while it was still running. It also emited sometimes > >> "unexpected event" warnings. Sometimes, it simply crashed because > >> Windows tried to write in a memory block which was release. I told > >> you, a lot of fun. > >> > >> The internal machinery of the Windows RegisterWaitForSingleObject() > >> function is very complex. > >> > >> Basically, RegisterWaitForSingleObject() is implemented with a > >> blocking call which is called in a thread. The annoying point is that > >> UnregisterWait() doesn't cancel immediatly the wait: it only > >> "schedules" the cancellation. This point is not clear in the > >> documentation, it took me hours to understand that. Ok, now it becomes > >> funnier. > >> > >> UnregisterWaitEx() exists to be notified when the wait is cancelled: > >> an event will be set. Ok, but how can we wait for this notification > >> using an IOCP? Using RegisterWaitForSingleObject() again! > >> > >> What? To cancel a first RegisterWaitForSingleObject(), we have to call > >> RegisterWaitForSingleObject() again on a new event? How can we cancel > >> the second wait? ... To protect my head against an obvious explosion, > >> I decided to deny the cancellation of the second kind of wait :-) > >> > >> Someone may find a more efficient way to wait for the cancellation of > >> the first wait. I don't know enough all Windows internals. > >> > >> Maybe we should reimplement RegisterWaitForSingleObject() in Python to > >> have a better control on threads and objects? I don't know yet if it > >> would make sense to reimplement it. > >> > >> -- > >> > >> More details! RegisterWaitForSingleObject() is implemented as a pool > >> of threads (500 max. by default). Each thread calls the blocking > >> WaitForMultipleObjects() function, which can only wait for 64 objects. > >> To be able to interact with these threads, each thread uses a timer > >> (so each thread can only wait for 63 objects). It computes the next > >> timeout of all registered wait operations. To modify the list of wait > >> operations (RegisterWait..., UnregisterWait...), the timer is reset to > >> wake up WaitForMultipleObjects(), and so wake up the thread. > >> > >> Since we are talking of threads, and even a pool of threads, all > >> operations are asynchronous. RegisterWaitForSingleObject() may spawn a > >> new thread, and UnregisterWait[Ex]() may stop a thread (which has > >> nothing to do). > >> > >> FYI it's also possible to use UnregisterWaitEx() in blocking mode. > >> It's not interesting in the context of asyncio. > >> > >> -- > >> > >> Full list of recent IOCP issues in Tulip and Python bug trackers. > >> There are now all closed. > >> > >> "_WaitHandleFuture.cancel() crash if the wait event was already > >> unregistered" > >> https://code.google.com/p/tulip/issues/detail?id=195 > >> > >> "_OverlappedFuture.set_result() should clear the its reference to the > >> overlapped object" > >> https://code.google.com/p/tulip/issues/detail?id=196 > >> > >> "Rewrite IocpProactor.connect_pipe() with non-blocking calls to avoid > >> non interruptible QueueUserWorkItem()" > >> https://code.google.com/p/tulip/issues/detail?id=197 > >> > >> "Investigate IocpProactor.accept_pipe() special case (don't register > >> overlapped)" > >> https://code.google.com/p/tulip/issues/detail?id=204 > >> > >> "race condition when cancelling a _WaitHandleFuture" > >> http://bugs.python.org/issue23095 > >> > >> "race condition related to IocpProactor.connect_pipe()" > >> http://bugs.python.org/issue23293 > >> > >> Victor > > > > > > > > > > -- > > --Guido van Rossum (python.org/~guido) >
