New submission from sbt <shibt...@gmail.com>: There are some problems with the new Windows overlapped implementation of PipeConnection in the default branch.
1) poll(0) can return False when an empty string is in the pipe: if the next message in the pipe is b"" then PeekNamedPipe() returns (0, 0) which is indistiguishable from the case where the pipe is empty. This affects versions 2.6-3.2 of Python on Windows as well. For old versions I would just document the fact that poll() will fail to see that there is data in the pipe if the next message is an empty string. In practice this is not a big deal since pipes are primarily used for pickled objects which will never be the empty string. 2) _poll() forgets to check self._buffered, so it can return False even if there is buffered data. 3) Even if (2) is fixed, _poll() with a non-zero timeout can mess up message boundaries if the pipe contains messages of length zero or one. To fix this overlapped_GetOverlappedResult() needs to be changed to not swallow and forget ERROR_MORE_DATA errors. 4) In _poll() there is a race condition: if the read operation completes after WaitForMultipleObjects() timesout, but before the operation is cancelled, then the data read will be discarded. 5) The code assumes that if CancelIo()/CancelIoEx() returns successfully then the OVERLAPPED structure and associated buffer may be deallocated immediately. But the documentation says that if CancelIo()/CancelIoEx() succeeds then cancellation has only been *requested*: http://msdn.microsoft.com/en-us/library/aa363792%28v=vs.85%29.aspx If [CancelIoEx()] succeeds, the return value is nonzero. The cancel operation for all pending I/O operations issued by the calling thread for the specified file handle was successfully *requested*. The application must not free or reuse the OVERLAPPED structure associated with the canceled I/O operations until they have *completed*. The thread can use the GetOverlappedResult function to determine when the I/O operations themselves have been completed. We should wait for cancellation to *complete* by using GetOverlappedResult(..., TRUE), otherwise we risk a crash. If the operation was cancelled then FALSE is returned with GetLastError() == ERROR_OPERATION_ABORTED. I would suggest making it a programming error for the overlapped object to be deallocated while the operation is still pending, and to print a RuntimeError if that happens. (This does not prevent us from still trying to prevent a crash using CancelIoEx().) 6) Not a bug exactly, but poll(timeout) is no longer interruptible with Ctrl-C. This also means that Queue.get() is no longer interruptible. -- The unit tests attached pass on Unix. On Windows versions up to 3.2, only test_empty_string fails. On Windows version 3.3 all three fail. ---------- components: Extension Modules files: test_pipe_poll.py messages: 138268 nosy: jnoller, pitrou, sbt priority: normal severity: normal status: open title: multiprocessing's overlapped PipeConnection on Windows type: behavior versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3 Added file: http://bugs.python.org/file22345/test_pipe_poll.py _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue12328> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com