I don't like mapping the exceptions; just document them. I do like always using a socket pair. And being able to send EOF reliably sounds good!
On Thu, Jul 17, 2014 at 4:53 AM, Victor Stinner <[email protected]> wrote: > Hi, > > I have a few questions about subprocesses, socket pairs and pipes. > > > (1) use pipe() for stdin in _UnixSubprocessTransport? > > asyncio creates a socket pair for subprocesses instead of a "classic" > pipe on UNIX. On AIX it is not possible to listen for read event on > the write end of a pipe to be notified when the read end is closed (by > the child process). > > Maybe we should only use socketpair() on AIX, and pipe() on other > UNIXes? According to this benchmark, performances of pipes and socket > pairs are almost the same: > http://www.opendmx.net/index.php/Pipe_vs_Unix_Socket_Performance > > > (2) Convert or document ConnectionResetError? > > A side effect of using socketpair() is that the stdin of a subprocess > (asyncio.Process.stdin) can raises a ConnectionResetError instead of a > BrokenPipeError. > > I just modified Process.communicate() to ignore BrokenPipeError. After > a quick test, I saw that ConnectionResetError must also be ignored. > > Maybe we should convert the ConnectionResetError into a > BrokenPipeError in the pipe transport? Maybe only if the pipe is used > with a subprocess? Or, simpler option, document that stdin.drain() may > raise a BrokenPipeError or a ConnectionResetError exception, depending > on the implementation? > > > (3) Use socket.shutdown(SHUT_WR) when writing EOF into stdin? > > Interesting article on StackOverflow: > > http://stackoverflow.com/questions/1583005/is-there-any-difference-between-socketpair-and-pair-of-unnamed-pipes > > The article contains an interesting information. Using a socket, it's > possible to have a reliable "write EOF": use socket.shutdown(SHUT_WR). > It's more reliable than pipe.close() because the read end is only > notified of the EOF when the last file descriptors of the write end is > closed. > > Duplicated file descriptors happen with inheritable file descriptors > and child processes. > > In Python 3.4, all file descriptors (files, pipes, sockets, etc.) are > created non-inheritable (PEP 446), with an atomic flag (O_CLOEXEC, > SOCK_CLOEXEC) if available. In Python 3.3, subprocess creates pipes > with non-inheritable file description, it can use pipe2(O_CLOEXEC) if > the function is available. > http://legacy.python.org/dev/peps/pep-0446/ > > So does it make sense to use socket.shutdown(SHUT_WR) in write_eof() > for the subprocess stdin? > > > (4) Non-inheritable file descriptors for the socketpair > > By the way, on Python 3.3, _UnixSubprocessTransport._start() should > make the file descriptors of the socket pair non-inheritable. (Or we > may rely on subprocess which creates a pipe with non-inheritable file > descriptors ...) > > Victor > -- --Guido van Rossum (python.org/~guido)
