Sorry, I've been ignoring this thread. I still don't think I have enough time but I'll reply to Victor's message below.
On Sat, Jan 25, 2014 at 2:42 PM, Victor Stinner <[email protected]>wrote: > (Guido hijicked my "Round timeout in BaseEventLoop._run_once()" thread > with the subprocess discussion, I prefer to reply here) > Sorry. > >> Hum, yes, something should be done for subprocess. IMO the API of > >> subprocess should be changed. The best would be to provide something > >> like StreamReader and StreamWriter for subprocess. If it's not > >> possible to develop it before Python 3.4 final, a compromise is to > >> ensure that the API allows to develop it later. According to my > >> analysis (see the other dedicated thread on Tulip mailing list), it's > >> not the case with the current API. > > > > It's clear that not enough people have actually tried to *use* the > > subprocess API. :-( > > I take a look at examples/child_process.py, but I don't understand > this example. It doesn't use EventLoop.subprocess_exec(). It doesn't > use SubprocessTransport, whereas I would like to use it to benefit of > the great FastChildWatcher. > That example is much older than the subprocess API. Please disregard it. Maybe we should even nix it and write a new one. > Having to write a connect_read_pipe() function to get a StreamReader > is too complex for me. I don't want to duplicate this function in each > script that want to run a command and use asyncio. > > The current API looks very low-level. I don't want to use it in my > application. > That's fine. It defines an abstraction on which we can build. > > My suggestion: first try to write code using the existing public API to > hook > > up the existing StreamReader / StreamWriter classes to stdin, stdout, > stderr > > of a subprocess. > > I did and I failed, especially for StreamWriter. > OK, that is good to know and points to a failing in the way StreamWriter was written. > > StreamReader and StreamWriter are explicitly intended to be hooked up in > > other ways than the code in open_connection(), start_server() and > > StreamReaderProtocol), and those three are intended to be simple enough > to > > be able to be copied into user code and then modified for the user's > > purpose. If changes to StreamReader/Writer are necessary we should > > prioritize those -- their implementation is *not* meant to be copied. > > StreamReaderProtocol and StreamWriter cannot be used for write pipes > (stdin) because these classes expect that a connected reader, which is > not the case. I don't care of having to use a completly different > class if it gives the same nice API (write, writeline and drain). > But you shouldn't have to write something completely different. Small changes should be enough. > I will play with the API to see how I can implement something like a > StreamWriter for write pipes. > Agreed. For bidirectional protocols, StreamReader is the more important one of the pair -- StreamWriter is *almost* a transparent wrapper around a (regular) transport, with the exception of the drain() method. It looks like we never got to implementing pause/resume for pipes. (Flow control was implemented very late in the process.) So your first approximation can probably subclass StreamWriter with a drain() that always returns an empty tuple, and then you can set the reader and the protocol to None. (Both are only used in drain(). The reader is used to raise the reader's exception; the protocol is expected to be a StreamReaderProtocol, as it uses private attributes of the latter. Adding checks for None should be simple enough here.) Eventually we should add flow control to _UnixPipeWriter. It would be very similar to the socket version. Then we would need a protocol similar to StreamReaderProtocol (but without a reader) to hold the pause state management. It's probably possible to refactor StreamReaderProtocol to move the pause state management to a base class. -- --Guido van Rossum (python.org/~guido)
