On Sat, Jan 11, 2014 at 12:46 AM, Danny Yoo <d...@hashcollision.org> wrote: > There is a warning in the documentation on subprocess that might be > relevant to your situation: > > Warning: > Use communicate() rather than .stdin.write, .stdout.read or > .stderr.read to avoid deadlocks due to any of the other OS pipe > buffers filling up and blocking the child process. > > Reference: http://docs.python.org/2/library/subprocess.html > > It's possible that the process is deadlocking due to this situation. > Do you run into this issue if you use communicate()?
If more than one standard file is piped (not the case for the OP), `communicate` avoids deadlocks by using `poll` or `select` on POSIX. On Windows, it uses the current thread to write to stdin and uses separate threads to read from stdout and stderr. If only one standard file is piped, then there's no deadlock. It's just blocked while waiting for the buffer to fill. `communicate` does nothing special in this case (at least prior to 3.3): http://hg.python.org/cpython/file/3a1db0d2747e/Lib/subprocess.py#l767 In 3.3, the new `timeout` option for `communicate` also uses the select/thread implementation. stdout FILE stream buffering could be a problem if output is intermittent and the program doesn't `fflush` the buffer. On Linux the `stdbuf` program may be able to circumvent this. It injects code (i.e. libstdbuf.so is added to LD_PRELOAD) that calls `setvbuf` before the target's `main` runs. This allows you to set the stream to line buffering mode (unless the program itself calls `setvbuf`). I don't think a similar utility exists on Windows. http://linux.die.net/man/1/stdbuf _______________________________________________ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
