On Tue, Oct 07, 2008 at 05:43:41PM -0700, Samuel A. Falvo II wrote: > p = subprocess.Popen( > command, > shell=True, > stdin=subprocess.PIPE, > stdout=subprocess.PIPE, > stderr=subprocess.STDOUT, > close_fds=True > ) > > outputChannel = p.stdout > output = outputChannel.read()
You call read with no arguments. This is highly problematic in the context of interprocess communication, unless you can be 100% positive that none of the children will write anywhere besides STDOUT, and won't try to read from STDIN in the meanwhile. Python's read() with no args reads until the end of file, which in IPC contexts is bad... Normally the child process won't close stdout until it exits. So, if it did any other output in between, say, to STDERR, the child will block waiting for the parent to read STDERR, meanwhile the parent is blocked waiting for input from the child's STDOUT, which results in a deadlock. Both processes sleep forever. The exact same thing can happen if either the shell script or a process started by the shell script tries to read from STDIN. Since Java is launched by the shell script, it inherits the shell script's STDIN, STDOUT, and STDERR file descriptors (i.e. the two processes share the same STDIO). Thus if the java process writes to STDERR, that also could be causing your deadlock. On Wed, Oct 08, 2008 at 11:24:39AM -0700, Samuel A. Falvo II wrote: > On Oct 7, 6:23 pm, "Gabriel Genellina" <[EMAIL PROTECTED]> wrote: > > Is your shell script doing something else, apart from invoking the java > > process? > > Obviously, yes. It's far from obvious. I can't count the number of scripts I've seen whose sole purpose was to launch a Java program (with the right environment)... You would do well to avoid being dismissive and listen to those who respond with help, when you are the one who obviously doesn't understand the behavior you're getting, and you're the one asking for help. > The script is some 150 lines long. But the hang-up > occurs because of the forked Java process, not the other lines. I'm betting it's because the Java program is writing warnings to STDERR (more data than will fit in the STDIO buffer), which you're not reading... > > If not, you could just invoke java directly from Python. Also, > > you set stdin=PIPE - is your java process expecting some input? you're not > > writing anything to stdin. > > It does not expect input from stdin. However, this does not affect > any OTHER scripts or commands I run. Irrelevant... Unless "any OTHER scripts" encompases all possible combinations of process interactions, and you can demontstrate that it does so, this proves nothing. > Let's remember to look at the objective facts: for shell scripts that > launch child processes of their own, Python hangs. For all other > types of commands, it works 100% as expected. These are not facts which are in evidence. We don't know what your script is doing, and it's clear that you yourself are not able to explain the behavior you are seeing, therefore there is no reason for us to conclude that the above statements are true and correct. Most likely, they are not. > > Anyway, it's better to use the communicate method instead (it uses select > > to read from both stdout and stderr): > > That doesn't help me. Please explain why it doesn't. The most likely cause of the behavior you are seeing is the deadlock I described, above. Using select() (i.e. using communicate()) should generally fix about half the cases... The rest would be fixed by redirecting STDIN of the child (or at least the Java process) from /dev/null, most likely. Then of course, there could be other explanations. But this being overwhelmingly the most likely one, if you don't try those solutions, there's no point in trying to help you further... -- Derek D. Martin http://www.pizzashack.org/ GPG Key ID: 0x81CFE75D
pgp8zdVYVQEhV.pgp
Description: PGP signature
-- http://mail.python.org/mailman/listinfo/python-list