[issue1256] subprocess Popen wait() function hangs when stdout is 20480

[Christian Heimes]

Christian Heimes added the comment:

The docs should be updated, see #1606

--
nosy: +tiran
resolution:  - wont fix
status: open - closed
superseder:  - Doc: subprocess wait() may lead to dead lock

__
Tracker [EMAIL PROTECTED]
http://bugs.python.org/issue1256
__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1256] subprocess Popen wait() function hangs when stdout is 20480

[Ivan Pozdeev]

Ivan Pozdeev added the comment:

I suggest adding this as a note to the subprocess module documentation instead. 
The 17.1.2 Popen Objects section seems to be the right place.

--
nosy: +Vano

__
Tracker [EMAIL PROTECTED]
http://bugs.python.org/issue1256
__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1256] subprocess Popen wait() function hangs when stdout is 20480

[Raghuram Devarakonda]

Raghuram Devarakonda added the comment:

PIPE on POSIX are implemented using pipe system call so it will have all
the limitations of the system buffering that is used for implementing
pipes. The buffer size is system dependent. For example, on my Linux, I
hit the hang condition at 64K and not 20480 bytes (as is your case).
What basically happens is that the child fills up the pipe buffer and is
blocked until space is available. Since the parent in your test script
is not reading the bytes, space will never be freed up. 

I suggest that the bug is closed as invalid.

--
nosy: +draghuram

__
Tracker [EMAIL PROTECTED]
http://bugs.python.org/issue1256
__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1256] subprocess Popen wait() function hangs when stdout is 20480

[superwesman]

New submission from superwesman:

hi - I noticed some odd behaviour with subprocess when the output
(stdout) is large - I did some sleuthing and found a reproduceable case
- I tested this on solaris using python 2.4.2 (sparc) and using 2.5
(opteron) - I've attached a python script that details the problem - I
created 2 files (small_file and large_file) that are 20480 and 20481
bytes, respectively 

 ls -lrt ./tmp/*_file
-rw-r--r--   1 wtorres  staff  20481 Oct 10 12:32 ./tmp/large_file
-rw-r--r--   1 wtorres  staff  20480 Oct 10 12:32 ./tmp/small_file

the script cats each file, thus generating an appropriately sized stdout
- the workaround we came up with was to use our own file handle and use
the parameter stdout=my_file_handle.fileno() instead of stdout=PIPE -
you can see this in the attached example as well

 python ./tmp/brent.py 
testing with a small file - these should both work
using my own file handle  ok
using PIPE ... ok
testing with a large file - the second one here should fail
using my own file handle  ok
using PIPE ...
Traceback (most recent call last):
  File ./tmp/brent.py, line 43, in ?
test_popen_bug( large_file )
  File ./tmp/brent.py, line 31, in test_popen_bug
rc = my_popen.wait()
  File /tools/python/2.4.2/solaris/lib/python2.4/subprocess.py, line
1007, in wait
pid, sts = os.waitpid(self.pid, 0)
KeyboardInterrupt

based on all this, it seems that the PIPE file handle has some buffer or
something that is filling up and not allowing us to continue... or
something.  Thoughts?
-w

--
components: None
files: brent.py
messages: 56317
nosy: superwesman
severity: normal
status: open
title: subprocess Popen wait() function hangs when stdout is  20480
type: behavior
versions: Python 2.4

__
Tracker [EMAIL PROTECTED]
http://bugs.python.org/issue1256
__

brent.py
Description: Binary data
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com