On Mon, 2005-03-21 at 17:32 +1200, Greg Ewing wrote: > > On 18 March 2005, Donovan Baarda said: > > >>Many Python library methods and classes like select.select(), os.popen2(), > >>and subprocess.Popen() return and/or operate on builtin file objects. > >>However even simple applications of these methods and classes require the > >>files to be in non-blocking mode. > > I don't agree with that. There's no need to use non-blocking > I/O when using select(), and in fact things are less confusing > if you don't.
You would think that... and the fact that select, popen2 etc all use file objects encourage you to think that. However, this is a trap that can catch you out badly. Check the attached python scripts that demonstrate the problem. Because staller.py outputs and flushes a fragment of data smaller than selector.py uses for its reads, the select statement is triggered, but the corresponding read blocks. A similar thing can happen with writes... if the child process consumes a fragment smaller than the write buffer of the selector process, then the select can trigger and the corresponding write can block because there is not enough space in the file buffer. The only ways to ensure that a select process does not block like this, without using non-blocking mode, are; 1) use a buffer size of 1 in the select process. 2) understand the child process's read/write behaviour and adjust the selector process accordingly... ie by making the buffer sizes just right for the child process, Note that it all interacts with the file objects buffer sizes too... making for some extremely hard to debug intermittent behaviour. > >>The read method's current behaviour needs to be documented, so its actual > >>behaviour can be used to differentiate between an empty non-blocking read, > >>and EOF. This means recording that IOError(EAGAIN) is raised for an empty > >>non-blocking read. > > Isn't that unix-specific? The file object is supposed to > provide a more or less platform-independent interface, I > thought. I think the fread/fwrite and read/write behaviour is posix standard and possibly C standard stuff... so it _should_ be the same on other platforms. -- Donovan Baarda <[EMAIL PROTECTED]> http://minkirri.apana.org.au/~abo/
selector.py
Description: application/python
staller.py
Description: application/python
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
