[issue1663329] subprocess/popen close_fds perform poor if SC_OPEN_MAX is hi

[STINNER Victor]

Change by STINNER Victor :


--
resolution: accepted -> fixed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1663329] subprocess/popen close_fds perform poor if SC_OPEN_MAX is hi

[sorin]

sorin  added the comment:

Am I wrong or we should reopen this bug. Try `ulimit -n unlimited` and you'll 
make python code that does popen practically blocking.

--
nosy: +sorin
type:  -> performance

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1663329] subprocess/popen close_fds perform poor if SC_OPEN_MAX is hi

[Georg Brandl]

Georg Brandl added the comment:

Reviewed and committed patch as r60097. Thanks for your work!

--
assignee: gvanrossum -> georg.brandl
nosy: +georg.brandl
resolution:  -> accepted
status: open -> closed

_
Tracker <[EMAIL PROTECTED]>

_
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1663329] subprocess/popen close_fds perform poor if SC_OPEN_MAX is hi

[Guido van Rossum]

Guido van Rossum added the comment:

I'll review this when I have time. Thanks! If you want speedier
response, please find someone else among the core developers; or perhaps
the bug day on Jan 19 will help.

--
assignee:  -> gvanrossum

_
Tracker <[EMAIL PROTECTED]>

_
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1663329] subprocess/popen close_fds perform poor if SC_OPEN_MAX is hi

[Mike Klaas]

Mike Klaas added the comment:

Updated patch, now with documentation and test.  Patch is against 
2.6trunk, but probably will apply against 3.0 as well

--
components: +Library (Lib) -Interpreter Core
versions: +Python 2.6 -Python 2.5
Added file: http://bugs.python.org/file9070/posix_closerange.patch

_
Tracker <[EMAIL PROTECTED]>

_
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1663329] subprocess/popen close_fds perform poor if SC_OPEN_MAX is hi

[Guido van Rossum]

Guido van Rossum added the comment:

Looks good to me.

--
nosy: +gvanrossum

_
Tracker <[EMAIL PROTECTED]>

_
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1663329] subprocess/popen close_fds perform poor if SC_OPEN_MAX is hi

[Martin v. Löwis]

Changes by Martin v. Löwis:


--
keywords: +patch

_
Tracker <[EMAIL PROTECTED]>

_
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1663329] subprocess/popen close_fds perform poor if SC_OPEN_MAX is hi

[Mike Klaas]

Mike Klaas added the comment:

Finally, I did not include any platform-specific optimizations, as I don't 
have AIX or solaris boxes on which to test them (and they can easily be 
added later).  An order-mag speedup on all *nix platforms is a good start.

_
Tracker <[EMAIL PROTECTED]>

_
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1663329] subprocess/popen close_fds perform poor if SC_OPEN_MAX is hi

[Mike Klaas]

Mike Klaas added the comment:

I see that some spaces crept in to the patch.  I can fix that (and perhaps 
rename the function to start with an underscore) if desired.

_
Tracker <[EMAIL PROTECTED]>

_
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1663329] subprocess/popen close_fds perform poor if SC_OPEN_MAX is hi

[Mike Klaas]

Mike Klaas added the comment:

This problem has also afflicted us.

Attached is a patch which adds closerange(fd_low, fd_high) to the posix 
(and consequently os) module, and modifies subprocess to use it.  Patch 
is against trunk but should work for 2.5maint.

I don't really think that this is useful enough to add to the public 
api, but it results in a huge performance benefit for subprocess:

[python-trunk]$ ./python -m timeit -s 'import python_close' 
'python_close.go(10)'
10 loops, best of 3: 358 msec per loop
[python-trunk]$ ./python -m timeit -s 'import os' 'os.closerange(4, 
10)'
10 loops, best of 3: 20.7 msec per loop
[python-trunk]$ ./python -m timeit -s 'import python_close' 
'python_close.go(30)'
10 loops, best of 3: 1.05 sec per loop
[python-trunk]$ ./python -m timeit -s 'import os' 'os.closerange(4, 
30)'10 loops, best of 3: 63 msec per loop

[python-trunk]$ cat python_close.py
import os, sys
def go(N):
for i in xrange(4, N):
try:
os.close(i)
except:
pass

--
nosy: +klaas

_
Tracker <[EMAIL PROTECTED]>

_

posix_closerange.patch
Description: Binary data
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com