[issue21618] POpen does not close fds when fds have been inherited from a process with a higher resource limit

2014-06-01 Thread Gregory P. Smith
Gregory P. Smith added the comment: Here's a patch with a unittest that reproduces the problem with fixes to stop using any end_fds. The max fd is only ever used in the absolute fallback situation where no way to get a list of open fd's is available. In that case it is obtained from

[issue21618] POpen does not close fds when fds have been inherited from a process with a higher resource limit

2014-06-01 Thread Steven Stewart-Gallus
Steven Stewart-Gallus added the comment: Thank you for the very quick patch Gregory P. Smith. It's fair enough if you don't bother to fix the EINTR issue. One small note: +Confirm that issue21618 is fixed (mail fail under valgrind). That's a typo right? Shouldn't it be may instead

[issue21618] POpen does not close fds when fds have been inherited from a process with a higher resource limit

2014-06-01 Thread Roundup Robot
Roundup Robot added the comment: New changeset 5453b9c59cd7 by Gregory P. Smith in branch '3.4': Don't restrict ourselves to a max fd when closing fds before exec() http://hg.python.org/cpython/rev/5453b9c59cd7 New changeset 012329c8c4ec by Gregory P. Smith in branch 'default': Don't restrict

[issue21618] POpen does not close fds when fds have been inherited from a process with a higher resource limit

2014-06-01 Thread Gregory P. Smith
Gregory P. Smith added the comment: Backported to subprocess32 in https://code.google.com/p/python-subprocess32/source/detail?r=1c27bfe7e98f78e6aaa746b5c0a4d902a956e2a5 -- resolution: - fixed stage: patch review - commit review status: open - closed versions: +Python 3.5

[issue21618] POpen does not close fds when fds have been inherited from a process with a higher resource limit

2014-05-31 Thread Gregory P. Smith
Gregory P. Smith added the comment: Are you aware that the subprocess module does use /proc/self/fd in Python 3.2 and later? The fd closing is not done from Python code. See Modules/_posixsubprocess.c - http://hg.python.org/cpython/file/53fa2c9523d4/Modules/_posixsubprocess.c --

[issue21618] POpen does not close fds when fds have been inherited from a process with a higher resource limit

2014-05-31 Thread Gregory P. Smith
Changes by Gregory P. Smith g...@krypto.org: -- nosy: -gps ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21618 ___ ___ Python-bugs-list mailing

[issue21618] POpen does not close fds when fds have been inherited from a process with a higher resource limit

2014-05-31 Thread Gregory P. Smith
Gregory P. Smith added the comment: regardless, the current C code for this does limit itself to the sysconf(_SC_OPEN_MAX) max_fd from module import time when closing fds found in /proc/self/fd so this code does still have a bug in that fds higher than that will remain unclosed (at which

[issue21618] POpen does not close fds when fds have been inherited from a process with a higher resource limit

2014-05-31 Thread Gregory P. Smith
Gregory P. Smith added the comment: There appear to be a two bugs here, depending on which platform subprocess is being used on. 1) on systems where it uses /prod/self/fd, /dev/fd or similar: It should not pay attention to end_fd at all. It knows the list of actual open fds and should use

[issue21618] POpen does not close fds when fds have been inherited from a process with a higher resource limit

2014-05-31 Thread Steven Stewart-Gallus
Steven Stewart-Gallus added the comment: I agree that this is not a likely scenario but I can think of one mildly plausible scenario. Suppose some web server runs a Python CGI script but has a bug that leaks a file descriptor into the script. The web server sandboxes the Python CGI script a

[issue21618] POpen does not close fds when fds have been inherited from a process with a higher resource limit

2014-05-31 Thread akira
akira added the comment: getrlimit and setrlimit are not exposed in the os module; you must use ctypes or an extension module to call them from Python: There is `resource` module: import resource resource.getrlimit(resource.RLIMIT_NOFILE) (1024, 4096) -- nosy: +akira

[issue21618] POpen does not close fds when fds have been inherited from a process with a higher resource limit

2014-05-31 Thread Steven Stewart-Gallus
Steven Stewart-Gallus added the comment: I found another problem with _close_open_fd_range_safe. POSIX leaves the state of a file descriptor given to close undefined if the close fails with EINTR. I believe but haven't double checked that close should not be retried on EINTR on all of our

[issue21618] POpen does not close fds when fds have been inherited from a process with a higher resource limit

2014-05-30 Thread Steven Stewart-Gallus
Changes by Steven Stewart-Gallus sstewartgallu...@mylangara.bc.ca: -- type: - security versions: +Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21618 ___

[issue21618] POpen does not close fds when fds have been inherited from a process with a higher resource limit

2014-05-30 Thread Steven Stewart-Gallus
New submission from Steven Stewart-Gallus: The sysconf(_SC_OPEN_MAX) approach to closing fds is kind of flawed. It is kind of hacky and slow (see http://bugs.python.org/issue1663329). Moreover, this approach is incorrect as fds can be inherited from previous processes that have had higher

[issue21618] POpen does not close fds when fds have been inherited from a process with a higher resource limit

2014-05-30 Thread Steven Stewart-Gallus
Steven Stewart-Gallus added the comment: Okay here's a stub patch that address FreeBSD, NetBSD and Linux. I'm not sure how to address the other platforms. -- keywords: +patch Added file: http://bugs.python.org/file35419/python.patch ___ Python

[issue21618] POpen does not close fds when fds have been inherited from a process with a higher resource limit

2014-05-30 Thread Steven Stewart-Gallus
Steven Stewart-Gallus added the comment: Oh right! I forgot a possible problem with my proposed patch. It is incompatible with Valgrind (see issue https://bugs.kde.org/show_bug.cgi?id=331311). Either this patch won't be applied, Valgrind compatibility is judged not essential or the Valgrind

[issue21618] POpen does not close fds when fds have been inherited from a process with a higher resource limit

2014-05-30 Thread Ned Deily
Changes by Ned Deily n...@acm.org: -- nosy: +gps, larry ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21618 ___ ___ Python-bugs-list mailing list