[issue16327] subprocess.Popen leaks file descriptors on os.fork() failure
Mark Gius added the comment: My contributor form has been accepted. Anything else I should be doing to work towards getting a fix applied? -- ___ Python tracker <http://bugs.python.org/issue16327> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16327] subprocess.Popen leaks file descriptors on os.fork() failure
Mark Gius added the comment: I've also submitted the contributor form requested. -- ___ Python tracker <http://bugs.python.org/issue16327> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16327] subprocess.Popen leaks file descriptors on os.fork() failure
Mark Gius added the comment: Here's more or less the same fix and test on 2.7. I jumped through the hoop to preserve the original exception and traceback even if os.close() raises an exception. This follows the 3.3 branch's cleanup behavior of silently suppressing errors in the cleanup code. -- Added file: http://bugs.python.org/file27781/fix_16327_on_2.7.diff ___ Python tracker <http://bugs.python.org/issue16327> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16327] subprocess.Popen leaks file descriptors on os.fork() failure
Mark Gius added the comment: Patch fixes and tests fd leak on Python 3.3. Test fails without fix, passes with fix. I found an existing test looking for fd leaks for another bug. Borrowed the verification bits from it. There were some other test failures when I ran the subprocess suite on my laptop, but it more like I had some environmental issue rather than having genuinely broken anything. If somebody else (or the test bots?) could run the tests I would appreciate it. -- Added file: http://bugs.python.org/file27780/fix_16327_on_3.3.diff ___ Python tracker <http://bugs.python.org/issue16327> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16327] subprocess.Popen leaks file descriptors on os.fork() failure
Mark Gius added the comment: Doesn't exhibit when execve fails, because by the time execve has been reached we've closed the pipes that we're supposed to close on the parent process, and the pipes that are meant to remain open on the parent process get caught by existing cleanup code. It's unfortunately got to fail somewhere in the vicinity of the fork to fake it using the actual _execute_child. -- ___ Python tracker <http://bugs.python.org/issue16327> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16327] subprocess.Popen leaks file descriptors on os.fork() failure
Mark Gius added the comment: attachment against 2.7 if I understand this code correctly, the fix shouldn't need to be fixed separately on windows and Linux, since the thing handled by init is just a file descriptor. Good idea on the testing. I'll give that a shot tomorrow. I think 3.3 will need some extra cleanup too. -- ___ Python tracker <http://bugs.python.org/issue16327> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16327] subprocess.Popen leaks file descriptors on os.fork() failure
Mark Gius added the comment: Patch now only closes pipe fds created by Popen -- Added file: http://bugs.python.org/file27720/fd_leak_fix_v2.diff ___ Python tracker <http://bugs.python.org/issue16327> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16327] subprocess.Popen leaks file descriptors on os.fork() failure
Mark Gius added the comment: Just read the docs for stdin and stdout. This patch will indtroduce a bug where a provided file descriptor can be closed. This definitely shouldn't close a file descriptor that isn't created by _get_handles(). I'll update. -- ___ Python tracker <http://bugs.python.org/issue16327> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16327] subprocess.Popen leaks file descriptors on os.fork() failure
New submission from Mark Gius:
When subprocess.Popen is called with subprocess.PIPE and os.fork() fails (due
to insufficient memory for example), the pipes created by _get_handles() are
not closed.
Steps to reproduce:
1) Launch a python shell, import subprocess
2) Create a situation where os.fork() will fail (I ran a program to eat up
nearly all of the memory on a system)
3) subprocess.Popen('/bin/echo', stdin=subprocess.PIPE)
4) OSError(12, 'Cannot allocate memory')
5) ls /proc/$pid/fd , There are now extra pipes.
I tested on Ubuntu 11.10 python (2.7.2-5ubuntu1). My reading of the 2.7 and
3.3 development trees suggest that this is an issue with both of those
branches, but I don't have a 3.3 installation to confirm with.
I've attached a snippet that fixes it for my version of Python on Ubuntu. No
idea what ramifications it will have for other versions/OS/etc. No automated
testing included because I'm not entirely sure how to replicate this without
eating up a ton of ram or doing something naughty with ulimit.
--
components: Library (Lib)
files: fd_leak_fix.diff
keywords: patch
messages: 173804
nosy: Mark.Gius
priority: normal
severity: normal
status: open
title: subprocess.Popen leaks file descriptors on os.fork() failure
type: resource usage
versions: Python 2.7
Added file: http://bugs.python.org/file27719/fd_leak_fix.diff
___
Python tracker
<http://bugs.python.org/issue16327>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
