Bug#772543: python-psutil: severak filehandle leaks
Hello Marc, upstream has worked on this regard[1] and have asked if you can run your tests on the git version of psutil - is that something you can do? [1] https://github.com/giampaolo/psutil/issues/556#issuecomment-68595246 Thanks in advance, Sandro On Tue, Dec 16, 2014 at 9:27 AM, Marc Leeman marc.lee...@gmail.com wrote: patch. Testing it now. On 16 December 2014 at 09:22, Marc Leeman marc.lee...@gmail.com wrote: Just to let you know that the current version upstream is not enough to solve the problem. I copied their version in the jessie release, backported it to wheezy and tried it. After less than 24h, the system is stuck again, The problem can be traced to: @wrap_exceptions def threads(self): Where there is still an open that does not close properly: 2014-12-16 09:21:21,185 - com.barco.mgs.TeamMgrM56094 - INFO - start - Received request to start a team sid T10621 - wid W90616 2014-12-16 09:21:21,185 - com.barco.mgs.TeamMgrM56094 - ERROR - start - [' File /usr/lib/pymodules/python2.7/Mgs2/Team/TeamMgr.py, line 831, in start\nlimit = status.cpu_percent\n', ' File /usr/lib/pymodules/python2.7/Mgs2/SystemStatusBase.py, line 346, in cpu_percent\n', ' File /usr/lib/pymodules/python2.7/Mgs2/SystemStatusBase.py, line 349, in get_cpu_percent\n', ' File /usr/lib/python2.7/dist-packages/psutil/__init__.py, line 1509, in cpu_percent\n_last_cpu_times = cpu_times()\n', ' File /usr/lib/python2.7/dist-packages/psutil/__init__.py, line 1439, in cpu_times\nreturn _psplatform.cpu_times()\n', ' File /usr/lib/python2.7/dist-packages/psutil/_pslinux.py, line 215, in cpu_times\nwith open(\'/proc/stat\', \'rb\') as f:\n'] 2014-12-16 09:21:21,186 - com.barco.mgs.TeamMgrM56094 - ERROR - start - [Errno 24] Too many open files: '/proc/stat' 2014-12-16 09:21:21,186 - com.barco.mgs.TeamMgrM56094 - ERROR - start - Error during worker creation [Errno 24] Too many open files: '/proc/stat' ...snip... rtsp_mgs. 26609 29136root txt unknown /proc/26609/task/29136/exe (readlink: Permission denied) rtsp_mgs. 26609 29136root NOFD /proc/26609/task/29136/fd (opendir: Permission denied) rtsp_mgs. 26609 29144root cwd unknown /proc/26609/task/29144/cwd (readlink: Permission denied) rtsp_mgs. 26609 29144root rtd unknown /proc/26609/task/29144/root (readlink: Permission denied) rtsp_mgs. 26609 29144root txt unknown /proc/26609/task/29144/exe (readlink: Permission denied) rtsp_mgs. 26609 29144root NOFD /proc/26609/task/29144/fd (opendir: Permission denied) rtsp_mgs. 26609 29147root cwd unknown /proc/26609/task/29147/cwd (readlink: Permission denied) rtsp_mgs. 26609 29147root rtd unknown /proc/26609/task/29147/root (readlink: Permission denied) rtsp_mgs. 26609 29147root txt unknown /proc/26609/task/29147/exe (readlink: Permission denied) rtsp_mgs. 26609 29147root NOFD /proc/26609/task/29147/fd (opendir: Permission denied) rtsp_mgs. 26609 29148root cwd unknown /proc/26609/task/29148/cwd (readlink: Permission denied) rtsp_mgs. 26609 29148root rtd unknown /proc/26609/task/29148/root (readlink: Permission denied) rtsp_mgs. 26609 29148root txt unknown /proc/26609/task/29148/exe (readlink: Permission denied) rtsp_mgs. 26609 29148root NOFD /proc/26609/task/29148/fd (opendir: Permission denied) rtsp_mgs. 26609 29149root cwd unknown /proc/26609/task/29149/cwd (readlink: Permission denied) rtsp_mgs. 26609 29149root rtd unknown /proc/26609/task/29149/root (readlink: Permission denied) rtsp_mgs. 26609 29149root txt unknown /proc/26609/task/29149/exe (readlink: Permission denied) rtsp_mgs. 26609 29149root NOFD /proc/26609/task/29149/fd (opendir: Permission denied) rtsp_mgs. 26609 29152root cwd unknown /proc/26609/task/29152/cwd (readlink: Permission denied) rtsp_mgs. 26609 29152root rtd unknown /proc/26609/task/29152/root (readlink: Permission denied) rtsp_mgs. 26609 29152root txt unknown /proc/26609/task/29152/exe (readlink: Permission denied) rtsp_mgs. 26609 29152root NOFD /proc/26609/task/29152/fd (opendir: Permission denied) rtsp_mgs. 26609 29153root cwd unknown /proc/26609/task/29153/cwd (readlink: Permission denied) rtsp_mgs. 26609 29153root rtd unknown /proc/26609/task/29153/root (readlink: Permission denied) rtsp_mgs. 26609 29153root txt unknown /proc/26609/task/29153/exe (readlink: Permission denied) rtsp_mgs. 26609 29153root NOFD /proc/26609/task/29153/fd (opendir: Permission denied) rtsp_mgs. 26609 29155root cwd unknown /proc/26609/task/29155/cwd (readlink: Permission denied) rtsp_mgs. 26609 29155root rtd unknown /proc/26609/task/29155/root (readlink:
Bug#772543: python-psutil: severak filehandle leaks
Just to let you know that the current version upstream is not enough to solve the problem. I copied their version in the jessie release, backported it to wheezy and tried it. After less than 24h, the system is stuck again, The problem can be traced to: @wrap_exceptions def threads(self): Where there is still an open that does not close properly: 2014-12-16 09:21:21,185 - com.barco.mgs.TeamMgrM56094 - INFO - start - Received request to start a team sid T10621 - wid W90616 2014-12-16 09:21:21,185 - com.barco.mgs.TeamMgrM56094 - ERROR - start - [' File /usr/lib/pymodules/python2.7/Mgs2/Team/TeamMgr.py, line 831, in start\nlimit = status.cpu_percent\n', ' File /usr/lib/pymodules/python2.7/Mgs2/SystemStatusBase.py, line 346, in cpu_percent\n', ' File /usr/lib/pymodules/python2.7/Mgs2/SystemStatusBase.py, line 349, in get_cpu_percent\n', ' File /usr/lib/python2.7/dist-packages/psutil/__init__.py, line 1509, in cpu_percent\n_last_cpu_times = cpu_times()\n', ' File /usr/lib/python2.7/dist-packages/psutil/__init__.py, line 1439, in cpu_times\nreturn _psplatform.cpu_times()\n', ' File /usr/lib/python2.7/dist-packages/psutil/_pslinux.py, line 215, in cpu_times\nwith open(\'/proc/stat\', \'rb\') as f:\n'] 2014-12-16 09:21:21,186 - com.barco.mgs.TeamMgrM56094 - ERROR - start - [Errno 24] Too many open files: '/proc/stat' 2014-12-16 09:21:21,186 - com.barco.mgs.TeamMgrM56094 - ERROR - start - Error during worker creation [Errno 24] Too many open files: '/proc/stat' ...snip... rtsp_mgs. 26609 29136root txt unknown /proc/26609/task/29136/exe (readlink: Permission denied) rtsp_mgs. 26609 29136root NOFD /proc/26609/task/29136/fd (opendir: Permission denied) rtsp_mgs. 26609 29144root cwd unknown /proc/26609/task/29144/cwd (readlink: Permission denied) rtsp_mgs. 26609 29144root rtd unknown /proc/26609/task/29144/root (readlink: Permission denied) rtsp_mgs. 26609 29144root txt unknown /proc/26609/task/29144/exe (readlink: Permission denied) rtsp_mgs. 26609 29144root NOFD /proc/26609/task/29144/fd (opendir: Permission denied) rtsp_mgs. 26609 29147root cwd unknown /proc/26609/task/29147/cwd (readlink: Permission denied) rtsp_mgs. 26609 29147root rtd unknown /proc/26609/task/29147/root (readlink: Permission denied) rtsp_mgs. 26609 29147root txt unknown /proc/26609/task/29147/exe (readlink: Permission denied) rtsp_mgs. 26609 29147root NOFD /proc/26609/task/29147/fd (opendir: Permission denied) rtsp_mgs. 26609 29148root cwd unknown /proc/26609/task/29148/cwd (readlink: Permission denied) rtsp_mgs. 26609 29148root rtd unknown /proc/26609/task/29148/root (readlink: Permission denied) rtsp_mgs. 26609 29148root txt unknown /proc/26609/task/29148/exe (readlink: Permission denied) rtsp_mgs. 26609 29148root NOFD /proc/26609/task/29148/fd (opendir: Permission denied) rtsp_mgs. 26609 29149root cwd unknown /proc/26609/task/29149/cwd (readlink: Permission denied) rtsp_mgs. 26609 29149root rtd unknown /proc/26609/task/29149/root (readlink: Permission denied) rtsp_mgs. 26609 29149root txt unknown /proc/26609/task/29149/exe (readlink: Permission denied) rtsp_mgs. 26609 29149root NOFD /proc/26609/task/29149/fd (opendir: Permission denied) rtsp_mgs. 26609 29152root cwd unknown /proc/26609/task/29152/cwd (readlink: Permission denied) rtsp_mgs. 26609 29152root rtd unknown /proc/26609/task/29152/root (readlink: Permission denied) rtsp_mgs. 26609 29152root txt unknown /proc/26609/task/29152/exe (readlink: Permission denied) rtsp_mgs. 26609 29152root NOFD /proc/26609/task/29152/fd (opendir: Permission denied) rtsp_mgs. 26609 29153root cwd unknown /proc/26609/task/29153/cwd (readlink: Permission denied) rtsp_mgs. 26609 29153root rtd unknown /proc/26609/task/29153/root (readlink: Permission denied) rtsp_mgs. 26609 29153root txt unknown /proc/26609/task/29153/exe (readlink: Permission denied) rtsp_mgs. 26609 29153root NOFD /proc/26609/task/29153/fd (opendir: Permission denied) rtsp_mgs. 26609 29155root cwd unknown /proc/26609/task/29155/cwd (readlink: Permission denied) rtsp_mgs. 26609 29155root rtd unknown /proc/26609/task/29155/root (readlink: Permission denied) rtsp_mgs. 26609 29155root txt unknown /proc/26609/task/29155/exe (readlink: Permission denied) rtsp_mgs. 26609 29155root NOFD /proc/26609/task/29155/fd (opendir: Permission denied) rtsp_mgs. 26609 29157root cwd unknown /proc/26609/task/29157/cwd (readlink: Permission denied) rtsp_mgs. 26609 29157root rtd unknown /proc/26609/task/29157/root (readlink: Permission denied) rtsp_mgs. 26609 29157root txt unknown
Bug#772543: python-psutil: severak filehandle leaks
patch. Testing it now. On 16 December 2014 at 09:22, Marc Leeman marc.lee...@gmail.com wrote: Just to let you know that the current version upstream is not enough to solve the problem. I copied their version in the jessie release, backported it to wheezy and tried it. After less than 24h, the system is stuck again, The problem can be traced to: @wrap_exceptions def threads(self): Where there is still an open that does not close properly: 2014-12-16 09:21:21,185 - com.barco.mgs.TeamMgrM56094 - INFO - start - Received request to start a team sid T10621 - wid W90616 2014-12-16 09:21:21,185 - com.barco.mgs.TeamMgrM56094 - ERROR - start - [' File /usr/lib/pymodules/python2.7/Mgs2/Team/TeamMgr.py, line 831, in start\nlimit = status.cpu_percent\n', ' File /usr/lib/pymodules/python2.7/Mgs2/SystemStatusBase.py, line 346, in cpu_percent\n', ' File /usr/lib/pymodules/python2.7/Mgs2/SystemStatusBase.py, line 349, in get_cpu_percent\n', ' File /usr/lib/python2.7/dist-packages/psutil/__init__.py, line 1509, in cpu_percent\n_last_cpu_times = cpu_times()\n', ' File /usr/lib/python2.7/dist-packages/psutil/__init__.py, line 1439, in cpu_times\nreturn _psplatform.cpu_times()\n', ' File /usr/lib/python2.7/dist-packages/psutil/_pslinux.py, line 215, in cpu_times\nwith open(\'/proc/stat\', \'rb\') as f:\n'] 2014-12-16 09:21:21,186 - com.barco.mgs.TeamMgrM56094 - ERROR - start - [Errno 24] Too many open files: '/proc/stat' 2014-12-16 09:21:21,186 - com.barco.mgs.TeamMgrM56094 - ERROR - start - Error during worker creation [Errno 24] Too many open files: '/proc/stat' ...snip... rtsp_mgs. 26609 29136root txt unknown /proc/26609/task/29136/exe (readlink: Permission denied) rtsp_mgs. 26609 29136root NOFD /proc/26609/task/29136/fd (opendir: Permission denied) rtsp_mgs. 26609 29144root cwd unknown /proc/26609/task/29144/cwd (readlink: Permission denied) rtsp_mgs. 26609 29144root rtd unknown /proc/26609/task/29144/root (readlink: Permission denied) rtsp_mgs. 26609 29144root txt unknown /proc/26609/task/29144/exe (readlink: Permission denied) rtsp_mgs. 26609 29144root NOFD /proc/26609/task/29144/fd (opendir: Permission denied) rtsp_mgs. 26609 29147root cwd unknown /proc/26609/task/29147/cwd (readlink: Permission denied) rtsp_mgs. 26609 29147root rtd unknown /proc/26609/task/29147/root (readlink: Permission denied) rtsp_mgs. 26609 29147root txt unknown /proc/26609/task/29147/exe (readlink: Permission denied) rtsp_mgs. 26609 29147root NOFD /proc/26609/task/29147/fd (opendir: Permission denied) rtsp_mgs. 26609 29148root cwd unknown /proc/26609/task/29148/cwd (readlink: Permission denied) rtsp_mgs. 26609 29148root rtd unknown /proc/26609/task/29148/root (readlink: Permission denied) rtsp_mgs. 26609 29148root txt unknown /proc/26609/task/29148/exe (readlink: Permission denied) rtsp_mgs. 26609 29148root NOFD /proc/26609/task/29148/fd (opendir: Permission denied) rtsp_mgs. 26609 29149root cwd unknown /proc/26609/task/29149/cwd (readlink: Permission denied) rtsp_mgs. 26609 29149root rtd unknown /proc/26609/task/29149/root (readlink: Permission denied) rtsp_mgs. 26609 29149root txt unknown /proc/26609/task/29149/exe (readlink: Permission denied) rtsp_mgs. 26609 29149root NOFD /proc/26609/task/29149/fd (opendir: Permission denied) rtsp_mgs. 26609 29152root cwd unknown /proc/26609/task/29152/cwd (readlink: Permission denied) rtsp_mgs. 26609 29152root rtd unknown /proc/26609/task/29152/root (readlink: Permission denied) rtsp_mgs. 26609 29152root txt unknown /proc/26609/task/29152/exe (readlink: Permission denied) rtsp_mgs. 26609 29152root NOFD /proc/26609/task/29152/fd (opendir: Permission denied) rtsp_mgs. 26609 29153root cwd unknown /proc/26609/task/29153/cwd (readlink: Permission denied) rtsp_mgs. 26609 29153root rtd unknown /proc/26609/task/29153/root (readlink: Permission denied) rtsp_mgs. 26609 29153root txt unknown /proc/26609/task/29153/exe (readlink: Permission denied) rtsp_mgs. 26609 29153root NOFD /proc/26609/task/29153/fd (opendir: Permission denied) rtsp_mgs. 26609 29155root cwd unknown /proc/26609/task/29155/cwd (readlink: Permission denied) rtsp_mgs. 26609 29155root rtd unknown /proc/26609/task/29155/root (readlink: Permission denied) rtsp_mgs. 26609 29155root txt unknown /proc/26609/task/29155/exe (readlink: Permission denied) rtsp_mgs. 26609 29155root NOFD /proc/26609/task/29155/fd (opendir: Permission denied) rtsp_mgs. 26609 29157root cwd unknown /proc/26609/task/29157/cwd (readlink: Permission denied)
Bug#772543: python-psutil: severak filehandle leaks
Package: python-psutil
Version: 2.1.1-1+b1
Severity: important
Tags: patch
When using psutil to query system statistics, several function leak a file
handle by returning without closing the file handle.
When this is used on a regular basis, the system will fail with too many open
file handles:
eg.
def per_cpu_times():
Return a list of namedtuple representing the CPU times
for every CPU available on the system.
cpus = []
f = open('/proc/stat', 'rb')
try:
# get rid of the first line which refers to system wide CPU stats
f.readline()
CPU = b('cpu')
for line in f:
if line.startswith(CPU):
values = line.split()
fields = values[1:len(scputimes._fields) + 1]
fields = [float(x) / CLOCK_TICKS for x in fields]
entry = scputimes(*fields)
cpus.append(entry)
return cpus
finally:
f.close()
-- System Information:
Debian Release: 8.0
APT prefers unstable
APT policy: (500, 'unstable'), (500, 'stable'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386
Kernel: Linux 3.17-1-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
Versions of packages python-psutil depends on:
ii libc6 2.19-13
ii python 2.7.8-2
pn python:any none
python-psutil recommends no packages.
python-psutil suggests no packages.
-- no debconf information
This message is subject to the following terms and conditions: MAIL
DISCLAIMERhttp://www.barco.com/en/maildisclaimer
Description: pslinux leaks several filehandles by returning prematurely
This patch closes the filehandles before returning. An alternative way
of handling this more cleanly, but a bit larger rewrite, would be to
use use a with statement. This would be more clean.
.
python-psutil (2.1.1-1+barco2) unstable; urgency=low
.
* patch for misc filehandle leaks in pslinux
Author: Marc Leeman marc.lee...@barco.com
--- python-psutil-2.1.1.orig/psutil/_pslinux.py
+++ python-psutil-2.1.1/psutil/_pslinux.py
@@ -248,6 +248,7 @@ def per_cpu_times():
fields = [float(x) / CLOCK_TICKS for x in fields]
entry = scputimes(*fields)
cpus.append(entry)
+ f.close()
return cpus
finally:
f.close()
@@ -339,6 +340,7 @@ def boot_time():
if line.startswith(BTIME):
ret = float(line.strip().split()[1])
BOOT_TIME = ret
+ f.close()
return ret
raise RuntimeError(line 'btime' not found)
finally:
@@ -768,7 +770,9 @@ class Process(object):
f = open(fname, rt)
try:
# return the args as a list
-return [x for x in f.read().split('\x00') if x]
+ r = [x for x in f.read().split('\x00') if x]
+ f.close()
+return r
finally:
f.close()
@@ -862,6 +866,7 @@ class Process(object):
f = open(/proc/%s/statm % self.pid, 'rb')
try:
vms, rss = f.readline().split()[:2]
+ f.close()
return _common.pmem(int(rss) * PAGESIZE,
int(vms) * PAGESIZE)
finally:
@@ -990,6 +995,7 @@ class Process(object):
elif line.startswith(NON_VOLUNTARY):
unvol = int(line.split()[1])
if vol is not None and unvol is not None:
+ f.close()
return _common.pctxsw(vol, unvol)
raise NotImplementedError(
'voluntary_ctxt_switches' and 'nonvoluntary_ctxt_switches'
@@ -1005,6 +1011,7 @@ class Process(object):
THREADS = b(Threads:)
for line in f:
if line.startswith(THREADS):
+ f.close()
return int(line.split()[1])
raise NotImplementedError(line not found)
finally:
@@ -1140,7 +1147,9 @@ class Process(object):
letter = letter.decode()
# XXX is '?' legit? (we're not supposed to return
# it anyway)
-return PROC_STATUSES.get(letter, '?')
+ r = PROC_STATUSES.get(letter, '?')
+ f.close()
+ return r
finally:
f.close()
@@ -1193,6 +1202,7 @@ class Process(object):
for line in f:
if line.startswith(PPID):
# PPid:
+ f.close()
return int(line.split()[1])
raise NotImplementedError(line not found)
finally:
@@ -1206,7 +1216,9 @@ class Process(object):
for line in f:
if line.startswith(UID):
_, real, effective, saved, fs = line.split()
-return _common.puids(int(real), int(effective), int(saved))
+ r = _common.puids(int(real),
Bug#772543: python-psutil: severak filehandle leaks
Hi, thanks for the report! On Mon, Dec 8, 2014 at 1:28 PM, Marc Leeman marc.lee...@gmail.com wrote: When using psutil to query system statistics, several function leak a file handle by returning without closing the file handle. When this is used on a regular basis, the system will fail with too many open file handles: it seems those were addressed in https://github.com/giampaolo/psutil/commit/3760d9593524e553e9e012c6b63cd802e3b25a46 but it was not released in the last upstream version - asked upstream for clarification on when they will be released. Regards. -- Sandro Tosi (aka morph, morpheus, matrixhasu) My website: http://matrixhasu.altervista.org/ Me at Debian: http://wiki.debian.org/SandroTosi -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#772543: python-psutil: severak filehandle leaks
it seems those were addressed in https://github.com/giampaolo/psutil/commit/3760d9593524e553e9e012c6b63cd802e3b25a46 but it was not released in the last upstream version - asked upstream for clarification on when they will be released. I don't think the commit solves all the problems: when looking at the commit, I still see a lot of returns in a try block, while the file handle is only closed in the finally block. This means that the control flow returns before closing. It does seem to solve some open file handles when something fails in the try. The problem here popped up after a weekend and there were no errors to be seen during operation before hitting the file handle limit. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#772543: python-psutil: severak filehandle leaks
control: forward -1 https://github.com/giampaolo/psutil/issues/556 On Mon, Dec 8, 2014 at 4:28 PM, Marc Leeman marc.lee...@gmail.com wrote: it seems those were addressed in https://github.com/giampaolo/psutil/commit/3760d9593524e553e9e012c6b63cd802e3b25a46 but it was not released in the last upstream version - asked upstream for clarification on when they will be released. I don't think the commit solves all the problems: when looking at the commit, I still see a lot of returns in a try block, while the file handle is only closed in the finally block. This means that the control flow returns before closing. It does seem to solve some open file handles when something fails in the try. The problem here popped up after a weekend and there were no errors to be seen during operation before hitting the file handle limit. yup agreed, reported upstream as referred above. Regards, -- Sandro Tosi (aka morph, morpheus, matrixhasu) My website: http://matrixhasu.altervista.org/ Me at Debian: http://wiki.debian.org/SandroTosi -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
