Bug#413661: libblkid1: leaks memory like crazy
Package: libblkid1 Version: 1.39+1.40-WIP-2006.11.14+dfsg-1 Severity: grave Tags: patch (RMs: I'm unsure if this should be fixed for etch or not, given that I do not know of anything in etch that actually uses this library enough for it to leak. Feel free to downgrade or tag etch-ignore, of course.) nfs-kernel-server 1.0.12 has started using libblkid, and leaks absurd amounts of memory (in the order of several megabytes per minute). Tracing using Omega (which is not yet in Debian, but nevertheless really useful) tracked the leaks down to libblkid. I've attached a diff that fixes the issues for me, and allows me to run nfs-kernel-server with only minor leaks (which I'll take with nfs-utils upstream, as they are not libblkid1 related). -- System Information: Debian Release: 4.0 APT prefers unstable APT policy: (500, 'unstable'), (500, 'testing'), (1, 'experimental') Architecture: amd64 (x86_64) Shell: /bin/sh linked to /bin/bash Kernel: Linux 2.6.19 Locale: LANG=en_DK.UTF-8, LC_CTYPE=en_DK.UTF-8 (charmap=UTF-8) Versions of packages libblkid1 depends on: ii libc6 2.3.6.ds1-13 GNU C Library: Shared libraries ii libdevmapper1.022:1.02.12-1 The Linux Kernel Device Mapper use ii libselinux1 1.32-3 SELinux shared libraries ii libsepol1 1.14-3 Security Enhanced Linux policy lib libblkid1 recommends no packages. -- debconf-show failed Only in e2fsprogs-patched/debian: BUILD-BF Only in e2fsprogs-patched/debian: BUILD-STD Only in e2fsprogs-patched/debian: comerr-dev Only in e2fsprogs-patched/debian: comerr-dev.postinst.debhelper Only in e2fsprogs-patched/debian: comerr-dev.prerm.debhelper Only in e2fsprogs-patched/debian: e2fsck-static Only in e2fsprogs-patched/debian: e2fslibs Only in e2fsprogs-patched/debian: e2fslibs-dev Only in e2fsprogs-patched/debian: e2fslibs-dev.postinst.debhelper Only in e2fsprogs-patched/debian: e2fslibs-dev.prerm.debhelper Only in e2fsprogs-patched/debian: e2fslibs.postinst.debhelper Only in e2fsprogs-patched/debian: e2fslibs.postrm.debhelper Only in e2fsprogs-patched/debian: e2fslibs.substvars Only in e2fsprogs-patched/debian: e2fsprogs Only in e2fsprogs-patched/debian: e2fsprogs-udeb Only in e2fsprogs-patched/debian: e2fsprogs-udeb.substvars Only in e2fsprogs-patched/debian: e2fsprogs.substvars Only in e2fsprogs-patched/debian: files Only in e2fsprogs-patched/debian: files~ Only in e2fsprogs-patched/debian: libblkid-dev Only in e2fsprogs-patched/debian: libblkid1 Only in e2fsprogs-patched/debian: libblkid1-udeb Only in e2fsprogs-patched/debian: libblkid1-udeb.substvars Only in e2fsprogs-patched/debian: libblkid1.postinst.debhelper Only in e2fsprogs-patched/debian: libblkid1.postrm.debhelper Only in e2fsprogs-patched/debian: libblkid1.substvars Only in e2fsprogs-patched/debian: libcomerr2 Only in e2fsprogs-patched/debian: libcomerr2.postinst.debhelper Only in e2fsprogs-patched/debian: libcomerr2.postrm.debhelper Only in e2fsprogs-patched/debian: libcomerr2.substvars Only in e2fsprogs-patched/debian: libss2 Only in e2fsprogs-patched/debian: libss2.postinst.debhelper Only in e2fsprogs-patched/debian: libss2.postrm.debhelper Only in e2fsprogs-patched/debian: libss2.substvars Only in e2fsprogs-patched/debian: libuuid1 Only in e2fsprogs-patched/debian: libuuid1-udeb Only in e2fsprogs-patched/debian: libuuid1-udeb.substvars Only in e2fsprogs-patched/debian: libuuid1.postinst.debhelper Only in e2fsprogs-patched/debian: libuuid1.postrm.debhelper Only in e2fsprogs-patched/debian: libuuid1.substvars diff -ur e2fsprogs-1.39+1.40-WIP-2006.11.14+dfsg/debian/rules e2fsprogs-patched/debian/rules --- e2fsprogs-1.39+1.40-WIP-2006.11.14+dfsg/debian/rules2006-11-13 02:19:40.0 +0100 +++ e2fsprogs-patched/debian/rules 2007-03-06 13:41:53.0 +0100 @@ -70,7 +70,7 @@ INSTALL_PROGRAM = $(INSTALL) -p -o root -g root -m 0755 ifeq (,$(findstring noopt,$(DEB_BUILD_OPTIONS))) -CCOPTS += -O2 +CCOPTS += -O0 endif ifeq (,$(findstring nostrip,$(DEB_BUILD_OPTIONS))) INSTALL_PROGRAM += -s @@ -356,9 +356,9 @@ -pe2fslibs -puuid-dev -pe2fsck-static ifneq ($(ismips),) - dh_strip -Xlib64ext2fs-nopic.a + #dh_strip -Xlib64ext2fs-nopic.a else - dh_strip + #dh_strip endif dh_compress Only in e2fsprogs-patched/debian: ss-dev Only in e2fsprogs-patched/debian: stampdir Only in e2fsprogs-patched/debian: tmp Only in e2fsprogs-patched/debian: uuid-dev diff -ur e2fsprogs-1.39+1.40-WIP-2006.11.14+dfsg/lib/blkid/devname.c e2fsprogs-patched/lib/blkid/devname.c --- e2fsprogs-1.39+1.40-WIP-2006.11.14+dfsg/lib/blkid/devname.c 2006-09-12 07:25:40.0 +0200 +++ e2fsprogs-patched/lib/blkid/devname.c 2007-03-06 14:00:03.0 +0100 @@ -305,6 +305,7 @@ probe_one(cache, device, dev, BLKID_PRI_DM, only_if_new); try_next: + free(device); next = names-next; } while (next); diff -ur
Bug#413661: libblkid1: leaks memory like crazy
On Tue, Mar 06, 2007 at 02:08:43PM +0100, Steinar H. Gunderson wrote:
Only in e2fsprogs-patched/debian: BUILD-BF
Only in e2fsprogs-patched/debian: BUILD-STD
Only in e2fsprogs-patched/debian: comerr-dev
Only in e2fsprogs-patched/debian: comerr-dev.postinst.debhelper
Feh, I attached the wrong patch. Disregarding the debian/rules hunks etc.,
the correct patch becomes:
diff -ur e2fsprogs-1.39+1.40-WIP-2006.11.14+dfsg/lib/blkid/devname.c
e2fsprogs-patched/lib/blkid/devname.c
--- e2fsprogs-1.39+1.40-WIP-2006.11.14+dfsg/lib/blkid/devname.c 2006-09-12
07:25:40.0 +0200
+++ e2fsprogs-patched/lib/blkid/devname.c 2007-03-06 14:00:03.0
+0100
@@ -305,6 +305,7 @@
probe_one(cache, device, dev, BLKID_PRI_DM, only_if_new);
try_next:
+ free(device);
next = names-next;
} while (next);
diff -ur e2fsprogs-1.39+1.40-WIP-2006.11.14+dfsg/lib/blkid/probe.c
e2fsprogs-patched/lib/blkid/probe.c
--- e2fsprogs-1.39+1.40-WIP-2006.11.14+dfsg/lib/blkid/probe.c 2006-09-18
03:12:28.0 +0200
+++ e2fsprogs-patched/lib/blkid/probe.c 2007-03-06 13:55:19.0 +0100
@@ -888,6 +888,10 @@
if (!dev-bid_type) {
if (probe.fd = 0) close(probe.fd);
blkid_free_dev(dev);
+ if (probe.sbbuf)
+ free(probe.sbbuf);
+ if (probe.buf)
+ free(probe.buf);
return NULL;
}
/* Steinar */
--
Homepage: http://www.sesse.net/
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#413661: libblkid1: leaks memory like crazy
On Tue, Mar 06, 2007 at 02:08:43PM +0100, Steinar H. Gunderson wrote: Package: libblkid1 Version: 1.39+1.40-WIP-2006.11.14+dfsg-1 Severity: grave Tags: patch (RMs: I'm unsure if this should be fixed for etch or not, given that I do not know of anything in etch that actually uses this library enough for it to leak. Feel free to downgrade or tag etch-ignore, of course.) nfs-kernel-server 1.0.12 has started using libblkid, and leaks absurd amounts of memory (in the order of several megabytes per minute). Tracing using Omega (which is not yet in Debian, but nevertheless really useful) tracked the leaks down to libblkid. I've attached a diff that fixes the issues for me, and allows me to run nfs-kernel-server with only minor leaks (which I'll take with nfs-utils upstream, as they are not libblkid1 related). Yikes, what blkid function is rpc.mountd calling all the time which is causing this kind of memory leakage? Sounds like a bad thing from a power/CPU utilization standpoint (note to self; shut down rpc.mountd when I'm running on batteries on my laptop). I'm also curious why we only discovered the problem this late in the freeze Oh, I see, because nfs-kernel-server 1.0.10 is in testing, and 1.0.12 is in unstable. I assume it's unlikely that nfs-kernel-server 1.0.12 is going to migrate to testing? OK, release managers, the memory leak primarily occurs in the device mapper probing code, and an additional leak when there is a partition which does not have a valid filesystem known to blkid. rpc.mountd is, as far as I know, the first long-lived daemon using blkid, and it's apparently using it in a somewhat interesting way which is the system to constantly reprobe the disk information (if it is leaking megabytes per minute). The patch looks reasonably sane and low risk based on a on-paper examination of the code changes --- but if nfs-kernel-server 1.0.12 isn't going into etch, I'd say it's borderline whether or not we include this patch into the e2fsprogs upload I was about to do. What say ye? - Ted P.S. I really want to look at the sources of nfs-kernel-server 1.0.12 and see what it's *doing* that could be provoking a memory leak of the rate that you are describing. The memory leak is clearly blkid's fault, but the rate at which memory is leaking smells very fishy. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#413661: libblkid1: leaks memory like crazy
On Tue, Mar 06, 2007 at 10:44:55AM -0500, Theodore Tso wrote: Yikes, what blkid function is rpc.mountd calling all the time which is causing this kind of memory leakage? blkid_probe_all_new() seems to be the one. It happens at every mount and umount, I believe; it's not like it's being called all the time (it leaks much faster on busy systems than on almost idle systems, AFAICS), it's that it leaks so _much_ memory all the time. Oh, I see, because nfs-kernel-server 1.0.10 is in testing, and 1.0.12 is in unstable. I assume it's unlikely that nfs-kernel-server 1.0.12 is going to migrate to testing? I was originally planning to ask for an exception (with associated RM bribes, the diff is quite large), but given 1.0.11's and 1.0.12's track record so far, I think that's quite far-fetched, yes. OK, release managers, the memory leak primarily occurs in the device mapper probing code, and an additional leak when there is a partition which does not have a valid filesystem known to blkid. rpc.mountd is, as far as I know, the first long-lived daemon using blkid, and it's apparently using it in a somewhat interesting way which is the system to constantly reprobe the disk information (if it is leaking megabytes per minute). The patch looks reasonably sane and low risk based on a on-paper examination of the code changes --- but if nfs-kernel-server 1.0.12 isn't going into etch, I'd say it's borderline whether or not we include this patch into the e2fsprogs upload I was about to do. FWIW, I agree with this analysis, possibly except the part about mountd's usage pattern. :-) /* Steinar */ -- Homepage: http://www.sesse.net/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#413661: libblkid1: leaks memory like crazy
On Tue, Mar 06, 2007 at 04:54:41PM +0100, Steinar H. Gunderson wrote: On Tue, Mar 06, 2007 at 10:44:55AM -0500, Theodore Tso wrote: Yikes, what blkid function is rpc.mountd calling all the time which is causing this kind of memory leakage? blkid_probe_all_new() seems to be the one. It happens at every mount and umount, I believe; it's not like it's being called all the time (it leaks much faster on busy systems than on almost idle systems, AFAICS), it's that it leaks so _much_ memory all the time. Well, the amount of memory leaked depends on how many device mapper volumes you have, and how many of them contain unitialized volumes. And I guess this must have been on a system with a lot of mounts and unmounts. - Ted -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#413661: libblkid1: leaks memory like crazy
severity 413661 important thanks On Tue, Mar 06, 2007 at 02:08:43PM +0100, Steinar H. Gunderson wrote: (RMs: I'm unsure if this should be fixed for etch or not, given that I do not know of anything in etch that actually uses this library enough for it to leak. Feel free to downgrade or tag etch-ignore, of course.) Indeed, nfs-kernel-server in unstable is the only daemon using the lib; all the other reverse-deps are short-lived utils. So, downgrading. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. [EMAIL PROTECTED] http://www.debian.org/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Processed: Re: Bug#413661: libblkid1: leaks memory like crazy
Processing commands for [EMAIL PROTECTED]: severity 413661 important Bug#413661: libblkid1: leaks memory like crazy Severity set to `important' from `grave' thanks Stopping processing here. Please contact me if you need assistance. Debian bug tracking system administrator (administrator, Debian Bugs database) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
