On Jan 8, 2007, at 12:42 PM, Nicholas Clark wrote:
On Mon, Jan 08, 2007 at 12:23:09PM -0800, Marvin Humphrey wrote:
Unfortunately, I've never succeeded in banishing all of Perl's leaks
when debugging XS under either 5.8.8 or 5.9.4. Now that we're having
this discussion, I will attempt to write up a test case illustrating
the problem.
Curious. We thought that we'd nailed (most?) every leak in current
blead.
As of about a week ago, Dave got the last known eval leaks.
(Although the fix is fragile, and if it causes more problems than
it solves,
he'll back it out)
Upon further investigation, it turns out that the leak only arises
when running under -Mblib. I'd never actually done any valgrind
testing against installed modules until now, so I was unaware that
blib was the culprit.
I've prepared a minimal distro which illustrates the problem: <http://
www.rectangular.com/downloads/XSLeaker.tar.gz>. As this is XS code,
I don't know of a way to simplify down to a single Perl test script,
but here are the relevant files:
leak.pl
use XSLeaker;
print "foo\n";
no_leak.pl
print "foo\n";
lib/XSLeaker.xs
#include "EXTERN.h"
#include "perl.h"
#include "XSUB.h"
MODULE = XSLeaker PACKAGE = XSLeaker
SV*
xs_dummy_func()
CODE:
RETVAL = newSVpvn("foo\n", 4);
OUTPUT: RETVAL
lib/XSLeaker.pm
package XSLeaker;
our $VERSION = '0.01';
use XSLoader;
XSLoader::load( 'XSLeaker', $VERSION );
1;
Pretty basic.
Below my sig, you'll find output from these commands:
$ valgrind --leak-check=full /usr/local/dblead/bin/perl5.9.5 -e1
$ valgrind --leak-check=full /usr/local/dblead/bin/perl5.9.5 leak.pl
$ valgrind --leak-check=full /usr/local/dblead/bin/perl5.9.5 -Mblib
no_leak.pl
$ valgrind --leak-check=full /usr/local/dblead/bin/perl5.9.5 -Mblib
leak.pl
$ valgrind --leak-check=full --show-reachable=yes /usr/local/dblead/
bin/perl5.9.5 -Mblib leak.pl
$ /usr/local/dblead/bin/perl5.9.5 -V
Marvin Humphrey
Rectangular Research
http://www.rectangular.com/
$ valgrind --leak-check=full /usr/local/dblead/bin/perl5.9.5 -e1
==1374== Memcheck, a memory error detector.
==1374== Copyright (C) 2002-2005, and GNU GPL'd, by Julian Seward et al.
==1374== Using LibVEX rev 1471, a library for dynamic binary
translation.
==1374== Copyright (C) 2004-2005, and GNU GPL'd, by OpenWorks LLP.
==1374== Using valgrind-3.1.0, a dynamic binary instrumentation
framework.
==1374== Copyright (C) 2000-2005, and GNU GPL'd, by Julian Seward et al.
==1374== For more details, rerun with: -v
==1374==
==1374==
==1374== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 21 from 1)
==1374== malloc/free: in use at exit: 0 bytes in 0 blocks.
==1374== malloc/free: 537 allocs, 537 frees, 85,111 bytes allocated.
==1374== For counts of detected errors, rerun with: -v
==1374== No malloc'd blocks -- no leaks are possible.
$ valgrind --leak-check=full /usr/local/dblead/bin/perl5.9.5 leak.pl
==1375== Memcheck, a memory error detector.
==1375== Copyright (C) 2002-2005, and GNU GPL'd, by Julian Seward et al.
==1375== Using LibVEX rev 1471, a library for dynamic binary
translation.
==1375== Copyright (C) 2004-2005, and GNU GPL'd, by OpenWorks LLP.
==1375== Using valgrind-3.1.0, a dynamic binary instrumentation
framework.
==1375== Copyright (C) 2000-2005, and GNU GPL'd, by Julian Seward et al.
==1375== For more details, rerun with: -v
==1375==
foo
==1375==
==1375== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 23 from 1)
==1375== malloc/free: in use at exit: 879 bytes in 5 blocks.
==1375== malloc/free: 1,606 allocs, 1,601 frees, 168,686 bytes
allocated.
==1375== For counts of detected errors, rerun with: -v
==1375== searching for pointers to 5 not-freed blocks.
==1375== checked 256,412 bytes.
==1375==
==1375== LEAK SUMMARY:
==1375== definitely lost: 0 bytes in 0 blocks.
==1375== possibly lost: 0 bytes in 0 blocks.
==1375== still reachable: 879 bytes in 5 blocks.
==1375== suppressed: 0 bytes in 0 blocks.
==1375== Reachable blocks (those to which a pointer was found) are
not shown.
==1375== To see them, rerun with: --show-reachable=yes
$ valgrind --leak-check=full /usr/local/dblead/bin/perl5.9.5 -Mblib
no_leak.pl
==1376== Memcheck, a memory error detector.
==1376== Copyright (C) 2002-2005, and GNU GPL'd, by Julian Seward et al.
==1376== Using LibVEX rev 1471, a library for dynamic binary
translation.
==1376== Copyright (C) 2004-2005, and GNU GPL'd, by OpenWorks LLP.
==1376== Using valgrind-3.1.0, a dynamic binary instrumentation
framework.
==1376== Copyright (C) 2000-2005, and GNU GPL'd, by Julian Seward et al.
==1376== For more details, rerun with: -v
==1376==
foo
==1376==
==1376== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 23 from 1)
==1376== malloc/free: in use at exit: 867 bytes in 5 blocks.
==1376== malloc/free: 14,974 allocs, 14,969 frees, 741,385 bytes
allocated.
==1376== For counts of detected errors, rerun with: -v
==1376== searching for pointers to 5 not-freed blocks.
==1376== checked 256,416 bytes.
==1376==
==1376== LEAK SUMMARY:
==1376== definitely lost: 0 bytes in 0 blocks.
==1376== possibly lost: 0 bytes in 0 blocks.
==1376== still reachable: 867 bytes in 5 blocks.
==1376== suppressed: 0 bytes in 0 blocks.
==1376== Reachable blocks (those to which a pointer was found) are
not shown.
==1376== To see them, rerun with: --show-reachable=yes
$ valgrind --leak-check=full /usr/local/dblead/bin/perl5.9.5 -Mblib
leak.pl
==1377== Memcheck, a memory error detector.
==1377== Copyright (C) 2002-2005, and GNU GPL'd, by Julian Seward et al.
==1377== Using LibVEX rev 1471, a library for dynamic binary
translation.
==1377== Copyright (C) 2004-2005, and GNU GPL'd, by OpenWorks LLP.
==1377== Using valgrind-3.1.0, a dynamic binary instrumentation
framework.
==1377== Copyright (C) 2000-2005, and GNU GPL'd, by Julian Seward et al.
==1377== For more details, rerun with: -v
==1377==
foo
==1377==
==1377== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 25 from 1)
==1377== malloc/free: in use at exit: 1,725 bytes in 11 blocks.
==1377== malloc/free: 19,217 allocs, 19,206 frees, 942,482 bytes
allocated.
==1377== For counts of detected errors, rerun with: -v
==1377== searching for pointers to 11 not-freed blocks.
==1377== checked 261,324 bytes.
==1377==
==1377== 24 bytes in 1 blocks are definitely lost in loss record 1 of 6
==1377== at 0x401A619: malloc (vg_replace_malloc.c:149)
==1377== by 0x80928B5: Perl_safesyscalloc (util.c:277)
==1377== by 0x80CA298: Perl_sv_magicext (sv.c:4379)
==1377== by 0x80CA543: Perl_sv_magic (sv.c:4619)
==1377== by 0x8154BAD: Perl_gv_fetchpvn_flags (gv.c:1052)
==1377== by 0x81544C0: Perl_gv_fetchsv (gv.c:792)
==1377== by 0x80D920A: Perl_pp_rv2gv (pp.c:215)
==1377== by 0x80921BE: Perl_runops_debug (dump.c:1898)
==1377== by 0x80AD209: Perl_call_sv (perl.c:2651)
==1377== by 0x80B191E: Perl_call_list (perl.c:5144)
==1377== by 0x8068BF9: Perl_newATTRSUB (op.c:5409)
==1377== by 0x8064697: Perl_utilize (op.c:3576)
==1377==
==1377== LEAK SUMMARY:
==1377== definitely lost: 24 bytes in 1 blocks.
==1377== possibly lost: 0 bytes in 0 blocks.
==1377== still reachable: 1,701 bytes in 10 blocks.
==1377== suppressed: 0 bytes in 0 blocks.
==1377== Reachable blocks (those to which a pointer was found) are
not shown.
==1377== To see them, rerun with: --show-reachable=yes
$ valgrind --leak-check=full --show-reachable=yes /usr/local/dblead/
bin/perl5.9.5 -Mblib leak.pl
==1378== Memcheck, a memory error detector.
==1378== Copyright (C) 2002-2005, and GNU GPL'd, by Julian Seward et al.
==1378== Using LibVEX rev 1471, a library for dynamic binary
translation.
==1378== Copyright (C) 2004-2005, and GNU GPL'd, by OpenWorks LLP.
==1378== Using valgrind-3.1.0, a dynamic binary instrumentation
framework.
==1378== Copyright (C) 2000-2005, and GNU GPL'd, by Julian Seward et al.
==1378== For more details, rerun with: -v
==1378==
foo
==1378==
==1378== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 25 from 1)
==1378== malloc/free: in use at exit: 1,725 bytes in 11 blocks.
==1378== malloc/free: 19,217 allocs, 19,206 frees, 942,482 bytes
allocated.
==1378== For counts of detected errors, rerun with: -v
==1378== searching for pointers to 11 not-freed blocks.
==1378== checked 261,324 bytes.
==1378==
==1378== 24 bytes in 1 blocks are definitely lost in loss record 1 of 6
==1378== at 0x401A619: malloc (vg_replace_malloc.c:149)
==1378== by 0x80928B5: Perl_safesyscalloc (util.c:277)
==1378== by 0x80CA298: Perl_sv_magicext (sv.c:4379)
==1378== by 0x80CA543: Perl_sv_magic (sv.c:4619)
==1378== by 0x8154BAD: Perl_gv_fetchpvn_flags (gv.c:1052)
==1378== by 0x81544C0: Perl_gv_fetchsv (gv.c:792)
==1378== by 0x80D920A: Perl_pp_rv2gv (pp.c:215)
==1378== by 0x80921BE: Perl_runops_debug (dump.c:1898)
==1378== by 0x80AD209: Perl_call_sv (perl.c:2651)
==1378== by 0x80B191E: Perl_call_list (perl.c:5144)
==1378== by 0x8068BF9: Perl_newATTRSUB (op.c:5409)
==1378== by 0x8064697: Perl_utilize (op.c:3576)
==1378==
==1378==
==1378== 56 bytes in 2 blocks are still reachable in loss record 2 of 6
==1378== at 0x401A619: malloc (vg_replace_malloc.c:149)
==1378== by 0x400B245: _dl_map_object_deps (in /lib/ld-2.3.2.so)
==1378== by 0x41A1DBA: dl_open_worker (in /lib/libc-2.3.2.so)
==1378== by 0x400C265: _dl_catch_error (in /lib/ld-2.3.2.so)
==1378== by 0x41A19AE: _dl_open (in /lib/libc-2.3.2.so)
==1378== by 0x4039F6A: dlopen_doit (in /lib/libdl-2.3.2.so)
==1378== by 0x400C265: _dl_catch_error (in /lib/ld-2.3.2.so)
==1378== by 0x403A315: _dlerror_run (in /lib/libdl-2.3.2.so)
==1378== by 0x4039F13: dlopen@@GLIBC_2.1 (in /lib/libdl-2.3.2.so)
==1378== by 0x8151060: XS_DynaLoader_dl_load_file (DynaLoader.xs:191)
==1378== by 0x80BC573: Perl_pp_entersub (pp_hot.c:2900)
==1378== by 0x80921BE: Perl_runops_debug (dump.c:1898)
==1378==
==1378==
==1378== 127 bytes in 2 blocks are still reachable in loss record 3 of 6
==1378== at 0x401A619: malloc (vg_replace_malloc.c:149)
==1378== by 0x400968D: _dl_new_object (in /lib/ld-2.3.2.so)
==1378== by 0x40055DA: _dl_map_object_from_fd (in /lib/ld-2.3.2.so)
==1378== by 0x400468A: _dl_map_object (in /lib/ld-2.3.2.so)
==1378== by 0x41A1B6A: dl_open_worker (in /lib/libc-2.3.2.so)
==1378== by 0x400C265: _dl_catch_error (in /lib/ld-2.3.2.so)
==1378== by 0x41A19AE: _dl_open (in /lib/libc-2.3.2.so)
==1378== by 0x4039F6A: dlopen_doit (in /lib/libdl-2.3.2.so)
==1378== by 0x400C265: _dl_catch_error (in /lib/ld-2.3.2.so)
==1378== by 0x403A315: _dlerror_run (in /lib/libdl-2.3.2.so)
==1378== by 0x4039F13: dlopen@@GLIBC_2.1 (in /lib/libdl-2.3.2.so)
==1378== by 0x8151060: XS_DynaLoader_dl_load_file (DynaLoader.xs:191)
==1378==
==1378==
==1378== 127 bytes in 2 blocks are still reachable in loss record 4 of 6
==1378== at 0x401A619: malloc (vg_replace_malloc.c:149)
==1378== by 0x4004868: _dl_map_object (in /lib/ld-2.3.2.so)
==1378== by 0x41A1B6A: dl_open_worker (in /lib/libc-2.3.2.so)
==1378== by 0x400C265: _dl_catch_error (in /lib/ld-2.3.2.so)
==1378== by 0x41A19AE: _dl_open (in /lib/libc-2.3.2.so)
==1378== by 0x4039F6A: dlopen_doit (in /lib/libdl-2.3.2.so)
==1378== by 0x400C265: _dl_catch_error (in /lib/ld-2.3.2.so)
==1378== by 0x403A315: _dlerror_run (in /lib/libdl-2.3.2.so)
==1378== by 0x4039F13: dlopen@@GLIBC_2.1 (in /lib/libdl-2.3.2.so)
==1378== by 0x8151060: XS_DynaLoader_dl_load_file (DynaLoader.xs:191)
==1378== by 0x80BC573: Perl_pp_entersub (pp_hot.c:2900)
==1378== by 0x80921BE: Perl_runops_debug (dump.c:1898)
==1378==
==1378==
==1378== 144 bytes in 2 blocks are still reachable in loss record 5 of 6
==1378== at 0x401B8F1: calloc (vg_replace_malloc.c:279)
==1378== by 0x400D035: _dl_check_map_versions (in /lib/ld-2.3.2.so)
==1378== by 0x41A23C8: dl_open_worker (in /lib/libc-2.3.2.so)
==1378== by 0x400C265: _dl_catch_error (in /lib/ld-2.3.2.so)
==1378== by 0x41A19AE: _dl_open (in /lib/libc-2.3.2.so)
==1378== by 0x4039F6A: dlopen_doit (in /lib/libdl-2.3.2.so)
==1378== by 0x400C265: _dl_catch_error (in /lib/ld-2.3.2.so)
==1378== by 0x403A315: _dlerror_run (in /lib/libdl-2.3.2.so)
==1378== by 0x4039F13: dlopen@@GLIBC_2.1 (in /lib/libdl-2.3.2.so)
==1378== by 0x8151060: XS_DynaLoader_dl_load_file (DynaLoader.xs:191)
==1378== by 0x80BC573: Perl_pp_entersub (pp_hot.c:2900)
==1378== by 0x80921BE: Perl_runops_debug (dump.c:1898)
==1378==
==1378==
==1378== 1,247 bytes in 2 blocks are still reachable in loss record 6
of 6
==1378== at 0x401B8F1: calloc (vg_replace_malloc.c:279)
==1378== by 0x4009472: _dl_new_object (in /lib/ld-2.3.2.so)
==1378== by 0x40055DA: _dl_map_object_from_fd (in /lib/ld-2.3.2.so)
==1378== by 0x400468A: _dl_map_object (in /lib/ld-2.3.2.so)
==1378== by 0x41A1B6A: dl_open_worker (in /lib/libc-2.3.2.so)
==1378== by 0x400C265: _dl_catch_error (in /lib/ld-2.3.2.so)
==1378== by 0x41A19AE: _dl_open (in /lib/libc-2.3.2.so)
==1378== by 0x4039F6A: dlopen_doit (in /lib/libdl-2.3.2.so)
==1378== by 0x400C265: _dl_catch_error (in /lib/ld-2.3.2.so)
==1378== by 0x403A315: _dlerror_run (in /lib/libdl-2.3.2.so)
==1378== by 0x4039F13: dlopen@@GLIBC_2.1 (in /lib/libdl-2.3.2.so)
==1378== by 0x8151060: XS_DynaLoader_dl_load_file (DynaLoader.xs:191)
==1378==
==1378== LEAK SUMMARY:
==1378== definitely lost: 24 bytes in 1 blocks.
==1378== possibly lost: 0 bytes in 0 blocks.
==1378== still reachable: 1,701 bytes in 10 blocks.
==1378== suppressed: 0 bytes in 0 blocks.
$ /usr/local/dblead/bin/perl5.9.5 -V
Summary of my perl5 (revision 5 version 9 subversion 5 patch 29719)
configuration:
Platform:
osname=linux, osvers=2.4.20-8bigmem, archname=i686-linux
uname='linux XXX.XXXXXX.com 2.4.20-8bigmem #1 smp thu mar 13
17:32:29 est 2003 i686 i686 i386 gnulinux '
config_args='-Dprefix=/usr/local/dblead -DEBUGGING=both -de -
Dusedevel'
hint=recommended, useposix=true, d_sigaction=define
useithreads=undef, usemultiplicity=undef
useperlio=define, d_sfio=undef, uselargefiles=define,
usesocks=undef
use64bitint=undef, use64bitall=undef, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='cc', ccflags ='-DDEBUGGING -fno-strict-aliasing -pipe -I/usr/
local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/
include/gdbm',
optimize='-O2 -g',
cppflags='-DDEBUGGING -fno-strict-aliasing -pipe -I/usr/local/
include -I/usr/include/gdbm'
ccversion='', gccversion='3.2.2 20030222 (Red Hat Linux
3.2.2-5)', gccosandvers=''
intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
ivtype='long', ivsize=4, nvtype='double', nvsize=8,
Off_t='off_t', lseeksize=8
alignbytes=4, prototype=define
Linker and Libraries:
ld='cc', ldflags =' -L/usr/local/lib'
libpth=/usr/local/lib /lib /usr/lib
libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc
perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc
libc=/lib/libc-2.3.2.so, so=so, useshrplib=false, libperl=libperl.a
gnulibc_version='2.3.2'
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib'
Characteristics of this binary (from libperl):
Compile-time options: DEBUGGING PERL_DONT_CREATE_GVSV
PERL_MALLOC_WRAP
USE_LARGE_FILES USE_PERLIO
Locally applied patches:
DEVEL
Built under linux
Compiled at Jan 8 2007 15:29:05
%ENV:
PERL_DESTRUCT_LEVEL="2"
@INC:
/usr/local/dblead/lib/perl5/5.9.5/i686-linux
/usr/local/dblead/lib/perl5/5.9.5
/usr/local/dblead/lib/perl5/site_perl/5.9.5/i686-linux
/usr/local/dblead/lib/perl5/site_perl/5.9.5
.
$
