On Fri, Feb 08, 2002 at 04:29:33PM +0000, Richard Clamp wrote:
> Last night at the London.pm meeting I mentioned to Nick Clark that I
> was puzzling over the problem of ensuring suitable test coverage for
> ..xs modules, to which he kindly put me out of my misery and pointed me
> at gcov.
>
> This is the recipie I'm currently working on. It's a little messy,
> and guaranteed fragile, but it works for me, and that's a step
> forwards.
>
> In Makefile.PL add:
>
> sub MY::postamble {
> return <<EOF
> gcov:
> \trm -f `basename \$(XS_FILES)`.{bb,bbg,da,o}
> \t\$(MAKE) test CCFLAGS="\$(CCFLAGS) -fprofile-arcs -ftest-coverage"
> \tgcov \$(XS_FILES)
> EOF
> }
I suspect that adding -O0 to the flags would be useful, otherwise the
optimiser can remove some of the contructs you are trying to test.
> Then C<make gcov> and poke around at the generated Foo.xs.gcov
> results.
I have been (slowly) investigating how to integrate gcov with the
backend of Devel::Cover. I deliberately designed the backend to be
language agnostic. Unfortunately, I've not had much time to work on it
recently.
One thing has become very apparent though. Perl still has many untested
corners.
Here's where I'm up to. This is from a debugging bleadperl last night:
---------------------------- ------ ------ ------ ------ ------ ------ ------
File stmt branch path cond pod time total
---------------------------- ------ ------ ------ ------ ------ ------ ------
av.c 80.84 n/a n/a n/a n/a n/a 80.84
deb.c 0.00 n/a n/a n/a n/a n/a 0.00
doio.c 75.70 n/a n/a n/a n/a n/a 75.70
doop.c 86.90 n/a n/a n/a n/a n/a 86.90
dump.c 55.46 n/a n/a n/a n/a n/a 55.46
ext/B/B.c 40.88 n/a n/a n/a n/a n/a 40.88
ext/B/B.xs 50.17 n/a n/a n/a n/a n/a 50.17
ext/B/C/C.c 0.00 n/a n/a n/a n/a n/a 0.00
ext/B/C/C.xs 0.00 n/a n/a n/a n/a n/a 0.00
ext/B/defsubs.h 100.00 n/a n/a n/a n/a n/a 100.00
ext/ByteLoader/ByteLoader.c 24.14 n/a n/a n/a n/a n/a 24.14
ext/ByteLoader/ByteLoader.xs 0.00 n/a n/a n/a n/a n/a 0.00
ext/ByteLoader/byterun.c 0.00 n/a n/a n/a n/a n/a 0.00
ext/Cwd/Cwd.c 89.66 n/a n/a n/a n/a n/a 89.66
ext/Cwd/Cwd.xs 69.05 n/a n/a n/a n/a n/a 69.05
ext/Data/Dumper/Dumper.c 93.33 n/a n/a n/a n/a n/a 93.33
ext/Data/Dumper/Dumper.xs 87.05 n/a n/a n/a n/a n/a 87.05
ext/Devel/DProf/DProf.c 65.22 n/a n/a n/a n/a n/a 65.22
ext/Devel/DProf/DProf.xs 81.72 n/a n/a n/a n/a n/a 81.72
ext/Devel/PPPort/PPPort.c 90.00 n/a n/a n/a n/a n/a 90.00
ext/Devel/PPPort/PPPort.xs 100.00 n/a n/a n/a n/a n/a 100.00
ext/Devel/PPPort/module2.c 100.00 n/a n/a n/a n/a n/a 100.00
ext/Devel/PPPort/module3.c 100.00 n/a n/a n/a n/a n/a 100.00
ext/Devel/Peek/Peek.c 23.27 n/a n/a n/a n/a n/a 23.27
ext/Devel/Peek/Peek.xs 5.22 n/a n/a n/a n/a n/a 5.22
ext/Digest/MD5/MD5.c 79.73 n/a n/a n/a n/a n/a 79.73
ext/Digest/MD5/MD5.xs 92.24 n/a n/a n/a n/a n/a 92.24
ext/DynaLoader/DynaLoader.c 63.08 n/a n/a n/a n/a n/a 63.08
ext/DynaLoader/DynaLoader.xs 67.57 n/a n/a n/a n/a n/a 67.57
ext/DynaLoader/dlutils.c 50.00 n/a n/a n/a n/a n/a 50.00
ext/Encode/8859_def.h 100.00 n/a n/a n/a n/a n/a 100.00
ext/Encode/EBCDIC_def.h 100.00 n/a n/a n/a n/a n/a 100.00
ext/Encode/EUC_JP/EUC_JP.c 0.00 n/a n/a n/a n/a n/a 0.00
....ncode/EUC_JP/EUC_JP_def.h 0.00 n/a n/a n/a n/a n/a 0.00
ext/Encode/Encode.c 47.06 n/a n/a n/a n/a n/a 47.06
ext/Encode/Encode.xs 57.36 n/a n/a n/a n/a n/a 57.36
ext/Encode/Symbols_def.h 100.00 n/a n/a n/a n/a n/a 100.00
ext/Encode/encengine.c 91.84 n/a n/a n/a n/a n/a 91.84
ext/Fcntl/Fcntl.c 93.75 n/a n/a n/a n/a n/a 93.75
ext/Fcntl/constants.c 17.22 n/a n/a n/a n/a n/a 17.22
ext/Fcntl/constants.xs 82.35 n/a n/a n/a n/a n/a 82.35
ext/File/Glob/Glob.c 92.31 n/a n/a n/a n/a n/a 92.31
ext/File/Glob/Glob.xs 100.00 n/a n/a n/a n/a n/a 100.00
ext/File/Glob/bsd_glob.c 69.60 n/a n/a n/a n/a n/a 69.60
ext/File/Glob/constants.c 34.41 n/a n/a n/a n/a n/a 34.41
ext/File/Glob/constants.xs 41.18 n/a n/a n/a n/a n/a 41.18
ext/Filter/Util/Call/Call.c 72.00 n/a n/a n/a n/a n/a 72.00
ext/Filter/Util/Call/Call.xs 77.55 n/a n/a n/a n/a n/a 77.55
ext/GDBM_File/GDBM_File.c 71.10 n/a n/a n/a n/a n/a 71.10
ext/GDBM_File/GDBM_File.xs 85.71 n/a n/a n/a n/a n/a 85.71
ext/GDBM_File/constants.c 11.11 n/a n/a n/a n/a n/a 11.11
ext/GDBM_File/constants.xs 41.18 n/a n/a n/a n/a n/a 41.18
ext/I18N/Langinfo/Langinfo.c 92.31 n/a n/a n/a n/a n/a 92.31
....I18N/Langinfo/Langinfo.xs 80.00 n/a n/a n/a n/a n/a 80.00
....I18N/Langinfo/constants.c 2.57 n/a n/a n/a n/a n/a 2.57
....18N/Langinfo/constants.xs 41.18 n/a n/a n/a n/a n/a 41.18
ext/IO/IO.c 53.33 n/a n/a n/a n/a n/a 53.33
ext/IO/IO.xs 59.59 n/a n/a n/a n/a n/a 59.59
ext/IPC/SysV/SysV.c 39.68 n/a n/a n/a n/a n/a 39.68
ext/IPC/SysV/SysV.xs 56.04 n/a n/a n/a n/a n/a 56.04
ext/List/Util/Util.c 81.30 n/a n/a n/a n/a n/a 81.30
ext/List/Util/Util.xs 93.41 n/a n/a n/a n/a n/a 93.41
ext/MIME/Base64/Base64.c 92.00 n/a n/a n/a n/a n/a 92.00
ext/MIME/Base64/Base64.xs 97.67 n/a n/a n/a n/a n/a 97.67
ext/NDBM_File/NDBM_File.c 76.34 n/a n/a n/a n/a n/a 76.34
ext/NDBM_File/NDBM_File.xs 84.21 n/a n/a n/a n/a n/a 84.21
ext/Opcode/Opcode.c 84.25 n/a n/a n/a n/a n/a 84.25
ext/Opcode/Opcode.xs 82.49 n/a n/a n/a n/a n/a 82.49
ext/POSIX/POSIX.c 16.94 n/a n/a n/a n/a n/a 16.94
ext/POSIX/POSIX.xs 27.95 n/a n/a n/a n/a n/a 27.95
ext/POSIX/constants.c 4.39 n/a n/a n/a n/a n/a 4.39
ext/POSIX/constants.xs 24.14 n/a n/a n/a n/a n/a 24.14
ext/PerlIO/Scalar/Scalar.c 100.00 n/a n/a n/a n/a n/a 100.00
ext/PerlIO/Scalar/Scalar.xs 75.00 n/a n/a n/a n/a n/a 75.00
ext/PerlIO/Via/Via.c 100.00 n/a n/a n/a n/a n/a 100.00
ext/PerlIO/Via/Via.xs 58.12 n/a n/a n/a n/a n/a 58.12
ext/SDBM_File/SDBM_File.c 78.28 n/a n/a n/a n/a n/a 78.28
ext/SDBM_File/SDBM_File.xs 76.67 n/a n/a n/a n/a n/a 76.67
ext/SDBM_File/sdbm/hash.c 78.57 n/a n/a n/a n/a n/a 78.57
ext/SDBM_File/sdbm/pair.c 80.51 n/a n/a n/a n/a n/a 80.51
ext/SDBM_File/sdbm/sdbm.c 58.42 n/a n/a n/a n/a n/a 58.42
ext/Socket/Socket.c 81.33 n/a n/a n/a n/a n/a 81.33
ext/Socket/Socket.xs 74.68 n/a n/a n/a n/a n/a 74.68
ext/Socket/constants.c 8.60 n/a n/a n/a n/a n/a 8.60
ext/Socket/constants.xs 52.38 n/a n/a n/a n/a n/a 52.38
ext/Storable/Storable.c 91.45 n/a n/a n/a n/a n/a 91.45
ext/Storable/Storable.xs 78.88 n/a n/a n/a n/a n/a 78.88
ext/Sys/Hostname/Hostname.c 93.33 n/a n/a n/a n/a n/a 93.33
ext/Sys/Hostname/Hostname.xs 88.89 n/a n/a n/a n/a n/a 88.89
ext/Sys/Syslog/Syslog.c 53.73 n/a n/a n/a n/a n/a 53.73
ext/Sys/Syslog/Syslog.xs 40.00 n/a n/a n/a n/a n/a 40.00
ext/Sys/Syslog/constants.c 12.93 n/a n/a n/a n/a n/a 12.93
ext/Sys/Syslog/constants.xs 52.38 n/a n/a n/a n/a n/a 52.38
ext/Time/HiRes/HiRes.c 90.57 n/a n/a n/a n/a n/a 90.57
ext/Time/HiRes/HiRes.xs 83.50 n/a n/a n/a n/a n/a 83.50
....ode/Normalize/Normalize.c 88.89 n/a n/a n/a n/a n/a 88.89
....de/Normalize/Normalize.xs 93.60 n/a n/a n/a n/a n/a 93.60
....nicode/Normalize/unfexc.h 100.00 n/a n/a n/a n/a n/a 100.00
ext/XS/Typemap/Typemap.c 87.28 n/a n/a n/a n/a n/a 87.28
ext/XS/Typemap/Typemap.xs 98.36 n/a n/a n/a n/a n/a 98.36
ext/XS/Typemap/stdio.c 75.00 n/a n/a n/a n/a n/a 75.00
ext/attrs/attrs.c 0.00 n/a n/a n/a n/a n/a 0.00
ext/attrs/attrs.xs 0.00 n/a n/a n/a n/a n/a 0.00
ext/re/re.c 63.16 n/a n/a n/a n/a n/a 63.16
ext/re/re.xs 52.17 n/a n/a n/a n/a n/a 52.17
ext/re/re_comp.c 0.00 n/a n/a n/a n/a n/a 0.00
ext/re/re_exec.c 0.00 n/a n/a n/a n/a n/a 0.00
globals.c 0.00 n/a n/a n/a n/a n/a 0.00
gv.c 89.26 n/a n/a n/a n/a n/a 89.26
hv.c 75.52 n/a n/a n/a n/a n/a 75.52
intrpvar.h 0.00 n/a n/a n/a n/a n/a 0.00
locale.c 63.96 n/a n/a n/a n/a n/a 63.96
mg.c 81.37 n/a n/a n/a n/a n/a 81.37
miniperlmain.c 85.71 n/a n/a n/a n/a n/a 85.71
numeric.c 85.29 n/a n/a n/a n/a n/a 85.29
op.c 91.13 n/a n/a n/a n/a n/a 91.13
perl.c 82.44 n/a n/a n/a n/a n/a 82.44
perlio.c 64.14 n/a n/a n/a n/a n/a 64.14
perlmain.c 86.36 n/a n/a n/a n/a n/a 86.36
perly.c 62.87 n/a n/a n/a n/a n/a 62.87
perly.y 97.00 n/a n/a n/a n/a n/a 97.00
pp.c 92.92 n/a n/a n/a n/a n/a 92.92
pp_ctl.c 82.74 n/a n/a n/a n/a n/a 82.74
pp_hot.c 90.89 n/a n/a n/a n/a n/a 90.89
pp_pack.c 89.89 n/a n/a n/a n/a n/a 89.89
pp_sort.c 47.66 n/a n/a n/a n/a n/a 47.66
pp_sys.c 77.18 n/a n/a n/a n/a n/a 77.18
regcomp.c 83.56 n/a n/a n/a n/a n/a 83.56
regexec.c 78.30 n/a n/a n/a n/a n/a 78.30
run.c 0.00 n/a n/a n/a n/a n/a 0.00
scope.c 78.00 n/a n/a n/a n/a n/a 78.00
sv.c 86.69 n/a n/a n/a n/a n/a 86.69
taint.c 91.94 n/a n/a n/a n/a n/a 91.94
thrdvar.h 0.00 n/a n/a n/a n/a n/a 0.00
toke.c 87.54 n/a n/a n/a n/a n/a 87.54
universal.c 83.71 n/a n/a n/a n/a n/a 83.71
utf8.c 68.38 n/a n/a n/a n/a n/a 68.38
util.c 64.17 n/a n/a n/a n/a n/a 64.17
x2p/a2p.y 0.00 n/a n/a n/a n/a n/a 0.00
x2p/a2py.c 0.00 n/a n/a n/a n/a n/a 0.00
x2p/hash.c 0.00 n/a n/a n/a n/a n/a 0.00
x2p/str.c 0.00 n/a n/a n/a n/a n/a 0.00
x2p/util.c 0.00 n/a n/a n/a n/a n/a 0.00
x2p/walk.c 0.00 n/a n/a n/a n/a n/a 0.00
xsutils.c 59.44 n/a n/a n/a n/a n/a 59.44
Total 63.36 n/a n/a n/a n/a n/a 63.36
---------------------------- ------ ------ ------ ------ ------ ------ ------
Caveats:
- Devel::Cover is still alpha code and there are obviously bugs.
- The data is from running make test.
- The fork test failed.
- gcov seems to have the odd quirk, or else I misunderstand some things.
- There's only statement coverage here.
- Usual coverage caveats apply.
- The URL is to my home machine so apologies if it is down or slow.
- Max upload speed from here is 128kbps and some of the files are
large - the biggest is 14M. The interesting ones are mostly < 1M
though.
- Special offer: download the lot at less than 2M.
- I'll leave the data there for a while, but not for ever.
http://pjcj.sytes.net/cover.12493/perl_c_db/cover_db.html
compressed data at
http://pjcj.sytes.net/cover.12493/
and the perl data from last November is still at
http://pjcj.sytes.net/cover.12493/cover_db/cover_db.html
username: perl
password: cover
Summary of my perl5 (revision 5.0 version 7 subversion 2 patch 14534) configuration:
Platform:
osname=linux, osvers=2.4.16, archname=i686-linux
uname='linux wesley 2.4.16 #4 sat dec 1 23:47:07 cet 2001 i686 unknown '
config_args='-des -Dprefix=/sw/packages/bleadperl -Doptimize=-g -Dusedevel
-Dcc=gcc'
hint=recommended, useposix=true, d_sigaction=define
usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef
useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
use64bitint=undef use64bitall=undef uselongdouble=undef
usemymalloc=n, bincompat5005=define
Compiler:
cc='gcc', ccflags ='-DDEBUGGING -fno-strict-aliasing -I/usr/local/include
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm',
optimize='-g',
cppflags='-DDEBUGGING -fno-strict-aliasing -I/usr/local/include
-I/usr/include/gdbm'
ccversion='', gccversion='2.95.2 19991024 (release)', gccosandvers=''
intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
alignbytes=4, prototype=define
Linker and Libraries:
ld='gcc', ldflags =' -L/usr/local/lib'
libpth=/usr/local/lib /lib /usr/lib
libs=-lnsl -lndbm -lgdbm -ldl -lm -lc -lcrypt -lutil
perllibs=-lnsl -ldl -lm -lc -lcrypt -lutil
libc=/lib/libc-2.2.2.so, so=so, useshrplib=false, libperl=libperl.a
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic'
cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib'
Characteristics of this binary (from libperl):
Compile-time options: DEBUGGING USE_LARGE_FILES
Locally applied patches:
DEVEL14574
Built under linux
Compiled at Feb 9 2002 02:01:34
@INC:
/sw/packages/bleadperl/lib/5.7.2/i686-linux
/sw/packages/bleadperl/lib/5.7.2
/sw/packages/bleadperl/lib/site_perl/5.7.2/i686-linux
/sw/packages/bleadperl/lib/site_perl/5.7.2
/sw/packages/bleadperl/lib/site_perl/5.7.1/i686-linux
/sw/packages/bleadperl/lib/site_perl/5.7.1
/sw/packages/bleadperl/lib/site_perl
.
--
Paul Johnson - [EMAIL PROTECTED]
http://www.pjcj.net
