On Thu, 2003-08-14 at 19:34, [EMAIL PROTECTED] wrote:
> > I'm trying to understand why scanning a 1000 line script
> > (albeit one using Tk and some other modules) takes 45 seconds
> > on an otherwise speedy Sun E4500.
>
> I wonder too. Can you do a dprofpp on it?
Oops, I tried Perl 5.6.1 on Solaris, ActiverPerl 5.6.1 on Windows XP
and Perl 5.8.0 on Linux and all three segfaulted when executed
a small script exercising Module::ScanDeps with -d:DProf :(
However, most of the 45 seconds are spent in user space and I guess
the lion's share are eaten by the regular expressions in scan_chunk.
BTW, on my Linux PC the script scans in 4 seconds,
so we probably see the effects of the speed difference between
a 400 MHz UltraSparc CPU (Sun E4500) and the 1.4 GHz Athlon in my PC.
Using strace I noticed that lotsa files are opened several times.
Responsible is the following line in scan_deps:
my @files = sort grep -T $_->{file}, values %$rv;
The following patch (against Module::ScanDeps 0.28)
will scan exactly the same files but will open any file only once,
but for no measurable gain in speed.
@@ -260,6 +260,7 @@
local *FH;
open FH, $file or die "Cannot open $file: $!";
+ close FH, next unless -T FH;
$SeenTk = 0;
# Line-by-line scanning {{{
@@ -295,7 +296,7 @@
# Top-level recursion handling {{{
while ($recurse) {
my $count = keys %$rv;
- my @files = sort grep -T $_->{file}, values %$rv;
+ my @files = sort values %$rv;
scan_deps(
files => [ map $_->{file}, @files ],
keys => [ map $_->{key}, @files ],
I tried several other ideas, e.g. caching "negative" calls to
add_deps, but no speed up yet.
But I did come across two bugs, the latter embarrasingly in code
contributed by me. Here's the patch
--- /opt/gnu/lib/perl5/5.6.1/Module/ScanDeps.pm Sun Aug 17 20:56:15 2003
+++ Module/ScanDeps.pm Mon Aug 25 16:18:16 2003
@@ -183,7 +183,7 @@
URI/URL.pm URI/http.pm LWP/Protocol/http.pm
)],
'Locale/Maketext/Lexicon.pm' => 'sub',
- 'Math/BigInt.pm' => [qw( Math::BigInt::Calc )],
+ 'Math/BigInt.pm' => [qw( Math/BigInt/Calc )],
'Module/Build.pm' => 'sub',
'MIME/Decoder.pm' => 'sub',
'Net/DNS/RR.pm' => 'sub',
@@ -346,9 +347,8 @@
# Module name extraction heuristics {{{
my $module = eval {
$_ = $chunk;
-
- return [ 'base.pm', grep { !/^q[qw]?$/ } split(/[^\w:]+/, $1) ]
- if /^\s* use \s+ base \s+ (.*)/x;
+ return [ 'base.pm', map { s/::/\//g; "$_.pm" } $1 =~ /([\w:]+)/g
]
+ if /(?:^|\s)use\s+base\s+(.*)/;
return $1 if
/(?:^|\s)(?:use|no|require)\s+([\w:\.\-\\\/\"\']+)/;
return $1 if
/(?:^|\s)(?:use|no|require)\s+\(\s*([\w:\.\-\\\/\"\']+)\s*\)/;
Cheers, Roderich
