I'm starting a new thread for this again because this is a different
approach altogether.
I added a file-caching mechanism to PAR that prevents unnecessary
seeking through the slow Archive::Zip::Archive implementation. It
doesn't break encapsulation unlike the previously suggested attempt.
The results of my test run are as follows. Keep in mind that this code
is loading several rather heavy modules. That time is *included in all
measurements*. Also, the memory use might be very off due to Devel::DProf.
- old PAR, first run (no disk cache):
Memory as per "top": 71980 37m 2428
Execution time: 1.366735s
- old PAR, second run (hence with disk cache):
71652 36m 2424
1.127241s
- cached PAR, first run (no disk cache):
71968 37m 2428
0.834193s
- cached PAR, second run (hence with disk cache):
71712 37m 2424
0.624825s
So the caching doesn't seem to impose a significant memory overhead but
gives us pretty good performance improvements. Note that the performance
improvements will get bigger with larger .par's. Those aren't uncommon!
And no, I won't show that last assertion.
I attached an explanation on how I did the benchmark to this mail. The
hacked PAR.pm can be *temporarily* found at
http://steffen-mueller.net/tmp/PAR.pm.caching
I expected the caching to be a pretty bad hit wrt. memory consumption,
but given that the footprint doesn't change appreciably, I would
consider making the caching the default. Right now, it has to be enabled
using "use PAR {.... caching => 1 ....}". What do you think?
Cheers,
Steffen
-----
This is what I did to produce the benchmarks:
Put this into package.pl:
------------
#!perl
use Math::Symbolic;
use Algorithm::CurveFit;
use Math::Symbolic::Custom::Transformation;
use Math::FFTW;
use PAR::Repository::Client;
use PAR::Repository;
use PPI;
use LWP::Simple;
use CGI;
use Mail::Box;
------------
Then run:
pp -o package.par -p package.pl
Then put this into bench.pl:
-----------
#!perl
# caching => 1 has no effect with old PAR's.
use PAR { file => 'package.par', caching => 1};
use Math::Symbolic;
use Algorithm::CurveFit;
use Math::Symbolic::Custom::Transformation;
use Math::FFTW;
use PAR::Repository::Client;
use PAR::Repository;
use PPI;
use LWP::Simple;
use Mail::Box;
use CGI;
-------------
Then run:
perl -d:DProf bench.pl
and:
dprofpp
