On Tue, Sep 12, 2000 at 06:56:36PM -0600, Nathan Torkington wrote:
> I'd rather see someone write a program that involves file reading
> and writing, regular expression manipulation, some string tweaking
> (length, concatenation, etc), and some object method calls. That
> covers the spectrum of Things That I'm Worried About.
This is a good place to start. "Things That I'm Worried About". We
all have our shit lists about what parts of Perl are slow. Air them,
in an organized manner, and from that we should be able to develop
benchmarks. As always, there's no reason to wait for Perl 6, we can
apply these to Perl 5 just as well.
The things we bench need not be simple low level bits. Adding
numbers, method lookups, etc... but may also cover things like whole
modules. Even comparisons. "Net::FTP downloads slower than
/bin/ftp". Things of that nature.
Another useful bench would be to compare the various perl operations
to their C (and other language) equivalents. $a.$b vs strcat(a,b);
and such. This should give us a normalizing process for the
benchmarks. It should be expected that Perl operations will be slower
than the equivalent C by a certain order of magnitude. Any departure
from this expected inefficiency should indicate a point where work it
needed. It will also help to dig out points where its not Perl's
fault, but C's (or the system hardware's) lacking which is causing a
performance problem.
It would also be useful to bench the performance of certain equivalent
Perl operations. For example C<$str = 'foo';> should be no faster
than the equivalent here-doc which would be about the same as C<$str =
q{foo}> (which is currently the case, if you're wondering). Such
inequalities will give away deficiencies in the implemenation and
optimizer, and possibly unneccessary duplication of effort in the code.
The resulting set of benchmarks should be distributed with Perl and
run with a simple "make bench" (or similar) producing immediate
results as well as a single, comprehensive report file (also
containing perl -V info). People will be encouraged to run the
benching suite and mail it back to us for evaluation.
As long as I'm rambling, if we're going to embed tests in the code,
there's no reason we couldn't also embed benchmarks. "=for benching"
or something similar. Same reasons as why embedded tests are a good
idea.
--
Michael G Schwern http://www.pobox.com/~schwern/ [EMAIL PROTECTED]
Just Another Stupid Consultant Perl6 Kwalitee Ashuranse
<GuRuThuG> make a channel called Perl, and infest it with joking and
fun....it doesnt make alot of sense.
