On Thu, Sep 2, 2010 at 04:39, Uri Guttman <u...@stemsystems.com> wrote:
snip
> if you want speed, that is not the best way to read in the file
> lines. File::Slurp (on cpan) can do that for you and is cleaner as well:
snip
If there was one thing I could change about this list, it would be
that to ban people from saying something is faster or slower without
providing the Big O Notation or a benchmark. File::Slurp is written
in Perl 5; there is no magical speed up from using it. In fact, there
is a slow down (see benchmark at the end).
And that is ignoring the fact that slurping the file is a terrible
idea. Further ignoring the fact that we are filling (and possibly
overfilling) memory uselessly, we are then looping over the lines.
This means we must loop once to read in the file in File::Slurp, and
once to read each line. This is wasted time.
The safest and fastest way to read a file line by line is the trusty
old while loop. If you are using a filesystem that provides a
readahead buffer, you can't beat it in terms of speed and memory
usage.
Note, there is a flaw in this benchmark: the filesystem plays an
important part in the read speed. In this case the fact that file is
already buffered in memory may play a part in while's speediness, but
everyone has that advantage. A similar test using around five hundred
seven megabyte files (on a box with two gigabytes of RAM) had similar
results
: while took 220 seconds, array took 233 seconds, and slurp took 293
seconds. The run order was array, while, slurp.
array: 65
while: 65
slurp: 65
Rate slurp array while
slurp 6280/s -- -35% -43%
array 9660/s 54% -- -12%
while 10960/s 75% 13% --
#!/usr/bin/perl
use strict;
use warnings;
use Benchmark;
use File::Slurp;
my %subs = (
slurp => sub {
my @lines = read_file "/etc/passwd";
return scalar @lines;
},
array => sub {
open my $fh, "<", "/etc/passwd" or die $!;
my @lines = <$fh>;
return scalar @lines;
},
while => sub {
open my $fh, "<", "/etc/passwd" or die $!;
my @lines;
while (my $line = <$fh>) {
push @lines, $line;
}
return scalar @lines;
},
);
for my $sub (keys %subs) {
print "$sub: ", $subs{$sub}(), "\n";
}
Benchmark::cmpthese -1, \%subs;
--
Chas. Owens
wonkden.net
The most important skill a programmer can have is the ability to read.
--
To unsubscribe, e-mail: beginners-unsubscr...@perl.org
For additional commands, e-mail: beginners-h...@perl.org
http://learn.perl.org/
