On Fri, Oct 13, 2006 at 01:04:01AM +1300, Robin Sheat wrote:
> On Thursday 12 October 2006 23:47, Paul Johnson wrote:
> > Not really a beginners topic, but ...
> Hmm, is there a more appropriate list? I just posted here because I've been
> subscribed since I started learning perl, quite some time ago :)
I wouldn't say we were off topic here, but you might find more people with
insights somewhere like comp.lang.perl.moderated or perlmonks.org.
> > In general, perl ops are expensive. Assuming you have enough memory and
> > are CPU bound you'll want to be reducing the number of ops. This whole
> Yeah, this stuff is mostly CPU bound, I've tried to optimise the disk as much
> as I know how. For reference, this is how the disk reads are done, coming
> from pre-built binary files and an index stored in a hash:
> sub get_user_details {
> my $userid = shift;
> my ($pos, $count) = @{$byuserindex{$userid}};
> my $buffer;
> seek $byuserfh, $pos*$recsize, 0;
> read $byuserfh, $buffer, $count*$recsize;
> my @res = unpack("($template)*", $buffer);
> return [EMAIL PROTECTED];
> }
Yeah, that looks pretty reasonable.
> $template='LC' (which fits my data type, being a large number and a small
> number). The files these are coming from are 500Mb each (there's two of
> them), and my machine has 2Gb RAM, so if something else in the script needs
> to build large in-memory indexes, then there's not really enough left over
> for that if I load these into RAM. If I had NZ$400 handy to buy 2Gb more, I'd
> probably have less of a problem :)
>
> > I'd probably try replacing the middle of your loop with something like:
> > my ($sum, $count, $i) = (0, @$details / 2, 1);
> > $sum += @$_ for grep $i ^= 1, @$details;
Thanks to John for pointing out the error in that code.
> As I mentioned in my other post, this gives a not-insignificant speedup. I
> can
> see I'm going to have to put some effort in to devising crafty uses of map
> and grep to do this stuff well.
I'm curious as to how much of the speedup comes from just pulling $count out
of the loop. As I said, these things can be a bit hard to predict sometimes.
> > You might even find newer versions of the talk floating around somewhere.
> Hmm, when i started looking on the net, that was one of the first things that
> came up. Also this:
> http://www-128.ibm.com/developerworks/library/l-optperl.html
> They're helpful, but unfortunately not largely applicable. Although, I may
> need to read them closer.
I'm not convinced by this one. Some of the advice is good, but some seems
just wrong.
For example:
- I'm not convinced that anyone is going to be able to measure any increase
in speed by using "string" instead of 'string'.
- I don't think that using map in void context instead of a for loop will
lead to any speeed increase. In earlier perls it will lead to a slowdown.
- The loops working on keys(%hash) should probably all be working on
values(%hash) which would allow the loop body to be simplified.
- Trying to use perlcc is generally a bad idea, and the code has been pulled
from the 5.10 release. He wouldn't have known about perlcc going away at
the time this was written, but using perlcc has been actively discouraged
for a long time since it never worked well, and didn't even seem to lead to
speed improvements when it did work.
- I don't agree with turning off warnings and strict in production. If
something goes wrong in production I want to know about it.
But as I say, there is also some good advice there too.
--
Paul Johnson - [EMAIL PROTECTED]
http://www.pjcj.net
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
<http://learn.perl.org/> <http://learn.perl.org/first-response>
