On Thu, 24 Nov 2005, Pierre Smolarek wrote:
> Lorenzo Caggioni wrote:
> > The program I written takes 25 sec for 10.000 line... too much....
>
> How quickly do you need to it if 25 seconds is too long?
If 10,000 lines take 25 seconds, you're doing 400 lines per second.
At that rate, 15,000,000 lines will take 37,500 seconds, or 10h25m.
While asking for a firmer definition for "faster" is a fair question,
it's fair to assume that he wants to do better than 10.4 hours :-)
That said, the canned answer applies here. If the problem is --
1 Read Line from an input file
2 Validate the raw (for example: is second char == 2?)
3 Split the line
4 Write the validated and splitted raw in an output file with a
different order (for example: last 2 digits I have to write as
first 2 digits)
-- then, in order to give *any* constructive advice, we need:
* to see the code in question
* to know if the code has been benchmarked
If we can't see the code, we can't possibly offer useful suggestions.
If we don't have benchmark info to know what part of the code is taking
so long, we can't even speculate as to where to start optimizing things.
One of the suggestions in Damian Conway's _Perl Best Practices_ is a
simple piece of advice: "Don't Optimize Code -- Benchmark It". For
details, look over this excerpt from the book:
http://www.perl.com/lpt/a/2005/07/14/bestpractices.html
It's sound advice. The book's next suggestion -- which I can't seem to
find a reference to online, so you're just going to have to find a copy
of the book itself -- is "Don't optimize data structures -- measure
them." This is also sound advice. If you use a module like Devel::Size
to determine how space is being allocated, you can get a better sense of
where you might be choking on data and, in turn, have a sense of where
you need to fix things.
Once you've used such tools to map out how your program is consuming
time and space, you can start making decisions about how to reduce that
consumption, by speeding up critical sections, reducing memory use, or
just throwing more RAM and CPU at the problem if you're starved there
and software optimizations seem like they might not be enough. But until
you've figured out where the time is being spent, or what system
resource is being exhausted, you can't properly address the problem.
Really, you could do a whole lot worse than by just getting a copy of
_Perl Best Practices_ and using its advice to rewrite your program from
scratch. Almost everyone could improve their code this way... :-)
--
Chris Devers
O3Õ×lQzSó®
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
<http://learn.perl.org/> <http://learn.perl.org/first-response>
