> I wrote a program that converts 5 gigs of emails stored in mysql to
> phisical messages on disk (resulting in approximately 10 gigs). The
program
> consumes way to much memory although I've wrote it in a very clean way
(use
> strict, no globals, use of udef $var; to help free memory). I start it
> telneting to :80 and through time process takes from 15 megs to more than
> 100 megs and only about 20% of messages are converted so far. The program
> uses vpopmail.pm and should run approximately one day. Maybe it would be
> faster if I used Apache::FakeRequest or something similar eleminating use
of
> mod_perl (plain cgi) ? I'm using HTML::Mason.
It will definitely not be faster if you use CGI. It may take care of your
memory problems, but only by avoiding them. You should be able to fix the
problem in your code and use mod_perl for this.
> Why is the memory consumption so high ? According to my calculations it
> should take less than 100 kilobytes.
What are you basing that on?
It sounds like you have some data structure which you are adding to and
never clearing out. One thing to be careful of is the MySQL DBI driver.
The last time I used it, it fetched all returned rows into memory. With a
large result set, that could take up tons of RAM.
- Perrin
