Speaking of top, another thing I noticed last night after the script had been running for a few hours was that the script was taking up a huge amount of memory, like more than 500M of RSIZE, and this size seemed to be constantly increasing... this surprised me, since the script is not doing anything that, in my naive view, would require progressively larger memory chunks...
use HTML::Parse; use HTML::FormatText; use Sys::AlarmCall;
There are two possible problem spots: 1) HTML::Parse uses a tree containing circularities internally. If it doesn't wrap that with a destructor layer, it will leak (I'm too lazy to check myself). 2) Sys::AlarmCall uses string eval internally, which is a rather bad idea from a performance point of view (and possibly security, and it also does not rethrow exceptions, enough that I won't use it), and could maybe show up perl problems. (I believe that the memory leak bug that was, in context of mod_perl (/AxKit) some time ago, attested to Error.pm, was really a perl core bug solved recently (didn't have the time to check or discuss that yet).)
Christian.
