On Sat, Sep 02, 2006 at 11:58:33PM -0400, Theo Van Dinter wrote: > My weekly run is still going (19 hours now -- MUCH longer than it used to > take). I started watching the processes, and I'm fairly positive we have a > major memory leak somewhere. > > So something happened in August (probably between the 11th and the 19th, > since the 19th was the first time I saw the problem). I haven't dug > around to debug it yet. <sigh>
Did some more debugging. The problem is r432244:
r432244 | jm | 2006-08-17 10:09:27 -0400 (Thu, 17 Aug 2006) | 1 line
optimisation: don't delete all keys in PerMsgStatus, just the ones that need it
432243 runs fine, but 432244 causes OOMs. I poked around some more to
determine the other keys that need to be deleted, and was going to just
add them in, but I got to thinking: Since any plugin can add in data to
PMS, there can be any number of circular refs that we don't know about, so
deleting all of the keys is the only way we can be sure to clear them out.
FWIW: I found resolver and uribl_scanstate which needed to be removed, but
even then I still got OOMs, it just took longer. I assume there are other
net-test-related things that I'm somehow not seeing via Data::Dumper.
I'm going to commit in a change that changes PMS::finish() to do:
%{$self} = ();
which ends up deleting all the keys while not requiring the foreach() method
that we used previous, so I hope this solves JM's original issue if that being
inefficient.
I do think that we still have leaks elsewhere since the process memory
continues to grow, but it's much lower than before. I'll keep digging.
--
Randomly Generated Tagline:
It looked like something resembling white marble, which was
probably what it was: something resembling white marble.
-- Douglas Adams, "The Hitchhikers Guide to the Galaxy"
pgprLMxbGO8R4.pgp
Description: PGP signature
