Re: httpd keeps growing
Andreas Schiffler wrote: I have a problem that I can't explain and don't know exactly how to debug: the httpd process keeps gobbling up memory over time. In reply to my own question, I found the cuplrit after some poking around in frequent restarts of the webserver using: apachectl restart After each code change, the new scripts were copied onto the server and apache restarted. Since we are in active development, this occured several times a day and each time apache was restarted this way, its memory consumption went up. Doing it the "hard" way with the /etc/rc.d/init.d script httpd restart solves the problem. Kindof a weired interaction between mod_perl and apache - can someone elaborate on "is this a bug or is it a feature"? Regards Andreas -- | Andreas Schiffler[EMAIL PROTECTED] | | Senior Systems Engineer-Deskplayer Inc., Buffalo | |
Re: httpd keeps growing
On Sat, 7 Oct 2000, Andreas Schiffler wrote: Andreas Schiffler wrote: I have a problem that I can't explain and don't know exactly how to debug: the httpd process keeps gobbling up memory over time. In reply to my own question, I found the cuplrit after some poking around in frequent restarts of the webserver using: apachectl restart After each code change, the new scripts were copied onto the server and apache restarted. Since we are in active development, this occured several times a day and each time apache was restarted this way, its memory consumption went up. Doing it the "hard" way with the /etc/rc.d/init.d script httpd restart solves the problem. Kindof a weired interaction between mod_perl and apache - can someone elaborate on "is this a bug or is it a feature"? Do you have PerlFreshRestart On ? If so, don't. Install Apache::Reload and try that instead. -- Matt/ ** Director and CTO ** ** AxKit.com Ltd ** ** XML Application Serving ** ** http://axkit.org ** ** XSLT, XPathScript, XSP ** ** Personal Web Site: http://sergeant.org/ **
Re: httpd keeps growing
I use (/etc/rc.d/init.d/http stop; sleep 2; /etc/rc.d/init.d/http start) Yeah, it causes a delay, but makes sure the dev box is in a clean state before I mess it up again with whatever change I just made grin --M. On Sat, 07 Oct 2000 12:33:10 -0700, Perrin Harkins wrote: Andreas Schiffler wrote: In reply to my own question, I found the cuplrit after some poking around in frequent restarts of the webserver using: apachectl restart After each code change, the new scripts were copied onto the server and apache restarted. Since we are in active development, this occured several times a day and each time apache was restarted this way, its memory consumption went up. Doing it the "hard" way with the /etc/rc.d/init.d script httpd restart solves the problem. Kindof a weired interaction between mod_perl and apache - can someone elaborate on "is this a bug or is it a feature"? It's well known that SIGHUP will cause the server to grow, although I'm not sure it's in the guide. I use apachectl stop apachectl start to restart my server. Anything else (including things like Apache::StatINC) will reduce your shared memory and increase the real size of the child processes. - Perrin
Re: httpd keeps growing
Matt Sergeant wrote: Do you have PerlFreshRestart On ? If so, don't. Install Apache::Reload and try that instead. Yeah, its ON. It seemed like a good thing to do, as per: http://perl.apache.org/guide/config.html#PerlFreshRestart and doesn't cause segfaults as per: http://perl.apache.org/guide/troubleshooting.html#Evil_things_might_happen_when_us I'll turn it OFF, do the "start" and "stop" procedure to reinitialize apache and have a look at Apache::Reload. Thanks for the tip! I still don't understand why a SIGHUP from apachectl would grab so much memory each time for the httpd process. Ideas anyone? Ciao Andreas -- | Andreas Schiffler[EMAIL PROTECTED] | | Senior Systems Engineer-Deskplayer Inc., Buffalo |
Re: httpd keeps growing
On Fri, 6 Oct 2000, Andreas Schiffler wrote:
Any ideas where this memory consumption might come from? or how to find
the culprit through some mod_perl code?
Go buy some coffee, get a copy of your local pizza shop's menu next to
your desk, and bring in a sleeping bag. That should help you get through
it.
The problem is that of all the methods people told me how to use, it
became clear that none work. The problem is that memory allocation doesn't
happen where the leak is. It happens in pools. I don't know if this is the
way it works just on Linux, but it sure made the leaks hard to find.
The way I finally found the leaks in AxKit was two-fold:
The first way was to simply create an entirely new module, and copy code
from the old one into the new one bit by bit until the new module starts
leaking too. Use ab (apachebench - in your Apache bin directory) to run
lots of hits on the URL, or just plain-old lwp-request. When you add the
bit of code that leaks, you've found a leak. (yes, this sounds like a
nightmare, and it is, but it was the only thing that worked).
The second thing was to make really good use of exception handlers. The
way I now structure my code is with Error.pm:
use Apache::Constants;
use Error qw/:try/;
sub handler {
my $r = Apache::Request-new(shift);
return try {
... code here.
return OK;
}
catch MyError::Declined with {
return DECLINED;
}
catch MyError::OK with {
return OK;
}
catch Error with {
my $E = shift;
die "handler died: $E\n";
}
}
(this is only a tiny fragment of the code, obviously)
Now what you can do is add a line:
throw MyError::Declined ( -text = "Find that leak!" );
somewhere in your code. Now start moving that line deeper and deeper into
your code, until the leak starts occuring. Then you know you've moved it
one line past the leak. You can then step into the code, if its a function
call, and add that line there.
Oh, and you need to define MyError::* somewhere. Thats all documented in
the Error.pm docs.
Sounds like a long winded process? Sure. But I'm saving you a lot of
heartache looking for this. And you'll be glad of structured exception
handling in the long run.
(Stas - maybe I should clean this up and add it to the guide?)
--
Matt/
** Director and CTO **
** AxKit.com Ltd ** ** XML Application Serving **
** http://axkit.org ** ** XSLT, XPathScript, XSP **
** Personal Web Site: http://sergeant.org/ **
Re: httpd keeps growing
At 09:06 AM 10/06/00 -0400, Andreas Schiffler wrote: Along with what Matt recommended (although I would have suggested beer, too), something I did yesterday to track down a leak was to go into every module I had modified lately and added warn's in both the new() and DESTROY methods that wrote a warning message saying that the object was created/destroyed. In my case I found I had a circular reference on an object and it was never being destroyed. Bill Moseley mailto:[EMAIL PROTECTED]
Re: httpd keeps growing
On Fri, 6 Oct 2000, Bill Moseley wrote:
At 09:06 AM 10/06/00 -0400, Andreas Schiffler wrote:
Along with what Matt recommended (although I would have suggested beer,
too), something I did yesterday to track down a leak was to go into every
module I had modified lately and added warn's in both the new() and DESTROY
methods that wrote a warning message saying that the object was
created/destroyed. In my case I found I had a circular reference on an
object and it was never being destroyed.
Forgot about that (my leak wasn't in circular refs, so I'd already done
that to no avail). Its often useful to have a counter:
package MyPackage;
use vars qw/$COUNTER/;
sub new {
my $class = shift;
warn "Now: ", ++$COUNTER, " refs of type $class\n";
bless {}, $class;
}
sub DESTROY {
my $obj = shift;
warn "Now: ", --$COUNTER, "refs of type ", ref($obj), "\n";
}
--
Matt/
** Director and CTO **
** AxKit.com Ltd ** ** XML Application Serving **
** http://axkit.org ** ** XSLT, XPathScript, XSP **
** Personal Web Site: http://sergeant.org/ **
