Re: leaks with Apache::Request?
[resent to modperl list; earlier copy mistakenly cc'd to "p5p"] Does anyone know what's causing the Apache::Request object to leak here? See # XXX comment below: package Apache::test; sub Apache::Request::DESTROY{warn "DEAD: $_[0]\n"} sub Apache::DESTROY{warn "Dead: $_[0]\n"} use Devel::Peek; use Apache::Request; sub handler { my $r = shift; my $apr = Apache::Request->new($r); Dump($apr); # OK $r = $apr; # XXX: what's going on here??? Dump($apr); # fscked $r->send_http_header; $r->print('apr test'); return 200; } 1; SetHandler perl-script PerlHandler Apache::test --> server error log for request to /test <-- [Wed Jul 10 07:38:32 2002] [notice] Apache/1.3.27-dev (Unix) mod_perl/1.27_01-dev Perl/v5.8.0 configured [...] SV = RV(0x87aca6c) at 0x89902b8 REFCNT = 1 FLAGS = (PADBUSY,PADMY,ROK) RV = 0x8626da4 SV = PVMG(0x8703130) at 0x8626da4 REFCNT = 1 FLAGS = (OBJECT,RMG,IOK,pIOK) IV = 148456972 NV = 0 PV = 0 MAGIC = 0x8d7b2c8 MG_VIRTUAL = 0 MG_TYPE = PERL_MAGIC_ext(~) MG_FLAGS = 0x02 REFCOUNTED MG_OBJ = 0x89975dc SV = RV(0x87aca64) at 0x89975dc REFCNT = 2 FLAGS = (PADBUSY,PADMY,ROK) RV = 0x8626c78 SV = PVMG(0x8703110) at 0x8626c78 REFCNT = 2 FLAGS = (OBJECT,IOK,pIOK) IV = 148451868 NV = 0 PV = 0 STASH = 0x8298510 "Apache" MG_LEN = -1 MG_PTR = 0x8d9321c - please notify IZ STASH = 0x835f8ec "Apache::Request" SV = RV(0x87aca6c) at 0x89902b8 REFCNT = 1 FLAGS = (PADBUSY,PADMY,ROK) RV = 0x8626da4 SV = PVMG(0x8703130) at 0x8626da4 REFCNT = 2 FLAGS = (OBJECT,RMG,IOK,pIOK) IV = 148456972 NV = 0 PV = 0 MAGIC = 0x8d7b2c8 MG_VIRTUAL = 0 MG_TYPE = PERL_MAGIC_ext(~) MG_FLAGS = 0x02 REFCOUNTED MG_OBJ = 0x89975dc SV = RV(0x87aca64) at 0x89975dc REFCNT = 2 FLAGS = (PADBUSY,PADMY,ROK) RV = 0x8626da4 SV = PVMG(0x8703130) at 0x8626da4 REFCNT = 2 FLAGS = (OBJECT,RMG,IOK,pIOK) IV = 148456972 NV = 0 PV = 0 MAGIC = 0x8d7b2c8 MG_VIRTUAL = 0 MG_TYPE = PERL_MAGIC_ext(~) MG_FLAGS = 0x02 REFCOUNTED MG_OBJ = 0x89975dc SV = RV(0x87aca64) at 0x89975dc REFCNT = 2 FLAGS = (PADBUSY,PADMY,ROK) RV = 0x8626da4 MG_LEN = -1 MG_PTR = 0x8d9321c - please notify IZ STASH = 0x835f8ec "Apache::Request" MG_LEN = -1 MG_PTR = 0x8d9321c - please notify IZ STASH = 0x835f8ec "Apache::Request" Dead: Apache=SCALAR(0x8626c78)
Re: leaks with Apache::Request?
Does anyone know what's causing the Apache::Request object to leak here? See # XXX comment below: package Apache::test; sub Apache::Request::DESTROY{warn "DEAD: $_[0]\n"} sub Apache::DESTROY{warn "Dead: $_[0]\n"} use Devel::Peek; use Apache::Request; sub handler { my $r = shift; my $apr = Apache::Request->new($r); Dump($apr); # OK $r = $apr; # XXX: what's going on here??? Dump($apr); # fscked $r->send_http_header; $r->print('apr test'); return 200; } 1; SetHandler perl-script PerlHandler Apache::test --> server error log for request to /test <-- [Wed Jul 10 07:38:32 2002] [notice] Apache/1.3.27-dev (Unix) mod_perl/1.27_01-dev Perl/v5.8.0 configured [...] SV = RV(0x87aca6c) at 0x89902b8 REFCNT = 1 FLAGS = (PADBUSY,PADMY,ROK) RV = 0x8626da4 SV = PVMG(0x8703130) at 0x8626da4 REFCNT = 1 FLAGS = (OBJECT,RMG,IOK,pIOK) IV = 148456972 NV = 0 PV = 0 MAGIC = 0x8d7b2c8 MG_VIRTUAL = 0 MG_TYPE = PERL_MAGIC_ext(~) MG_FLAGS = 0x02 REFCOUNTED MG_OBJ = 0x89975dc SV = RV(0x87aca64) at 0x89975dc REFCNT = 2 FLAGS = (PADBUSY,PADMY,ROK) RV = 0x8626c78 SV = PVMG(0x8703110) at 0x8626c78 REFCNT = 2 FLAGS = (OBJECT,IOK,pIOK) IV = 148451868 NV = 0 PV = 0 STASH = 0x8298510 "Apache" MG_LEN = -1 MG_PTR = 0x8d9321c - please notify IZ STASH = 0x835f8ec "Apache::Request" SV = RV(0x87aca6c) at 0x89902b8 REFCNT = 1 FLAGS = (PADBUSY,PADMY,ROK) RV = 0x8626da4 SV = PVMG(0x8703130) at 0x8626da4 REFCNT = 2 FLAGS = (OBJECT,RMG,IOK,pIOK) IV = 148456972 NV = 0 PV = 0 MAGIC = 0x8d7b2c8 MG_VIRTUAL = 0 MG_TYPE = PERL_MAGIC_ext(~) MG_FLAGS = 0x02 REFCOUNTED MG_OBJ = 0x89975dc SV = RV(0x87aca64) at 0x89975dc REFCNT = 2 FLAGS = (PADBUSY,PADMY,ROK) RV = 0x8626da4 SV = PVMG(0x8703130) at 0x8626da4 REFCNT = 2 FLAGS = (OBJECT,RMG,IOK,pIOK) IV = 148456972 NV = 0 PV = 0 MAGIC = 0x8d7b2c8 MG_VIRTUAL = 0 MG_TYPE = PERL_MAGIC_ext(~) MG_FLAGS = 0x02 REFCOUNTED MG_OBJ = 0x89975dc SV = RV(0x87aca64) at 0x89975dc REFCNT = 2 FLAGS = (PADBUSY,PADMY,ROK) RV = 0x8626da4 MG_LEN = -1 MG_PTR = 0x8d9321c - please notify IZ STASH = 0x835f8ec "Apache::Request" MG_LEN = -1 MG_PTR = 0x8d9321c - please notify IZ STASH = 0x835f8ec "Apache::Request" Dead: Apache=SCALAR(0x8626c78)
Re: leaks with Apache::Request?
Joe Schaefer wrote: >>Somehow the assignment operator MUST be involved in the leak here. >>(You only get a leak when the *same* reference (*SV) is on both sides >>of the assignment). > > > Could someone with modperl 1.2x built using a perl 5.8 release candidate > please test this out: I got the same behavior (Apache/1.3.26 (Unix) mod_perl/1.27 perl@17440) with a modified version of your code (it doesn't work as is.) HTH.
Re: leaks with Apache::Request?
Joe Schaefer <[EMAIL PROTECTED]> writes: [...] > Somehow the assignment operator MUST be involved in the leak here. > (You only get a leak when the *same* reference (*SV) is on both sides > of the assignment). Could someone with modperl 1.2x built using a perl 5.8 release candidate please test this out: sub Apache::Request::DESTROY{warn "DEAD: $_[0]\n"} sub Apache::DESTROY{warn "Dead: $_[0]\n"} use Devel::Peek; use Apache:Request; package Apache::test; sub handler { my $r = shift; my $apr = Apache::Request->new($r); Dump($apr); # OK $r = $apr; # XXX: what's going on here??? Dump($apr); # fscked $r->send_header; $r->print('apr test'); return 200; } 1; SetHandler perl-script PerlHandler Apache::test My error log for a request to /test is below. The above assignment on the line marked "XXX" is modifying $apr, which it shouldn't. I don't have a 5.8-RC handy to test this on, but if somebody else sees the same problem on 5.8, I'll file a bug report to p5p. Thanks alot. -- SV = RV(0x814b154) at 0x82309b8 REFCNT = 1 FLAGS = (ROK) RV = 0x82472f4 SV = PVMG(0x8244f58) at 0x82472f4 REFCNT = 1 FLAGS = (OBJECT,RMG,IOK,pIOK) IV = 136639644 NV = 0 PV = 0 MAGIC = 0x8244f80 MG_VIRTUAL = 0 MG_TYPE = '~' MG_FLAGS = 0x02 REFCOUNTED MG_OBJ = 0x822dec4 SV = RV(0x814b14c) at 0x822dec4 REFCNT = 2 FLAGS = (PADBUSY,PADMY,ROK) RV = 0x82472dc SV = PVMG(0x8244f30) at 0x82472dc REFCNT = 2 FLAGS = (OBJECT,IOK,pIOK) IV = 136628428 NV = 0 PV = 0 STASH = 0x81420e4 "Apache" MG_LEN = -1 MG_PTR = 0x824c8cc - please notify IZ STASH = 0x8224a18 "Apache::Request" SV = RV(0x814b154) at 0x82309b8 REFCNT = 1 FLAGS = (ROK) RV = 0x82472f4 SV = PVMG(0x8244f58) at 0x82472f4 REFCNT = 2 FLAGS = (OBJECT,RMG,IOK,pIOK) IV = 136639644 NV = 0 PV = 0 MAGIC = 0x8244f80 MG_VIRTUAL = 0 MG_TYPE = '~' MG_FLAGS = 0x02 REFCOUNTED MG_OBJ = 0x822dec4 SV = RV(0x814b14c) at 0x822dec4 REFCNT = 2 FLAGS = (PADBUSY,PADMY,ROK) RV = 0x82472f4 SV = PVMG(0x8244f58) at 0x82472f4 REFCNT = 2 FLAGS = (OBJECT,RMG,IOK,pIOK) IV = 136639644 NV = 0 PV = 0 MAGIC = 0x8244f80 MG_VIRTUAL = 0 MG_TYPE = '~' MG_FLAGS = 0x02 REFCOUNTED MG_OBJ = 0x822dec4 SV = RV(0x814b14c) at 0x822dec4 REFCNT = 2 FLAGS = (PADBUSY,PADMY,ROK) RV = 0x82472f4 MG_LEN = -1 MG_PTR = 0x824c8cc - please notify IZ STASH = 0x8224a18 "Apache::Request" MG_LEN = -1 MG_PTR = 0x824c8cc - please notify IZ STASH = 0x8224a18 "Apache::Request" Dead: Apache=SCALAR(0x82472dc) -- -- Joe Schaefer
Re: leaks with Apache::Request?
darren chamberlain <[EMAIL PROTECTED]> writes: > * Joe Schaefer <[EMAIL PROTECTED]> [2002-07-09 12:47]: > > Dave Rolsky <[EMAIL PROTECTED]> writes: > > > On 8 Jul 2002, Joe Schaefer wrote: > > > If I do this: > > > > > > my $x = shift; > > > $x = make_something_from($x); > > > > > > then it seems like the original $x should go out of scope when it is > > > assigned to, so its refcount should stay at 1. > >^^ > > > > Right, it should stay at 1. But all bets are off when > > $x is has magic and make_something_from() is an XS sub :-). > > But the leak is not with $x, it's with what $_[0] is; $x is just a > reference to that, and the reassignment in the second line should > reassign $x, and decrement the ref count for what $x is pointing to at > that point. So, it all depends on what make_something_from() does with > the $x's referent. I don't think it's as simple as that- THIS LEAKS: my $r = shift; $r = Apache::Request->new($r); my $r = shift; $r = Apache::Request->new($_) for $r; DOES NOT LEAK: my $apr = Apache::Request->new(shift); my $r = shift; my $apr = $r; # $r and $apr have the same referent now right? $apr = Apache::Request->new($r); my $r = shift; my $apr = Apache::Request->new($r); my $r = shift; $r = Apache::Request->new($_) for map $_, $r; Somehow the assignment operator MUST be involved in the leak here. (You only get a leak when the *same* reference (*SV) is on both sides of the assignment). -- Joe Schaefer
Re: leaks with Apache::Request?
* Joe Schaefer <[EMAIL PROTECTED]> [2002-07-09 12:47]: > Dave Rolsky <[EMAIL PROTECTED]> writes: > > On 8 Jul 2002, Joe Schaefer wrote: > > If I do this: > > > > my $x = shift; > > $x = make_something_from($x); > > > > then it seems like the original $x should go out of scope when it is > > assigned to, so its refcount should stay at 1. >^^ > > Right, it should stay at 1. But all bets are off when > $x is has magic and make_something_from() is an XS sub :-). But the leak is not with $x, it's with what $_[0] is; $x is just a reference to that, and the reassignment in the second line should reassign $x, and decrement the ref count for what $x is pointing to at that point. So, it all depends on what make_something_from() does with the $x's referent. (darren) -- Great minds discuss ideas. Average minds discuss events. Small minds discuss people. -- Admiral Hyman G. Rickover
Re: leaks with Apache::Request?
Dave Rolsky <[EMAIL PROTECTED]> writes: > On 8 Jul 2002, Joe Schaefer wrote: [...] > > my $r = shift; > > my $apr = Apache::Request->new($r); > > > > That's not going to leak, either. At least I hope not :-) > > I ended up using something like this and the leak went away. > > It seems to me that this might actually be a Perl bug. I doubt it's solely perl's fault. The problem is that you have to be an internals wizard to write "safe" XS; otherwise you just keep your fingers crossed. :-) Hopefully some generous soul will post a patch to Request.xs that prevents this problem from ever arising. > If I do this: > > my $x = shift; > $x = make_something_from($x); > > then it seems like the original $x should go out of scope when it is > assigned to, so its refcount should stay at 1. ^^ Right, it should stay at 1. But all bets are off when $x is has magic and make_something_from() is an XS sub :-). -- Joe Schaefer
Re: leaks with Apache::Request?
On 8 Jul 2002, Joe Schaefer wrote: > Write that like this, and I think your leak will > disappear: > > my $r = Apache::Request->new( shift ); > > AFAICT, Apache::Request::new is NOT leaking here, since the > REFCNT of its returned object IS 1. There might be some > magic-related bug in perl that causes the assignment to bump > $r's refcount to 2. This MIGHT be circumventable with some better > code in Request.xs, but I really don't know how to fix it. > > Until some perl guru enlightens us, as a personal rule I > try hard to avoid expressions like > > $foo = make_something_out_of($foo); > > I realize that this isn't always possible, but it often/usually > is. Such advice would serve you well in this case; you could > even get away with this > > my $r = shift; > my $apr = Apache::Request->new($r); > > That's not going to leak, either. At least I hope not :-) I ended up using something like this and the leak went away. It seems to me that this might actually be a Perl bug. If I do this: my $x = shift; $x = make_something_from($x); then it seems like the original $x should go out of scope when it is assigned to, so its refcount should stay at 1. -dave /*== www.urth.org we await the New Sun ==*/
Re: leaks with Apache::Request?
Dave Rolsky <[EMAIL PROTECTED]> writes: [...] > Here's some code that I think demonstrates the leak: > package My::APRTest; > > use strict; > > use Apache::Request; > sub Apache::Request::DESTROY{warn "DEAD: $_[0]\n"} > sub handler > { > my $r = shift; > $r = Apache::Request->new($r); ^ Write that like this, and I think your leak will disappear: my $r = Apache::Request->new( shift ); AFAICT, Apache::Request::new is NOT leaking here, since the REFCNT of its returned object IS 1. There might be some magic-related bug in perl that causes the assignment to bump $r's refcount to 2. This MIGHT be circumventable with some better code in Request.xs, but I really don't know how to fix it. Until some perl guru enlightens us, as a personal rule I try hard to avoid expressions like $foo = make_something_out_of($foo); I realize that this isn't always possible, but it often/usually is. Such advice would serve you well in this case; you could even get away with this my $r = shift; my $apr = Apache::Request->new($r); That's not going to leak, either. At least I hope not :-) HTH -- Joe Schaefer
Re: leaks with Apache::Request?
On Mon, 8 Jul 2002, Richard Clarke wrote: > "During the child exit phase, mod_perl invokes the Perl API function > perl_destruct( ) to run the contents of END blocks and to invoke the > DESTROY method for any global objects that have not gone out of scope > already." Notice where it says "... for any global objects that have not gone out of scope already". In the code I posted, $r goes out of scope at the end of the handler subroutine. > So I think i'm right in saying that, "When I run this, the DESTROY > method is not called until the server shuts down." is perfectly normal > behaviour. I don't know how you are actually testing your "memory Well, creating an Apache::DESTROY method works and shows the Apache object going out of scope on a per-request basis. > usage", but I might suggest that if you are sending larger amounts of > data than previously, just once per 5000 requests then this memory is > gonna be consumed by apache forever (at least until httpd is killed). I tested with ab sending the same request thousands of times in a row. -dave /*== www.urth.org we await the New Sun ==*/
Re: leaks with Apache::Request?
Dave, Perhaps this is why from the eagle book, "During the child exit phase, mod_perl invokes the Perl API function perl_destruct( ) to run the contents of END blocks and to invoke the DESTROY method for any global objects that have not gone out of scope already." So I think i'm right in saying that, "When I run this, the DESTROY method is not called until the server shuts down." is perfectly normal behaviour. I don't know how you are actually testing your "memory usage", but I might suggest that if you are sending larger amounts of data than previously, just once per 5000 requests then this memory is gonna be consumed by apache forever (at least until httpd is killed). Richard Dave Rolsky wrote: >It looks like there may be a memory leak with Apache::Request. I'm using >version 1.0 with Perl 5.6.1, mod_perl 1.26, and Apache 1.3.26. mod_perl >is statically compiled into Apache. > >Here's some code that I think demonstrates the leak: > > package My::APRTest; > > use strict; > > use Apache::Request; > sub Apache::Request::DESTROY{warn "DEAD: $_[0]\n"} > sub handler > { > my $r = shift; > $r = Apache::Request->new($r); > > $r->send_http_header; > $r->print('apr test'); > > return 200; > } > > 1; > >When I run this, the DESTROY method is not called until the server shuts >down. > >Watching memory using with top (against a server running as httpd -X) I >can see that memory usage is growing a little less the 500K every 5000 >requests. > >This isn't catastrophic but fixing it would be a good thing. > > >-dave > >/*== >www.urth.org >we await the New Sun >==*/ > > > > >
leaks with Apache::Request?
It looks like there may be a memory leak with Apache::Request. I'm using version 1.0 with Perl 5.6.1, mod_perl 1.26, and Apache 1.3.26. mod_perl is statically compiled into Apache. Here's some code that I think demonstrates the leak: package My::APRTest; use strict; use Apache::Request; sub Apache::Request::DESTROY{warn "DEAD: $_[0]\n"} sub handler { my $r = shift; $r = Apache::Request->new($r); $r->send_http_header; $r->print('apr test'); return 200; } 1; When I run this, the DESTROY method is not called until the server shuts down. Watching memory using with top (against a server running as httpd -X) I can see that memory usage is growing a little less the 500K every 5000 requests. This isn't catastrophic but fixing it would be a good thing. -dave /*== www.urth.org we await the New Sun ==*/