[Bug 422138] Re: Slow memory leak, seen on two machines, appears to be dupe of 224945 even after -updates
** Changed in: apache2 (Ubuntu) Assignee: Chuck Short (zulcss) = (unassigned) ** Changed in: apache2 (Ubuntu) Importance: Low = Medium ** Changed in: apache2 (Ubuntu) Status: Incomplete = Triaged ** Changed in: apache2 (Ubuntu) Assignee: (unassigned) = Chuck Short (zulcss) -- Slow memory leak, seen on two machines, appears to be dupe of 224945 even after -updates https://bugs.launchpad.net/bugs/422138 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to apache2 in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 422138] Re: Slow memory leak, seen on two machines, appears to be dupe of 224945 even after -updates
Chuck, you're thinking this is that Debian bug that had an advisory last week right? -- Slow memory leak, seen on two machines, appears to be dupe of 224945 even after -updates https://bugs.launchpad.net/bugs/422138 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to apache2 in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 422138] Re: Slow memory leak, seen on two machines, appears to be dupe of 224945 even after -updates
probably -- Slow memory leak, seen on two machines, appears to be dupe of 224945 even after -updates https://bugs.launchpad.net/bugs/422138 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to apache2 in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 422138] Re: Slow memory leak, seen on two machines, appears to be dupe of 224945 even after -updates
@Joe, have you tried the configuratoin change as suggested? Thanks chuck -- Slow memory leak, seen on two machines, appears to be dupe of 224945 even after -updates https://bugs.launchpad.net/bugs/422138 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to apache2 in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 422138] Re: Slow memory leak, seen on two machines, appears to be dupe of 224945 even after -updates
Chuck Short wrote: @Joe, have you tried the configuratoin change as suggested? Thanks chuck Hey Chuck, yea to the extreme I think it's at maybe 1 or 100 for MaxRequestsPerChild, but it seems to resemble the old bug on the surface. Either way it's quieted down for me, thanks for your attention to the issue. -- Joe McDonagh Operations Engineer Silent Penguin Services AIM: YoosingYoonickz IRC: joe-mac on freenode www.colonfail.com -- Slow memory leak, seen on two machines, appears to be dupe of 224945 even after -updates https://bugs.launchpad.net/bugs/422138 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to apache2 in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 422138] Re: Slow memory leak, seen on two machines, appears to be dupe of 224945 even after -updates
Chuck Short wrote: @Joe: Its marked Low so our qa scripts pick it up. Ill take a look at this again after karmic has been released. Regards chuck Ah, ok, cool. Thanks Chuck. -- Slow memory leak, seen on two machines, appears to be dupe of 224945 even after -updates https://bugs.launchpad.net/bugs/422138 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to apache2 in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 422138] Re: Slow memory leak, seen on two machines, appears to be dupe of 224945 even after -updates
@Joe: Its marked Low so our qa scripts pick it up. Ill take a look at this again after karmic has been released. Regards chuck -- Slow memory leak, seen on two machines, appears to be dupe of 224945 even after -updates https://bugs.launchpad.net/bugs/422138 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to apache2 in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 422138] Re: Slow memory leak, seen on two machines, appears to be dupe of 224945 even after -updates
Chuck Short wrote: ** Changed in: apache2 (Ubuntu) Importance: Undecided = Low Chuck, was wondering, should this be marked Low from Undecided if it hasn't been confirmed nor denied by anyone? -- Joe McDonagh Operations Engineer www.colonfail.com -- Slow memory leak, seen on two machines, appears to be dupe of 224945 even after -updates https://bugs.launchpad.net/bugs/422138 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to apache2 in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 422138] Re: Slow memory leak, seen on two machines, appears to be dupe of 224945 even after -updates
** Changed in: apache2 (Ubuntu) Importance: Undecided = Low -- Slow memory leak, seen on two machines, appears to be dupe of 224945 even after -updates https://bugs.launchpad.net/bugs/422138 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to apache2 in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 422138] Re: Slow memory leak, seen on two machines, appears to be dupe of 224945 even after -updates
Stefan, thanks for re-iterating the MaxRequestsPerChild tidbit, I had forgotten that option was for lifetime, not concurrency. That'll probably help workaround momentarily. Also, I am not currently doing any tweaking wrt the SSLSessionCache, I recall messing with it when I first started seeing this problem, but had removed the configuration options since it didn't seem to help much. I would just like to see some other pair of eyes confirm that a default install with SSL still leaks. This is apparent because when I run the same tests against the HTTP only port things look normal. -- Slow memory leak, seen on two machines, appears to be dupe of 224945 even after -updates https://bugs.launchpad.net/bugs/422138 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to apache2 in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 422138] Re: Slow memory leak, seen on two machines, appears to be dupe of 224945 even after -updates
A workaround may be to add MaxRequestsPerChild 1000 (or an even lower value) to your configuration. That wouldn't really help since long-running children just hold onto memory no matter what it seems. A lower MaxRequestsPerChild should make the children terminate earlier. Does the package information I put in the beginning look right? I think so. Do you use SSLSessionCache dbm? If yes, try switching to shmcb. I seem to remeber a known memory leak in dbm. You can also try if disabling mod_deflate helps. -- Slow memory leak, seen on two machines, appears to be dupe of 224945 even after -updates https://bugs.launchpad.net/bugs/422138 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to apache2 in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 422138] Re: Slow memory leak, seen on two machines, appears to be dupe of 224945 even after -updates
Is the parameter -f TLS1 necessary to reproduce the problem? Is the URL / of your webserver a php page? If yes, why is the content length of the page 0, what does the php script do? If not php, what is it? A simple redirect? A workaround may be to add MaxRequestsPerChild 1000 (or an even lower value) to your configuration. -- Slow memory leak, seen on two machines, appears to be dupe of 224945 even after -updates https://bugs.launchpad.net/bugs/422138 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to apache2 in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 422138] Re: Slow memory leak, seen on two machines, appears to be dupe of 224945 even after -updates
This took down a box today, had to visit the DC for it... is there any more information-gathering I can do for you or myself here? -- Slow memory leak, seen on two machines, appears to be dupe of 224945 even after -updates https://bugs.launchpad.net/bugs/422138 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to apache2 in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 422138] Re: Slow memory leak, seen on two machines, appears to be dupe of 224945 even after -updates
Chuck, is there anything else you need from me to promote this from incomplete? -- Slow memory leak, seen on two machines, appears to be dupe of 224945 even after -updates https://bugs.launchpad.net/bugs/422138 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to apache2 in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 422138] Re: Slow memory leak, seen on two machines, appears to be dupe of 224945 even after -updates
Interesting, Can you specify the ab command that you used to reproduce this? Thanks chuck ** Changed in: apache2 (Ubuntu) Status: New = Incomplete ** Changed in: apache2 (Ubuntu) Assignee: (unassigned) = Chuck Short (zulcss) -- Slow memory leak, seen on two machines, appears to be dupe of 224945 even after -updates https://bugs.launchpad.net/bugs/422138 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to apache2 in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 422138] Re: Slow memory leak, seen on two machines, appears to be dupe of 224945 even after -updates
Hey Chuck, thanks for the response. The command is: ab -n 200 -c 100 -f TLS11 https://webserver/ At ten seconds: 30253 www-data 20 0 254m 75m 4432 S 31 0.5 0:03.50 apache2 30252 www-data 20 0 251m 72m 4464 S 30 0.4 0:03.34 apache2 30250 www-data 20 0 250m 70m 4420 S 29 0.4 0:03.26 apache2 30251 www-data 20 0 250m 70m 4448 S 29 0.4 0:03.28 apache2 30249 www-data 20 0 249m 70m S 28 0.4 0:03.20 apache2 30331 www-data 20 0 244m 65m 4404 R 30 0.4 0:02.96 apache2 30369 www-data 20 0 233m 53m 4432 R 24 0.3 0:02.36 apache2 30410 www-data 20 0 231m 51m 4348 S 22 0.3 0:02.18 apache2 30458 www-data 20 0 224m 45m 4372 S 19 0.3 0:01.86 apache2 30506 www-data 20 0 217m 38m 4324 R 15 0.2 0:01.50 apache2 30557 www-data 20 0 212m 32m 4356 R 12 0.2 0:01.20 apache2 30620 www-data 20 0 207m 27m 4356 S9 0.2 0:00.92 apache2 This is a significant jump in memory allocation, and children get spawned. I expect the children per my configuration, but the memory use is obscene. At three minutes (stopped ab at this point) 30253 www-data 20 0 366m 187m 4636 S3 1.2 0:09.52 apache2 30252 www-data 20 0 364m 185m 4656 S3 1.2 0:09.46 apache2 30251 www-data 20 0 363m 184m 4668 S3 1.1 0:09.40 apache2 30250 www-data 20 0 361m 182m 4632 S3 1.1 0:09.30 apache2 30249 www-data 20 0 359m 179m 4676 S3 1.1 0:09.14 apache2 30331 www-data 20 0 355m 176m 4668 S3 1.1 0:08.94 apache2 30369 www-data 20 0 343m 164m 4636 S3 1.0 0:08.34 apache2 30410 www-data 20 0 343m 163m 4628 S3 1.0 0:08.26 apache2 30458 www-data 20 0 337m 158m 4684 S3 1.0 0:07.96 apache2 30557 www-data 20 0 320m 141m 4608 R3 0.9 0:07.08 apache2 30620 www-data 20 0 320m 140m 4676 S3 0.9 0:07.06 apache2 30675 www-data 20 0 315m 135m 4620 S3 0.8 0:06.74 apache2 30731 www-data 20 0 311m 131m 4624 S3 0.8 0:06.54 apache2 30841 www-data 20 0 300m 121m 4656 S3 0.8 0:05.96 apache2 30901 www-data 20 0 295m 116m 4608 S3 0.7 0:05.70 apache2 30957 www-data 20 0 294m 114m 4624 S3 0.7 0:05.62 apache2 Now ab is not running, but my stopwatch program is. Now at 5 minutes: 30458 www-data 20 0 341m 162m 4692 S0 1.0 0:08.16 apache2 31126 www-data 20 0 290m 111m 4596 S0 0.7 0:05.42 apache2 31112 www-data 20 0 288m 109m 4604 S0 0.7 0:05.30 apache2 31210 www-data 20 0 287m 108m 4612 S0 0.7 0:05.24 apache2 31257 www-data 20 0 287m 107m 4600 S0 0.7 0:05.24 apache2 31222 www-data 20 0 286m 107m 4636 S0 0.7 0:05.20 apache2 31410 www-data 20 0 285m 105m 4588 S0 0.7 0:05.10 apache2 31412 www-data 20 0 284m 105m 4656 S0 0.7 0:05.12 apache2 31396 www-data 20 0 284m 105m 4632 S0 0.7 0:05.12 apache2 Note that although some threads do disappear (normal, expect) the ones that stay never free up any RAM. If I were to let this just chill by itself for a few days, res would be a gig or so and virt would be up there. And for good measure, ten minutes: 30458 www-data 20 0 341m 162m 4692 S0 1.0 0:08.22 apache2 31126 www-data 20 0 291m 111m 4604 S0 0.7 0:05.48 apache2 31112 www-data 20 0 290m 110m 4620 S0 0.7 0:05.42 apache2 31222 www-data 20 0 288m 109m 4652 S0 0.7 0:05.34 apache2 31210 www-data 20 0 288m 109m 4616 S0 0.7 0:05.36 apache2 31257 www-data 20 0 287m 108m 4604 S0 0.7 0:05.30 apache2 31410 www-data 20 0 286m 107m 4604 S0 0.7 0:05.22 apache2 31396 www-data 20 0 286m 106m 4636 S0 0.7 0:05.24 apache2 31384 www-data 20 0 286m 106m 4616 S0 0.7 0:05.22 apache2 31412 www-data 20 0 285m 106m 4660 S0 0.7 0:05.18 apache2 The server isn't being used much right now, but as you can see, no memory has really been free'd. Right now I am working around this by reloading apache at 11:59 every night (the saddest cron job on earth), which appears to bring memory usage back down. -- Slow memory leak, seen on two machines, appears to be dupe of 224945 even after -updates https://bugs.launchpad.net/bugs/422138 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to apache2 in ubuntu. -- Ubuntu-server-bugs mailing list
