I'm pretty sure you're experiencing a standard problem that most of us have been bitten by at one stage or another. Ged is right when he says that 120 apache children at 40MB a pop will eat your RAM up. The piece of the puzzle that you're missing is this: due to the dopey VM code in Linux, when you run out of physical ram, processes get swapped out and at that instant, they become UNSHARED. In other words, if your process is taking up 40MB, of which 30MB is shared, and it gets swapped out, it will get swapped out as a 40MB process. When it gets swapped back in, it will take up 40MB of your precious RAM, and will probably force another apache process to get swapped out and unshared, which will cause another...
It's a tricky problem, but here's how you solve it:
You must, at ALL times, keep your machine from swapping out Apache children. This is tricky. If you're running other things on your machine, like a DB (I assume you're not since you say you're using Oracle) then it becomes nearly impossible to prevent. But if your machine is pretty much just Linux + Apache, you're OK: just make sure that
MaxClients * LargestPossibleChildSize + AllRamUsedByOSAndOtherProcesses < 4GB
Since you're running a front-end Apache too, you need to figure that into the equation. But the key here is that you can NEVER swap out an apache process, or you'll end up in the downward spiral of death, which seems to be what's happening to you. There's no recovery from that unless you can stop apache; only other solution is to reboot. If that's the case, you might as well be using Windows XP.
Hope this helps
Kyle Dawkins
Central Park Software
On Monday, Oct 13, 2003, at 05:34 US/Pacific, Mike Norton wrote:
We are currently experiencing some problems with our mod_perl setup the server we are using is a Dual Xeon 2.4Ghz running Redhat AS 2.1, 4GB Ram , Apache/1.3.27 (Unix) mod_perl/1.27. When the server is under a heavy load it all of a sudden consumes a large amount of swap and the server load shoots through the roof, the machine normally sits quite happily not even touching the swap normally however it seems to consume anything up to 3GB of swap all in one go. Does anyone have any ideas on how to resolve this issue or why it happens ?
Thanks
Mike
