Re: Apache::Session::File and free memory weirdness
Hi Perrin - > > Odd thing #2: (This part seems most bizarre to me.) At 5:15 AM, we run a > > Perl script that finds and deletes Apache::Session::File session and lock > > files that are older than 28 days. Usually there are about 50,000 old > > files that get deleted out of about 2,300,000 total. Almost immediately, > > free memory on the machine jumps back up to 1.3GB. What's up with that? > > Sounds like you're counting the buffers and cache in your "used" memory. > Depending on what OS you're on, you may want to look at a tool other than > top. The memory used for buffers and cache will be available to > applications if they need it. That's exactly it, thanks for the pointer Perrin! Turns out "free" memory as reported on Solaris includes the file system cache. On a busy system it will always look like there's no almost free memory, when in reality memory used by the file cache gets released immediately when an application needs it. I found some good general info at: http://www.sun.com/sun-on-net/performance/vmsizing.pdf Larry Leszczynski [EMAIL PROTECTED]
Re: Apache::Session::File and free memory weirdness
Perrin Harkins wrote: > > > Odd thing #1: As it gets into evening time, load on the machine drops off > > and there are fewer httpd children running, but I am not seeing free > > memory return to that 1.3GB level. At most it comes back up to 400MB or > > so. I don't think the httpd children are hanging on to memory, because > > they cycle through pretty quickly - MaxRequestsPerChild is set to 512 and > > none of the processes are ever more than a couple minutes old when I look > > in. Is there any reason to think the parent httpd process would hang on > > to anything? > > I have noticed that over time the new processes will spawn with less memory > shared. I'm not sure exactly why this is, but it does seem to happen. A > nightly complete restart of the server will reset things, but you may not > have that option. > I have noticed this also. I have a theory about its cause, but I haven't had the free time to hack with it yet. My theory is that when the VM subsystem swaps pages of the mod_perl parent process, those pages become forever unshared (even after they return to memory from swap). So I was going to test the mlockall() Linux system call to see if it made a difference. (Search the archives for "mlockall"). However, I am using the Linux platform - the VM behavior may be totally different on Solaris. If anyone else has some time to play with mlockall() on Linux, I'd like to know the results. It'll be a few more weeks till I get a chance. -Adi
Re: Apache::Session::File and free memory weirdness
> Odd thing #1: As it gets into evening time, load on the machine drops off > and there are fewer httpd children running, but I am not seeing free > memory return to that 1.3GB level. At most it comes back up to 400MB or > so. I don't think the httpd children are hanging on to memory, because > they cycle through pretty quickly - MaxRequestsPerChild is set to 512 and > none of the processes are ever more than a couple minutes old when I look > in. Is there any reason to think the parent httpd process would hang on > to anything? I have noticed that over time the new processes will spawn with less memory shared. I'm not sure exactly why this is, but it does seem to happen. A nightly complete restart of the server will reset things, but you may not have that option. > Odd thing #2: (This part seems most bizarre to me.) At 5:15 AM, we run a > Perl script that finds and deletes Apache::Session::File session and lock > files that are older than 28 days. Usually there are about 50,000 old > files that get deleted out of about 2,300,000 total. Almost immediately, > free memory on the machine jumps back up to 1.3GB. What's up with that? Sounds like you're counting the buffers and cache in your "used" memory. Depending on what OS you're on, you may want to look at a tool other than top. The memory used for buffers and cache will be available to applications if they need it. - Perrin
Apache::Session::File and free memory weirdness
Hi All - I'm running Apache, mod_perl and HTML::Mason on Solaris 2.6, and using Apache::Session::File for session management. I've been monitoring free memory as reported by top, and I'm seeing some behavior that is totally baffling me. (If you're interested, there's a graph at: http://www.furph.com/graph.png) Here's the scenario: Around 6 AM, when things are relatively quiet, the graph shows about 1.3GB free memory (out of 4GB total). As traffic picks up during the course of the day, free memory drops to about 300MB by 3 or 4 PM. So far so good, no big surprise - there's a lot more httpd processes running so you'd expect more memory in use. Odd thing #1: As it gets into evening time, load on the machine drops off and there are fewer httpd children running, but I am not seeing free memory return to that 1.3GB level. At most it comes back up to 400MB or so. I don't think the httpd children are hanging on to memory, because they cycle through pretty quickly - MaxRequestsPerChild is set to 512 and none of the processes are ever more than a couple minutes old when I look in. Is there any reason to think the parent httpd process would hang on to anything? Odd thing #2: (This part seems most bizarre to me.) At 5:15 AM, we run a Perl script that finds and deletes Apache::Session::File session and lock files that are older than 28 days. Usually there are about 50,000 old files that get deleted out of about 2,300,000 total. Almost immediately, free memory on the machine jumps back up to 1.3GB. What's up with that? If I run the script during the middle of the day, when things are busier, I still see the free memory jump up although not all the way to 1.3GB - maybe to 800MB or so. Because of the rate the httpd children cycle, I don't think it's possible any of them could be holding open filehandles to session files that haven't changed for 28 days. Is there something weird about the way top reports free memory? The numbers I get seem consistent with the "free" column from vmstat. Why would deleting a bunch of files free up 1GB of memory? Any ideas or explanations would be much appreciated! Thanks! Larry Leszczynski [EMAIL PROTECTED]
