[ Sorry again for the delayed reply. ] Stefan Fritsch wrote on Tue, Oct 11, 2011 at 21:35:08 +0200: > On Tuesday 11 October 2011, Jim Jagielski wrote: > > On Oct 11, 2011, at 3:01 PM, Jim Jagielski wrote: > > > On Oct 11, 2011, at 2:09 PM, Daniel Shahaf wrote: > > >> Paul Querna wrote on Sun, Sep 11, 2011 at 12:10:53 -0700: > > >>> Infra has upgraded eos, aka the main webserver for *.apache.org > > >>> to 2.3.15-dev-r1167603 > > >> > > >> We observe processes whose memory, in trend, grows over time up > > >> to 2.8GB and more, for example: > > >> > > >> http://www.apache.org/eos/mem.txt > > > > > > which mom? > > > > mpm even :) > > mpm event: http://www.apache.org/server-status > At least I assume that's eos. >
Yes. eos.apache.org == 140.211.11.131 > Is this a new problem since you upgraded? > I'm told that it's not a new problem. > > I think the problem may be the combination of > - by default, httpd never gives memory back to the OS > - mpm event creates transaction pool with a separate allocator for > every connection > - by default, it never destroys transaction pools ... > Can you try setting MaxMemFree, maybe to 4000? This will limit the > maximum free memory kept in each allocator to 4000K. It will also > cause mpm event to destroy transaction pools if more than 128 * 0.75 > == 96 are unused, but I am not sure that will make a difference if the > load is more or less constant. > Thanks for your reply. Besides the theory you suggest above, #asfinfra residents suggested to isolate mod_mbox. I started by testing that theory (sorry, it does exist even though it wasn't on-list). I changed mail-archives.apache.org DNS entry to point at aurora (in Amsterdam) rather than eos (in OSUOSL). Halfway through the one hour DNS propagation period a graceful restart was done. Looking at top now (after DNS finished propagating), the memory use is at around 650MB and at no point exceeded 750MB (in manual spot checks, not yet in a scripted every-5-seconds check; I'll do that later). By comparison, before the DNS change and the graceful, I saw two processes at 2GB+, and in an unscientific glimpse it did seem to be growing. So, this is preliminary data that appears to corroborate the mod_mbox theory. It does not rule out the allocators theory, and I plan to collect more data in the current configuration to see if the processes continue to behave. > > Cheers, > Stefan Thanks, Daniel
