Re: Reducing pylons app memory usage?

[Ben Bangert]

On Oct 31, 2007, at 9:41 AM, Marcin Kasperski wrote:

This is true. But this is also relatively easy to measure, it suffices
to compare virtual size of the running app with virtual size of python
which just loaded all the libraries. The latter is fairly well
approximated by paster shell.

The latter allocates about 27MB of virtual on my host (but almost 20MB
of resident).

At 15 threadpool:
VM: 65MB, Res: 17.13MB

At 65 threadpool:
VM: 90.65MB, Res: 18.13MB





Can we please stop worrying over the virtual numbers now?



In the meantime, I'm considering the virtual memory issue a moot point.

Cheers,
Ben

To try and illustrate how useless the virtual number is that you're so worried about. On my OSX unix box, according to the OS, its using 9.6GB of Virtual Memory. Now, I know where the real swap file it uses, is actually stored, and while top says 9.6GB of virtual, the swap file is 64megs.... For another test, to really try and gauge what is occurring, I opened up some swap counters, to watch swap files page in, and page out. Then I started increasing the threadpool workers to see whether swapping was occurring. So yes, increasing threadpool workers is definitely making virtual memory increase. Should we all be worried and stress out over it? Well, I was watching paging and swap activity the entire time, and there was *Zero* pager activity. Zero pageins, Zero pageouts, Zero increase in swap. Let me re-iterate this again, the increase in virtual memory did *not* cause any increase in swap. If you have some sort of hard data about real swap activity, I'd love to see it, but posting virtual numbers is pointless and does not represent swap usage. If there's some other unix expert that can shine more light on what the heck this virtual number does mean, that'd be great.

smime.p7s
Description: S/MIME cryptographic signature