> Any ideas how could I reduce pylons app RAM usage? At the moment
> pylons process takes above 100MB (almost static site serving some
> templates) - both when run with paste, and when run under mod_wsgi.
> Quite a lot, considering that for example whole moinmoin manages to
> work on ~25MB.

Short summary for possible future readers:

a) High pylons virtual memory usage is almost solely caused by the
   huge stack allocated on Linux by default (default Linux stack size
   is 8MB). It is still not quite clear whether this is really
   troublesome (it looks like in typical configurations this stack
   remains purely virtual number), but in some configurations may
   theoretically cause trouble (like reaching VPS limits)

b) Easy way to force this memory down is to use ulimit to narrow
   the stack size. For example
              ulimit -s 512
   to use 512kB stack (the command may be incjected in shell script
   starting paste or in similar place).

   More or less equivalent solution for python>=2.5
   is to call
              import thread
              thread.stack_size(512 * 1024)
   in the application initialization code. An elegant way is to move
   this number to the configuration (like threadpool_worker_stack_size)

c) While tuning the memory consumption it also makes sense to 
   pay attention to the following runtime parameters:

     threadpool_workers = 10    # 10 is default

     sqlalchemy.default.pool_size = 3     # 5 is default
     sqlalchemy.default.max_overflow = 7  # pool_size+max_overflow = max 
simultaneous database sessions

   Those impact not only virtual, but also real memory usage....

The text above is related to running application under paste, but
similar idea can be applied while using mod_wsgi. If using embedded
mod_wsgi, one may consider using Apache ThreadStackSize directive, if
using mod_wsgi daemon mode (which allows one to configure amount of
working processess and threads in detail) ulimit -s may be used
(future releases of mod_wsgi are also to have configuration parameters
to tune this).

Thanks everybody for many valuable suggestions.

PS This thread did not resolve the old 'what is the best pylons
hosting method' question, as it was not its purpose. There are many
options, for example direct paste, paste hidden behind reverse proxy
(probably best if light, like nginx or cherokee), embedded mod_wsgi,
mod_wsgi in daemon mode, fastcgi, mod_python...

Personally I feel that in typical application it is Good Thing to
concentrate Pylons in one (or a few) multithreaded processess (so
python code, template caches, database pools etc may be shared between
threads) dedicated solely to this task (so their memory is not wasted
for trivial tasks like serving images) - so either paste shielded
behind light reverse proxy (which also directly serves static data),
or mod_wsgi daemon mode seem to be most promising. But in case
somebody would like to discuss this, please ... start new thread ;-)

You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 

Reply via email to