> Any ideas how could I reduce pylons app RAM usage? At the moment
> pylons process takes above 100MB (almost static site serving some
> templates) - both when run with paste, and when run under mod_wsgi.
> Quite a lot, considering that for example whole moinmoin manages to
> work on ~25MB.
Short summary for possible future readers:
a) High pylons virtual memory usage is almost solely caused by the
huge stack allocated on Linux by default (default Linux stack size
is 8MB). It is still not quite clear whether this is really
troublesome (it looks like in typical configurations this stack
remains purely virtual number), but in some configurations may
theoretically cause trouble (like reaching VPS limits)
b) Easy way to force this memory down is to use ulimit to narrow
the stack size. For example
ulimit -s 512
to use 512kB stack (the command may be incjected in shell script
starting paste or in similar place).
More or less equivalent solution for python>=2.5
is to call
import thread
thread.stack_size(512 * 1024)
in the application initialization code. An elegant way is to move
this number to the configuration (like threadpool_worker_stack_size)
c) While tuning the memory consumption it also makes sense to
pay attention to the following runtime parameters:
[server:main]
threadpool_workers = 10 # 10 is default
[app:main]
sqlalchemy.default.pool_size = 3 # 5 is default
sqlalchemy.default.max_overflow = 7 # pool_size+max_overflow = max
simultaneous database sessions
Those impact not only virtual, but also real memory usage....
The text above is related to running application under paste, but
similar idea can be applied while using mod_wsgi. If using embedded
mod_wsgi, one may consider using Apache ThreadStackSize directive, if
using mod_wsgi daemon mode (which allows one to configure amount of
working processess and threads in detail) ulimit -s may be used
(future releases of mod_wsgi are also to have configuration parameters
to tune this).
Thanks everybody for many valuable suggestions.
PS This thread did not resolve the old 'what is the best pylons
hosting method' question, as it was not its purpose. There are many
options, for example direct paste, paste hidden behind reverse proxy
(probably best if light, like nginx or cherokee), embedded mod_wsgi,
mod_wsgi in daemon mode, fastcgi, mod_python...
Personally I feel that in typical application it is Good Thing to
concentrate Pylons in one (or a few) multithreaded processess (so
python code, template caches, database pools etc may be shared between
threads) dedicated solely to this task (so their memory is not wasted
for trivial tasks like serving images) - so either paste shielded
behind light reverse proxy (which also directly serves static data),
or mod_wsgi daemon mode seem to be most promising. But in case
somebody would like to discuss this, please ... start new thread ;-)
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"pylons-discuss" group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/pylons-discuss?hl=en
-~----------~----~----~----~------~----~------~--~---
