> Any ideas how could I reduce pylons app RAM usage? At the moment > pylons process takes above 100MB (almost static site serving some > templates) - both when run with paste, and when run under mod_wsgi. > Quite a lot, considering that for example whole moinmoin manages to > work on ~25MB.
Short summary for possible future readers: a) High pylons virtual memory usage is almost solely caused by the huge stack allocated on Linux by default (default Linux stack size is 8MB). It is still not quite clear whether this is really troublesome (it looks like in typical configurations this stack remains purely virtual number), but in some configurations may theoretically cause trouble (like reaching VPS limits) b) Easy way to force this memory down is to use ulimit to narrow the stack size. For example ulimit -s 512 to use 512kB stack (the command may be incjected in shell script starting paste or in similar place). More or less equivalent solution for python>=2.5 is to call import thread thread.stack_size(512 * 1024) in the application initialization code. An elegant way is to move this number to the configuration (like threadpool_worker_stack_size) c) While tuning the memory consumption it also makes sense to pay attention to the following runtime parameters: [server:main] threadpool_workers = 10 # 10 is default [app:main] sqlalchemy.default.pool_size = 3 # 5 is default sqlalchemy.default.max_overflow = 7 # pool_size+max_overflow = max simultaneous database sessions Those impact not only virtual, but also real memory usage.... The text above is related to running application under paste, but similar idea can be applied while using mod_wsgi. If using embedded mod_wsgi, one may consider using Apache ThreadStackSize directive, if using mod_wsgi daemon mode (which allows one to configure amount of working processess and threads in detail) ulimit -s may be used (future releases of mod_wsgi are also to have configuration parameters to tune this). Thanks everybody for many valuable suggestions. PS This thread did not resolve the old 'what is the best pylons hosting method' question, as it was not its purpose. There are many options, for example direct paste, paste hidden behind reverse proxy (probably best if light, like nginx or cherokee), embedded mod_wsgi, mod_wsgi in daemon mode, fastcgi, mod_python... Personally I feel that in typical application it is Good Thing to concentrate Pylons in one (or a few) multithreaded processess (so python code, template caches, database pools etc may be shared between threads) dedicated solely to this task (so their memory is not wasted for trivial tasks like serving images) - so either paste shielded behind light reverse proxy (which also directly serves static data), or mod_wsgi daemon mode seem to be most promising. But in case somebody would like to discuss this, please ... start new thread ;-) --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "pylons-discuss" group. To post to this group, send email to email@example.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~----------~----~----~----~------~----~------~--~---