I've traced the problem down to one of my controller action that does some in depth analysis on large text files (source code actually). In certain cases, this controller action can have upwards of 10000 files being worked on at the same time, with each one currently represented as a model in the system. It looks like when the files are being analyzed, each model is loaded into the session, but is kept in memory even after the action is complete. Is there a way to "force" sqlalchemy to release models from memory and just (lazily) load them from the database next time their contents are requested? If so, I think that adding this bit to the end of this particular controller action would help take care of my memory issue.
-Charlie On Sat, May 8, 2010 at 7:56 PM, cd34 <[email protected]> wrote: > You can use daemon mode which won't run into quite that much memory > growth if you're not running multiple instances. Embedded mode has > some different memory requirements. > > You might post over on the mod_wsgi list as Graham Dumpleton is very > responsive, and knows the ins and outs and probably will be able to > tell you the steps to take to debug things. > > In a recent version he did put in some limit commands. Are you > running prefork or worker? You could turn down the max requests per > child to cause apache to recycle those processes a little more > quickly. If prefork, do you also have mod_php enabled? If you don't > need mod_php, mpm-worker/mod_wsgi works quite well. > > The only thing I can think that would cause that much memory to be > used, and the growth over time MIGHT be a memory leak somewhere in the > application, or, shared memory that is allocated but not released > somewhere. > > As a last resort you could utilize mod_proxy and cherrypy/paster, but, > I think you'll end up with better performance with mod_wsgi/apache2. > > -- > You received this message because you are subscribed to the Google Groups > "pylons-discuss" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]<pylons-discuss%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/pylons-discuss?hl=en. > > -- Charlie Meyer [email protected] http://www.charliemeyer.net -- You received this message because you are subscribed to the Google Groups "pylons-discuss" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en.
