iain duncan wrote: > On Mon, 2007-24-12 at 00:19 -0800, iain duncan wrote: >> Hi folks, I'm clearly doing something wrong because my tg apps get all >> bloated after they have been running a while. I'm wondering if anyone >> can comment on gotchas they encountered with their apps in that regard. >> I'm using CherryPy session variables, but I believe they are getting >> deleted properly ( I could well be wrong there, will be investigating >> that ). I also have some widgets instantiated in static space. Other >> than that I'm confused, because I *know* they aren't getting hammered >> with hits. Deployment is apache mod_proxy to the app with supervisord >> keeping it up. >> >> One question, my superivor app was checking on the site by hitting a >> method of the root controller once a minute. Could that be eating ram >> from a keep-alive scenario? > > FWIW, the apps seem to start at about 20megs and then after 10 minutes > or so get up to 90! And when I checked on it after it had been running > for days, it was at 130 megs. :/ > In Fedora Infrastructure, we've had issues with our TG apps taking up more and more memory after time. This growth is seen to some extent whether we're using SQLObect or SQLAlchemy, kid or genshi. The worst offender is an SQLObject/kid application that handles data about our mirrors. It used to grow unreasonably when it served the mirrorlist for each person requesting updates to their system so we had to split that into its own cron job and now we have a cron job that caches fresh data about our mirrors from the TG app once an hour.
We have our TG apps served from multiple servers so we have the luxury of creating a cron job that checks how much memory the apps are using and restarts them if they go over a limit (staggered on each server). I've noticed that our apps which are handling small amounts of data and infrequent connections are pretty well behaved (ie: I've never seen them restarted from our memory-reaper, although we tend to deploy new versions every two-three weeks so the server processes don't run longer than that.) The mirrorlist server still gets restarted every couple days (when it hits 800MB RSS). It's still handling a large amount of data in some of its requests -- we've just cut down on the number of requests it has to serve. Our smolt server, which is using kid and a model converted from SO to SA is our next big offender... but only at the beginning of the month while/after it is hammered with updated hardware details from installed clients. -Toshio
signature.asc
Description: OpenPGP digital signature
