2010/1/22 danjac <[email protected]>: > We are running a number of fairly average volume Django sites (approx > 20000 uniques a day). > > Since moving to mod_wsgi from mod_python a month ago, we have been > experiencing occasional high memory usage (every couple of days or so) > leading to > serious unresponsiveness, which is only resolved by restarting Apache.
Have you checked the obvious of ensure that DEBUG is set to False in Django configuration? > During these periods our traffic has been minimal as has been our > database load. > > We are running the sites on two load-balanced servers (the database > and > media server are on different machines). There are approx 60 sites in > total, all running on the same codebase but in separate VirtualHosts > and WSGIDaemonProcesses. > > I was wondering if the issue could be with the large number of child > processes, as we have so many. > We have tried tinkering with the configuration by adding maximum- > requests and inactivity-timeout (the > traffic varies from site to site a great deal) but the problem is > recurring. > > I'd like to be able to eliminate mod_wsgi as the issue by ensuring > that > we have the optimum configuration for our case and we aren't doing > anything stupid. > > If Apache/mod_wsgi isn't the problem then the issue is most likely our > own code, but I'd like to > avoid premature optimization. > > Here is our Apache configuration: > > Apache settings (prefork): > > StartServers 8 > MinSpareServers 5 > MaxSpareServers 30 > ServerLimit 30 > MaxClients 30 > MaxRequestsPerChild 150 Are you running PHP on the same site? If you aren't can you switch to worker MPM for Apache. That will at least drop down the number of Apache server child processes. You also seem to have MaxRequestsPerChild set version low for where mod_wsgi daemon mode is used and Apache server child processes therefore only handling static files and proxying. Even if using PHP, it wouldn't normally need to be that low and in fact could be 0 so that processes don't even get recycled based on number of requests. > and a typical WSGI configuration (60+ of them, identical configuration > except for names): > > <VirtualHost *.80> > ServerName site1.com > ServerAlias site1.com > WSGIScriptAlias / /path/to/site1/site.wsgi > WSGIDaemonProcess site1.com threads=10 display-name=wsgi.site1 > maximum-requests=3000 deadlock-timeout=30 inactivity-timeout=300 You could drop inactivity-timeout to 60 seconds if want to be a bit more aggressive about throwing out idle applications. Really depends a bit on what application is used for > WSGIProcessGroup site1.com > </VirtualHost> Other than, can't see anything out of the ordinary that I would be worried about. If running on a VPS, you could try dropping down default per thread stack size as documented in: http://code.google.com/p/modwsgi/wiki/ApplicationIssues#Memory_Constrained_VPS_Systems Not sure this is really a good match for your problem though. Would suggest perhaps looking at the size of results from database queries in your application. Maybe there are specific URLs which have a large transient memory usage for the data set or the resultant objects from processing the data. Obviously once this memory is allocated it stays allocated by the process. Not sure whether you might be able to have a basic memory usage monitor going using 'ps' and see if you can spot where memory jumps and align that with specific URL requests occurring. Even if can't get anything from that, monitoring would show whether memory usage is incremental or sudden. If incremental, maybe you have a object reference cycle with multiple __del__ methods and garbage collector can't break the cycle and reclaim the Python objects and the memory they use. Graham -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en.
