2009/12/24 Szymon <[email protected]>: > Hi, > > I've removed PHP, turned Apache to worker, changed wsgi daemon to > processes=4 threads=75... and omg. It's HUGE difference. :O Really. > From 60-70 requests currently being processed Apache goes down to > 10-15. Thanks for that blog note! Link should be in some visible place > on mod_wsgi home page! :)
You probably don't even need 75 threads per process and may even find default of 15 per process is more than sufficient. Whether that will be true though depends on how many long running requests you expect to get. If all requests are handled in under 100ms, then even with only 4x15 threads, that is theoretical 600 requests/sec you could handle, although in practice overheads mean that will come down. I have seen people who were still getting very good performance on a high volume site out of running 3 single threaded processes. Granted they were good at what they did and had tuned the hell out of their application and database and used caching well to remove all the bottlenecks from it so request times were very very short. Do you have any idea what you average request times are and what proportion of requests are longer and what maximum times are. As I alluded to by my questions and as someone else pointed out, you can also get better utilisation out of Apache/mod_wsgi by using a nginx front end proxy. Can be same one serving static files. Why this works is that nginx acts to isolate Apache/mod_wsgi from slow HTTP clients as nginx for request body under a certain size will buffer up request headers and request content and only proxy it on when it has it all. This means that Apache/mod_wsgi will only get the request when it has all of it and is able to process it immediately and dump back response. The buffering implicit in socket connections and nginx for response also means Apache/mod_wsgi can release the connection quicker as well and nginx again will deal with a slow client consuming the response. Using nginx as front end proxy like this will allow you to get more out of Apache/mod_wsgi with less processes/threads and thus less memory overall. Likely a lot of your problems were due to memory needing to be paged and different processes worken up, due to having many large fat processes. Graham > On 23 Gru, 22:14, Szymon <[email protected]> wrote: >> On 23 Gru, 22:00, Graham Dumpleton <[email protected]> wrote: >> >> > 1. Are you using separate media server for static files? >> >> Nope. Static content is served from same server, but using nginx. >> >> > 2. Are you using a front end proxy to Apache/mod_wsgi like nginx? >> >> No. >> >> > 3. Are you using prefork or worker MPM? >> >> Prefork. >> >> > 4. Are you running other web application on same Apache using mod_php >> > or other systems? >> >> Yes, there is one PHP app, but it's not busy at all. But if it's make >> difference I can resign from that app at all. >> >> > 5. Is your Python web application running in embedded mode or daemon mode? >> >> Daemon mode. >> >> > 6. Have you read: >> >> > http://blog.dscpl.com.au/2009/03/load-spikes-and-excessive-memory-usa... >> >> No. :) But I'll do. :) >> >> > It is hard to give any advice without knowing. For all we know, you >> > could just have configured your current system wrongly and if tweaked >> > it could work fine. >> >> Here is my configuration: >> >> WSGIDaemonProcess app user=app group=app processes=30 maximum- >> requests=10000 >> WSGIProcessGroup app >> >> And my prefork config: >> >> StartServers 5 >> MinSpareServers 10 >> MaxSpareServers 25 >> MaxClients 250 >> MaxRequestsPerChild 0 >> >> Keep-Alive is turned off. I'm on FreeBSD if that's make difference. > > -- > > You received this message because you are subscribed to the Google Groups > "modwsgi" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/modwsgi?hl=en. > > > -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en.
