As an additional note. The listener socket backlog for daemon mode processes is 100. I think Apache is similar, but is configurable where as that for daemon mode isn't.
What this means is that for single daemon process group, if number of Apache processes threads is more than 100 greater than across the daemon process group, and daemon process group threads all block doing something, the connections from Apache processes may eventually start failing when the listen backlog of 100 fills up. Rather than fail outright, mod_wsgi attempts to reconnect with period between reconnects to daemon processes increasing each time. It will only give up after a total of 30 seconds of trying. This retry and back off is mainly to cope with when daemon processes are restarting, but also comes into play in general when threads all consumed in daemon process and the listener queue has filled up as well. So, still in the end takes quite a bit for Apache processes themselves to start failing requests with HTTP status of Service Unavailable due to not being able to get a connection to daemon processes. Although no hard fast rule, I would probably try with number of Apache processes total threads be 25% greater than across daemon process group, but have nothing scientific to back that up. When NewRelic available for Python then maybe can get some more useful data about what is best configuration as it can graph request queue time. That is, for a request, how long was spent between when Apache process first accepted request, and when WSGI application started handling it in daemon process. Graham On 16 February 2011 10:23, Angus <[email protected]> wrote: > Definitely makes sense to me. I decided to try this 1 to 1 setup > anyway, but having read your email I now have a clearer idea of what > properties are likely useful to measure and compare. If I find > something interesting or unexpected, I'll post it here. Thanks a lot. > > Wol > > > > On Wed, Feb 16, 2011 at 10:47 AM, Graham Dumpleton > <[email protected]> wrote: >> On 15 February 2011 20:41, Wol Degodver <[email protected]> wrote: >>> I have Apache (2.2, worker mpm) dedicated to serving only mod_wsgi >>> (django) stuff (a nginx frontend does the rest). mod_wsgi is running >>> in daemon mode with the default of 1 process and 15 threads >>> (susceptible to change if I get more visitors). Since those values are >>> set as hard limits, I figure I set Apache to limit the process and >>> thread count to the same values like so: >>> >>> ServerLimit 1 >>> StartServers 1 >>> MaxClients 15 >>> MinSpareThreads 1 >>> MaxSpareThreads 15 >>> ThreadsPerChild 15 >>> >>> Is this indeed a good idea? If not, why does my dedicated apache >>> perhaps need to get some extra leeway? >> >> Because nginx only uses HTTP/1.0 when it proxies and doesn't support >> keep alive connections, then technically a 1 to 1 relationship of >> number of threads in Apache processes with number of threads across >> daemon mode processes should be fine. >> >> The only implication of this is that if more concurrent requests queue >> up than number of threads that they will queue up on the main listener >> socket of the Apache processes themselves. If the number of threads in >> Apache processes were more than across the number in the daemon >> processes, then instead of queueing up as socket connects on listener >> socket, they would get accepted by Apache processes and instead queue >> up on the listener socket used by the daemon processes internally. >> >> Right now not sure there is any significance to the distinction. May >> only be relevant if nginx was load balancing between multiple >> Apache/mod_wsgi backends and only for requests which have more than >> about 1MB in body of request. This is because if the connection to >> Apache fails, for a >1MB size request, because it hasn't started >> streaming the request content, can still possibly fail over to another >> backend if the connection ultimately fails. If Apache has instead >> accepted it, then it would have already started streaming the data and >> can't fail over if there is an error with socket connection being lost >> before a response is received. >> >> Overall, would need a nginx proxying/load balancing expert to comment >> about all that though as I am not sure how nginx load balancing and >> failover works. >> >> Not sure if that is helpful or just confusing. >> >> Graham >> >> -- >> You received this message because you are subscribed to the Google Groups >> "modwsgi" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]. >> For more options, visit this group at >> http://groups.google.com/group/modwsgi?hl=en. >> >> > > -- > You received this message because you are subscribed to the Google Groups > "modwsgi" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/modwsgi?hl=en. > > -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en.
