Thanks for the quick response, Graham, that's a lot of good info. If I ever find a definitive cause I'll post to this thread. -Greg
On Saturday, February 28, 2015 at 1:20:23 AM UTC-8, Graham Dumpleton wrote: > > > On 28/02/2015, at 7:02 PM, Greg Janée <[email protected] <javascript:>> > wrote: > > On Friday, February 27, 2015 at 7:34:17 PM UTC-8, Graham Dumpleton wrote: >> >> Before I explain a few things, can you tell me what you have the Timeout >> directive set to in Apache? > > > I'm not setting it (actually, I didn't know there was such a directive), > so I guess it has the default value of 5min. > > Also, do you have the WSGIProcessGroup directive definitely set so that >> requests are delegated to the daemon mode process? >> > > I'm setting: > > WSGIApplicationGroup %{GLOBAL} > WSGIProcessGroup site-1 > > Regarding how I know the delay is previous to my code: I've got Apache > logging %t (time request was received) and %D (time the request took), and > my code separately logs when it gets called. > > > The more likely cause is going to be a slow HTTP client. Specifically, a > mobile device. > > The initial time stamp and when Apache starts timing from is the initial > accept of the HTTP request list. If the HTTP client is then slow in > delivering the actual HTTP headers, then it will be blocked up waiting for > those to be received. > > In my day job I see this a lot in customers data where they have a high > percentage of mobile devices where that is their targeted customer base. > > Now you are actually on an ancient mod_wsgi version which no one should > really be using, especially in conjunction for Apache 2.2. > > Apache 2.2 works in certain ways which cause memory bloat especially in > the case of slow HTTP clients. > > Apache 2.4 solves those problems, with latest mod_wsgi versions also have > workaround for that Apache 2.2 issue. > > More recent mod_wsgi versions also dispense with Apache 1.3 compatibility > which also allows it to dispense with using certain older Apache APIs which > themselves also can cause excess memory usage for slow HTTP clients. > > In Apache 2.4, the default Timeout directive value is now 60 and not 300. > Using 300 is way to high and 60 or lower is generally regarded as preferred > these days. > > So, I would very much recommend you try and upgrade from Apache 2.2 and > mod_wsgi 3.3. At the minimum, use latest mod_wsgi version if can't upgrade > Apache. > > Now one of the things that is also in more recent mod_wsgi versions are > extra key/values in the WSGI environ dictionary which allow you to better > track where a request is being held up. > > mod_wsgi.request_start: '1425114049967656' > mod_wsgi.queue_start: '1425114049971600' > mod_wsgi.script_start: '1425114049972063' > > The value of mod_wsgi.request_start is the time stamp for when Apache > accepted the initial HTTP request line. > > The value of mod_wsgi.queue_start is when mod_wsgi proxies the request > across to the mod_wsgi daemon process. > > And the value of mod_wsgi.script_start is when the request arrives at the > mod_wsgi daemon process and is about to be handled. > > It is therefore much easier to work out where a blockage may be occurring. > > Recent mod_wsgi versions also have a queue-timeout option that can be set > for mod_wsgi daemon processes. If the time between request start and script > start is more than this queue timeout, the request will be rejected > immediately with a HTTP_GATEWAY_TIME_OUT error response. > > This can be used to stop requests that get blocked up in this way even > being passed to the WSGI application on the basis that the HTTP client is > long gone anyway and so there is no point handling it. > > Reducing the Timeout directive down to 60 is also a good idea as the worst > of cases for slow HTTP client would be rejected by Apache even before it > gets to mod_wsgi to begin with. > > As to seeing lots of processes when have spiky traffic, that is normal. > This is because requests are first received by the Apache child worker > processes, which when the request is then handed off to mod_wsgi within > that processes, are then proxied through to the appropriate mod_wsgi daemon > process. > > So on spiky traffic, Apache's dynamic management of the number of Apache > child worker processes kicks in and it creates more. This can be a problem > in itself. > > In general I would say worker MPM or event MPM (Apache 2.4) is a much > better choice if all you are using Apache for is to run a WSGI application > under mod_wsgi daemon mode. You also should ensure you are disabling Python > interpreter initialisation in the Apache child worker processes if the WSGI > application is running in daemon mode. > > For more talk about the issue of process churn and prefork MPM, make sure > you watch: > > http://lanyrd.com/2013/pycon/scdyzk/ > > Also have a read of: > > http://blog.dscpl.com.au/2009/11/save-on-memory-with-modwsgi-30.html > > Graham > > -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/modwsgi. For more options, visit https://groups.google.com/d/optout.
