2010/1/8 qrilka <[email protected]>: > OK, I'll investigate how that could be solved. > Actually application is very simple and the only problem could be with > some operations on large images. > But I do not see how that could lead to some operations taking more > than 1 second.
That time of 5 seconds isn't just for processing of the image but also inclusive of the time it takes to upload the image to the application if what you are doing first involves an upload. Thus, if dealing with large images or slow HTTP clients and clients are talking direct to Apache, then you may well exceed that time. What you can do to partly isolate yourself from problem of slow HTTP clients is to put nginx proxy in front of Apache. At least for files up to some default, nginx will buffer the upload before actually triggering the proxy to the Apache back end. This means that request only passed onto Apache when data is available and so Apache can do its job quickly and not be tied up with dealing with slow HTTP request. Thus less risk of request being interrupted if process does indeed fall within that 5 seconds. Only passing on request when request data available, also means you will get better utilisation from Apache processes/threads and can configure it for less, thus reducing its memory overhead. Right now I can't find the part of the nginx documentation that talks about request buffering. The proxy documentation tends only to talk about response buffering as far as configuration parameters. > I though that using maximum-requests could prevent possible memory > leaks and exessive memory consumption. > Isn't it a right supposition? It can, but as described can cause conflict with long running uploads/requests if they are greater than default shutdown timeout of 5 seconds. You can adjust the shutdown timeout using shutdown-timeout option to WSGIDaemonProcess, but make it too long and you risk perceived delays by user if all daemon mode processes in group restart about the same time. > And also if I understand it right I will get the same errors on > application update when my WSGI application will be restarted. Yes, the shutdown timeout comes into play on any self restart of mod_wsgi daemon processes. The only time that shutdown timeout doesn't apply is when you do a full Apache 'restart' or 'graceful'. In that case Apache itself applies a 3 second timeout and will forcibly kill the mod_wsgi daemon mode processes after that. Can't override that specific Apache timeout. > Do you have any thoughts how could I find any misbehaving long-running > process with a stacktrace? One could use WSGI wrappers around your application object to add a request timer, but would first contemplate on whether it is the upload time for file rather than processing time. Graham > Best regards, > Kirill Zaborsky > > On Jan 8, 1:06 am, Graham Dumpleton <[email protected]> > wrote: >> 2010/1/8 qrilka <[email protected]>: >> >> > From VirtualHost specific log: >> > ------------------------ >> > [Thu Jan 07 21:09:49 2010] [info] mod_wsgi (pid=12366): Maximum >> > requests reached 'av_factory'. >> > [Thu Jan 07 21:09:49 2010] [info] mod_wsgi (pid=12366): Shutdown >> > requested 'av_factory'. >> > [Thu Jan 07 21:09:54 2010] [info] mod_wsgi (pid=12366): Aborting >> > process 'av_factory'. >> >> This line indicates that what I described previously is occurring and >> is likely the cause. >> >> That is, when reaching maximum-requests, there are long running >> requests or stuck requests that don't complete within the default 5 >> second window for shutting down a daemon process. >> >> When that occurs, even though still running the process is forcibly >> exited, even without shutting down Python interpreter properly. As a >> result, the Apache server child process which is proxying that >> specific request to the mod_wsgi daemon mode process sees the >> connection to daemon process abruptly cut off and as such you may see >> errors about premature end of script headers or the various filter >> errors or broken pipe messages depending on where a request was up to. >> >> Do you have any idea about whether you legitimately have requests that >> take longer than 5 seconds to process? >> >> For what reason are you using maximum-requests in the first place? If >> you don't have to use that option for some reason, the issue should be >> avoided. >> >> Graham> [Thu Jan 07 21:09:54 2010] [info] mod_wsgi (pid=28423): Attach >> > interpreter ''. >> > [Thu Jan 07 21:09:54 2010] [info] mod_wsgi (pid=28423): Enable monitor >> > thread in process 'av_factory'. >> > [Thu Jan 07 21:09:54 2010] [info] mod_wsgi (pid=28423): Enable >> > deadlock thread in process 'av_factory'. >> > [Thu Jan 07 21:09:54 2010] [info] [client 188.113.58.162] mod_wsgi > > -- > You received this message because you are subscribed to the Google Groups > "modwsgi" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/modwsgi?hl=en. > > > >
-- You received this message because you are subscribed to the Google Groups "modwsgi" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en.
