Behind nginx Django in mod_wsgi works like a charm. (Actually I forgot to update code to use upload module and it appeared to be unnecessary :) )
Thanks for you help, Graham! Best regards, Kirill Zaborsky On Jan 14, 12:50 am, Graham Dumpleton <[email protected]> wrote: > 2010/1/14 qrilka <[email protected]>: > > > I could not find anything about uploads proxying in nginx docs. > > Did I miss something? > > The mod_proxy module doesn't distinguish types of requests, any HTTP > request can have a body content and all mod_proxy does is pass that > data through to the backend as is for the original web application to > process. The module you are talking about actually tries to interpret > the data and splits it into files as I understand it. The web > application would then need to be modified somewhat to be able to pick > up special headers about where uploaded files were placed. As such, > that module will be incompatible with an existing Python web > frameworks file upload mechanisms. > > Graham > > > Anyways looks like we will try that module as it seem to work OK (but > > some hours later) > > > Best regards, > > Kirill Zaborsky > > > On Jan 13, 1:00 pm, Graham Dumpleton <[email protected]> > > wrote: > >> 2010/1/13 qrilka <[email protected]>: > > >> > No, I don't see anything like that (at least at the time of the latest > >> > 'Premature end' message). > >> > Nginx upload module [1] requires some extra work and testing so it is > >> > not on the production server yet. > > >> FWIW, the benefits of using nginx in front to isolate Apache from slow > >> clients doesn't require that module. What I was talking about is > >> simply an aspect of how the standard nginx mod_proxy implementation > >> works. > > >> Graham > > >> > I will let know if there will be any new details on this issue. > >> > BTW it seems to be quite unlikely that I have some 'special' web > >> > application which requires something like a voodoo magic and I don't > >> > find any reports about such problems in the internet. It appears to me > >> > to be some kind of server misconfiguration but I don't see any new > >> > places where that misconfiguration could be. > > >> > [1]http://www.grid.net.ru/nginx/upload.en.html > > >> > Thanks once again. > > >> > Best regards, > >> > Kirill Zaborsky > > >> > On Jan 13, 2:49 am, Graham Dumpleton <[email protected]> > >> > wrote: > >> >> 2010/1/13 qrilka <[email protected]>: > > >> >> > There is no CGI scripts on that server. > >> >> > Just some PHP sites and Django. > >> >> > And the message appears in Django virtual host log. > > >> >> Be aware that if this is being caused by a process crash, then the > >> >> Segmentation Fault message will appear in the main Apache error log > >> >> and not in the virtual host specific error log as is Apache parent > >> >> monitoring process which is monitoring the fact the process died and > >> >> it isn't linked to a specific virtual host. > > >> >> So, ensure you are paying attention to any error messages in the main > >> >> Apache error log at the same time, including those which aren't > >> >> specifically tagged as coming from mod_wsgi. If you find anything of > >> >> relevance, then let me know. > > >> >> Graham > > >> >> > Best regards, > >> >> > Kirill Zaborsky > > >> >> > On Jan 13, 1:33 am, Graham Dumpleton <[email protected]> > >> >> > wrote: > >> >> >> 2010/1/13 qrilka <[email protected]>: > > >> >> >> > Nobody else is doing anything with Apache but still I see the same > >> >> >> > errors. > >> >> >> > Today I'll try to setup nginx to see if it will help. > > >> >> >> The message 'Premature end of script headers' also gets generated by > >> >> >> broken CGI scripts. So, you need to look quite closely at error logs > >> >> >> and ensure that they are in fact linked to a WSGI request. > > >> >> >> Graham > > >> >> >> > Best regards, > >> >> >> > Kirill Zaborsky > > >> >> >> > On Jan 12, 1:39 am, Graham Dumpleton <[email protected]> > >> >> >> > wrote: > >> >> >> >> 2010/1/11 qrilka <[email protected]>: > > >> >> >> >> > Thanks for your reply, Graham. > >> >> >> >> > It was quite clear before that I need to spend more time on > >> >> >> >> > proper > >> >> >> >> > server configuration (e.g. Apache now isn't quite tuned at > >> >> >> >> > least I > >> >> >> >> > need to separate python from PHP). I will look into you > >> >> >> >> > proposal on > >> >> >> >> > using nginx for proxying long-running requests. > >> >> >> >> > BTW I removed maximum-requests but still see 'Premature end of > >> >> >> >> > script > >> >> >> >> > headers' messages in error log. > >> >> >> >> > What could be a reason for that? > >> >> >> >> > Are there any other restarts for mod_wsgi except changed wsgi > >> >> >> >> > file and > >> >> >> >> > maximum-requests option? > > >> >> >> >> Could also occur if someone is doing an: > > >> >> >> >> apachectl graceful > > >> >> >> >> Ie., Apache graceful restart. > > >> >> >> >> Can occur here because Apache will allow its own server child > >> >> >> >> process > >> >> >> >> to keep running until active requests complete, but due to way APR > >> >> >> >> library handles the other processes such as mod_wsgi daemon > >> >> >> >> processes, > >> >> >> >> it will kill them off regardless after 3 seconds. Thus, mod_wsgi > >> >> >> >> daemon process goes away and the Apache server child process > >> >> >> >> proxying > >> >> >> >> request to the daemon process will then see connection close and > >> >> >> >> get > >> >> >> >> that error message. > > >> >> >> >> Presumably an Apache graceful shutdown could cause something > >> >> >> >> similar. > > >> >> >> >> The question is therefore if anyone is doing graceful restarts on > >> >> >> >> Apache at same time as you still see the message. > > >> >> >> >> The only other times that error message could arise is if the > >> >> >> >> mod_wsgi > >> >> >> >> daemon process crashed, or if someone has sent an explicit signal > >> >> >> >> to > >> >> >> >> the mod_wsgi daemon process to make it shutdown. > > >> >> >> >> Graham > > >> >> >> >> > Many thanks for you help. > > >> >> >> >> > Best regards, > >> >> >> >> > Kirill Zaborsky > > >> >> >> >> > On Jan 11, 3:23 am, Graham Dumpleton > >> >> >> >> > <[email protected]> > >> >> >> >> > wrote: > >> >> >> >> >> 2010/1/8 qrilka <[email protected]>: > > >> >> >> >> >> > OK, I'll investigate how that could be solved. > >> >> >> >> >> > Actually application is very simple and the only problem > >> >> >> >> >> > could be with > >> >> >> >> >> > some operations on large images. > >> >> >> >> >> > But I do not see how that could lead to some operations > >> >> >> >> >> > taking more > >> >> >> >> >> > than 1 second. > > >> >> >> >> >> That time of 5 seconds isn't just for processing of the image > >> >> >> >> >> but also > >> >> >> >> >> inclusive of the time it takes to upload the image to the > >> >> >> >> >> application > >> >> >> >> >> if what you are doing first involves an upload. > > >> >> >> >> >> Thus, if dealing with large images or slow HTTP clients and > >> >> >> >> >> clients > >> >> >> >> >> are talking direct to Apache, then you may well exceed that > >> >> >> >> >> time. > > >> >> >> >> >> What you can do to partly isolate yourself from problem of > >> >> >> >> >> slow HTTP > >> >> >> >> >> clients is to put nginx proxy in front of Apache. At least for > >> >> >> >> >> files > >> >> >> >> >> up to some default, nginx will buffer the upload before > >> >> >> >> >> actually > >> >> >> >> >> triggering the proxy to the Apache back end. This means that > >> >> >> >> >> request > >> >> >> >> >> only passed onto Apache when data is available and so Apache > >> >> >> >> >> can do > >> >> >> >> >> its job quickly and not be tied up with dealing with slow HTTP > >> >> >> >> >> request. Thus less risk of request being interrupted if > >> >> >> >> >> process does > >> >> >> >> >> indeed fall within that 5 seconds. Only passing on request when > >> >> >> >> >> request data available, also means you will get better > >> >> >> >> >> utilisation > >> >> >> >> >> from Apache processes/threads and can configure it for less, > >> >> >> >> >> thus > >> >> >> >> >> reducing its memory overhead. > > >> >> >> >> >> Right now I can't find the part of the nginx documentation > >> >> >> >> >> that talks > >> >> >> >> >> about request buffering. The proxy documentation tends only to > >> >> >> >> >> talk > >> >> >> >> >> about response buffering as far as configuration parameters. > > >> >> >> >> >> > I though that using maximum-requests could prevent possible > >> >> >> >> >> > memory > >> >> >> >> >> > leaks and exessive memory consumption. > >> >> >> >> >> > Isn't it a right supposition? > > >> >> >> >> >> It can, but as described can cause conflict with long running > >> >> >> >> >> uploads/requests if they are greater than default shutdown > >> >> >> >> >> timeout of > >> >> >> >> >> 5 seconds. You can adjust the shutdown timeout using > >> >> >> >> >> shutdown-timeout > >> >> >> >> >> option to WSGIDaemonProcess, but make it too long and you risk > >> >> >> >> >> perceived delays by user if all daemon mode processes in group > >> >> >> >> >> restart > >> >> >> >> >> about the same time. > > >> >> >> >> >> > And also if I understand it right I will get the same errors > >> >> >> >> >> > on > >> >> >> >> >> > application update when my WSGI application will be > >> >> >> >> >> > restarted. > > >> >> >> >> >> Yes, the shutdown timeout comes into play on any self restart > >> >> >> >> >> of > >> >> >> >> >> mod_wsgi daemon processes. > > >> >> >> >> >> The only time that shutdown timeout doesn't apply is when you > >> >> >> >> >> do a > >> >> >> >> >> full Apache 'restart' or 'graceful'. In that case Apache itself > >> >> >> >> >> applies a 3 second timeout and will forcibly kill the mod_wsgi > >> >> >> >> >> daemon > >> >> >> >> >> mode processes after that. Can't override that specific Apache > >> >> >> >> >> timeout. > > >> >> >> >> >> > Do you have any thoughts how could I find any misbehaving > >> >> >> >> >> > long-running > >> >> >> >> >> > process with a stacktrace? > > >> >> >> >> >> One could use WSGI wrappers around your application object to > >> >> >> >> >> add a > >> >> >> >> >> request timer, but would first contemplate on whether it is > >> >> >> >> >> the upload > >> >> >> >> >> time for file rather than processing time. > > >> >> >> >> >> Graham > > >> >> >> >> >> > Best regards, > >> >> >> >> >> > Kirill Zaborsky > > >> >> >> >> >> > On Jan 8, 1:06 am, Graham Dumpleton > >> >> >> >> >> > <[email protected]> > >> >> >> >> >> > wrote: > >> >> >> >> >> >> 2010/1/8 qrilka <[email protected]>: > > >> >> >> >> >> >> > From VirtualHost specific log: > >> >> >> >> >> >> > ------------------------ > >> >> >> >> >> >> > [Thu Jan 07 21:09:49 2010] [info] mod_wsgi (pid=12366): > >> >> >> >> >> >> > Maximum > >> >> >> >> >> >> > requests reached 'av_factory'. > >> >> >> >> >> >> > [Thu Jan 07 21:09:49 2010] [info] mod_wsgi (pid=12366): > >> >> >> >> >> >> > Shutdown > >> >> >> >> >> >> > requested 'av_factory'. > >> >> >> >> >> >> > [Thu Jan 07 21:09:54 2010] [info] mod_wsgi (pid=12366): > >> >> >> >> >> >> > Aborting > >> >> >> >> >> >> > process 'av_factory'. > > >> >> >> >> >> >> This line indicates that what I > > ... > > read more »
-- You received this message because you are subscribed to the Google Groups "modwsgi" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en.
