cool. it's not a big issue for me. I've decided to manage the issue in a different way. I'm gonna log in a database the start and end time of each request. Then I can separately monitor any out of control reports/pages and get general stats on the slow requests in the system.
Out of curiosity though is it alright to be spawning threads in mod_wsgi if I'm using daemon mode multi-process single-thread. Assuming I take proper care of starting and ending the threads. I didn't have a specific requirement to do that, but I was just wonder if doing that sort of stuff can cause problems. Marwan On Tue, Jun 29, 2010 at 9:38 AM, Graham Dumpleton < [email protected]> wrote: > On 29 June 2010 16:33, Marwan Al-Sabbagh <[email protected]> > wrote: > > thanks for the detailed response. How was PyCon? I saw your slides on > your > > blog. I'd like to go to PyCon one of these days. So I like what you've > > suggested as an approach for dealing with these scripts. In production I > > plan to give a decent timeout of around 30 seconds, I'm running 10 > seconds > > just to test out the settings. I plan to run the server as you suggested > in > > daemon mode multi-processed but single threaded so that i can safely have > > processes die that take too long to serve their requests. I added the > > setting for the inactivity-timeout=10 and ran the following script to do > the > > testing. > > > > import time, os > > > > def application(environ, start_response): > > start_response('200 OK', [('Content-Type', 'text/plain')]) > > yield 'pid: ' + str(os.getpid()) + '\n' > > time.sleep(30) > > yield 'done' > > > > The first time I ran it after restarting apache the process was killed at > > 10+5 seconds as expected. But then in subsequent requests the script ran > for > > 30 seconds to completion displaying done at the end. Any ideas what might > be > > going wrong? > > Bug in mod_wsgi. See: > > http://code.google.com/p/modwsgi/issues/detail?id=182 > > Fixed for mod_wsgi 4.0 in subversion. > > In short, for the first request after a reset the inactivity timeout > isn't working properly, instead will only actually timeout when the > dead lock timeout expires, even though not an actual dead lock. > > If this is a big issue, I should look at back porting to 3.X and > releasing new version. > > Graham > > > cheers, > > Marwan > > > > On Tue, Jun 29, 2010 at 5:49 AM, Graham Dumpleton > > <[email protected]> wrote: > >> > >> On 28 June 2010 02:19, marwan <[email protected]> wrote: > >> > Hi all, > >> > I want to put a limit on the execution time of page requests, to > >> > protect against some pages that might run large or inefficient > >> > queries. I've looked around the docs and the Configuration Directives > >> > but couldn't find out how to do it that way. I'm running the following > >> > stack: > >> > mod_wsgi 2.8 Apache/2.2.14 Ubuntu 10.04 Python 2.6.5 Linux > >> > > >> > my httpd.conf is as follows: > >> > ServerName 127.0.0.1 > >> > SetEnv no-gzip > >> > WSGIDaemonProcess myapp processes=1 threads=1 python-path=/var/www/ > >> > WSGIProcessGroup myapp > >> > WSGIScriptAlias /myapp /var/www/myapp.wsgi > >> > > >> > I have implemented the following solution, and it seems to work but I > >> > wanted to get peoples feedback on whether or not this is the best way > >> > to do this: > >> > > >> > this is the contents of /var/www/myapp.wsgi: > >> > > >> > import time, os, threading, signal, Queue, cgi > >> > > >> > class TimeoutMonitor(threading.Thread): > >> > def __init__(self, timeoutQ, timeout): > >> > threading.Thread.__init__(self) > >> > self.timeoutQ = timeoutQ > >> > self.timeout = timeout > >> > > >> > def run(self): > >> > try: > >> > self.timeoutQ.get(timeout=self.timeout) > >> > except Queue.Empty: > >> > os.kill(os.getpid(), signal.SIGKILL) > >> > > >> > def application(environ, start_response): > >> > parameters = cgi.parse_qs(environ.get('QUERY_STRING', '')) > >> > start_response('200 OK', [('Content-type', 'text/plain')]) > >> > timeoutQ = Queue.Queue() > >> > TimeoutMonitor(timeoutQ, timeout=10).start() > >> > > >> > for i in range(int(parameters['sleep'][0])): > >> > time.sleep(1) > >> > yield str(i) + '\n' > >> > yield 'done' > >> > timeoutQ.put('done') > >> > > >> > I have a few questions: > >> > - Is there another simpler or better way of doing this? maybe through > >> > some configuration parameter? > >> > - In general is it alright to spawn threads in modwsgi (in daemon > >> > mode)? > >> > - Is it better to send a SIGKILL, SIGTERM, or SIGINT signal? > >> > >> If the application or URL is delegated to a daemon process group where > >> each process has only a single thread, then you can use the > >> inactivity-timeout option to WSGIDaemonProcess. Ie., > >> > >> WSGIDaemonProcess myapp processes=1 threads=1 python-path=/var/www/ > >> inactivity-timeout=10 > >> WSGIProcessGroup myapp > >> WSGIScriptAlias /myapp /var/www/myapp.wsgi > >> > >> What this will do, given that only single thread, is that if a request > >> itself takes longer than 10 seconds to generate a response, a shutdown > >> and restart of the process will occur. In doing this there is a > >> default shutdown timeout of 5 seconds. That is, request is given an > >> additional 5 seconds to finish normally, but if not complete by then, > >> then process will be forcibly restarted. Thus, in reality leaving 15 > >> seconds for request to finish. You could adjust inactivity timeout > >> down to 5 seconds for overall time of 10, but you really want to be > >> careful about using such short timeouts. > >> > >> Also be aware that although the inactivity timeout applies to where no > >> request content reading and no response body yielding from active > >> requests for that timeout, if there are no requests which arrive at > >> all for that time after process has already handled at least one > >> request, process will also be shutdown and restarted. > >> > >> So, can be made to do what you want, but not necessarily a good thing > >> to do for a whole application. > >> > >> As such, what you may be better off doing is only delegating > >> problematic URLs to a daemon process with restrictive timeouts. Thus > >> you might have: > >> > >> WSGIDaemonProcess myapp processes=1 threads=15 python-path=/var/www/ > >> WSGIDaemonProcess myapp+timeout processes=2 threads=1 > >> python-path=/var/www/ inactivity-timeout=10 > >> WSGIProcessGroup myapp > >> WSGIScriptAlias /myapp /var/www/myapp.wsgi > >> <Location /myap/some/some/url> > >> WSGIProcessGroup myapp+timeout > >> </Location> > >> > >> That is, all requests normally go to multithread daemon process. If > >> however URL under sub URL of /myapp/some/url arrives, then that > >> instead will be delegated to and run in the daemon process group that > >> has more aggressive timeouts. This way you don't affect operation of > >> application as a whole. > >> > >> BTW, sorry for late reply. Still catching up on emails from past few > >> days when was at Conference. > >> > >> > >> Graham > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups > >> "modwsgi" group. > >> To post to this group, send email to [email protected]. > >> To unsubscribe from this group, send email to > >> [email protected]<modwsgi%[email protected]> > . > >> For more options, visit this group at > >> http://groups.google.com/group/modwsgi?hl=en. > >> > > > > -- > > You received this message because you are subscribed to the Google Groups > > "modwsgi" group. > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > > [email protected]<modwsgi%[email protected]> > . > > For more options, visit this group at > > http://groups.google.com/group/modwsgi?hl=en. > > > > -- > You received this message because you are subscribed to the Google Groups > "modwsgi" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]<modwsgi%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/modwsgi?hl=en. > > -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en.
