2009/2/18 Tom Davis <[email protected]>: > >> It is a bit antisocial of library type component modules to think they >> control the process execution environment and register signals. :-( >> >> Was it only for signal 15? Is that SIGTERM on your operating system? >> >> Are you using embedded mode or daemon mode? There are currently issues >> with creating sub processes from daemon mode processes where sub >> processes need to handle signals. > > True, it is a bit anti-social, but Twisted is a controlling beast ;) > > Other signals I noticed were "2" and "20". I use embedded mode, > though I tried out daemon mode just for kicks and it did not garner > different results. > >> Any particular reason you are using Twisted just to do HTTP requests >> rather than simpler Python HTTP client modules. > > The functionality this method provides is a sort of "live update" via > numerous third-party web APIs. To make these requests synchronously > would take far more time since the requests need to be made, parsed, > then inserted into a database. Meanwhile, users are waiting for the > results of an "AJAX" call to view the requested data. > > I could make these calls asynchronously outside of the main thread, > perhaps on another machine not tied to the oddities of mod_wsgi and > apache, but then I wouldn't know *when* they completed. I could call > them using subprocess.call as well and wait for the results, but this > would involve importing all the necessary modules each request which > adds precious time to the transaction as well. These are the reasons I > have chosen to do it asynchronously using Twisted running in a > separate process.
Have you considered running Twisted process by itself with XML-RPC service enabled so could call into it using xmlrpclib from web application processes under mod_wsgi. The XML-RPC calling interface on client side is really quite simple. Not sure about the Twisted side though. The XML-RPC interface also gives you a separate admin interface as well if you need to control it from management scripts. Graham > Thank you for the very quick replies and your continued efforts! > > On Feb 17, 11:03 pm, Graham Dumpleton <[email protected]> > wrote: >> 2009/2/18 Tom Davis <[email protected]>: >> >> >> >> > Graham, >> >> > I did as you recommended and got the following output in error_log: >> >> > subprocess pid 30467 >> > hello grumpy >> >> > However, following that were various warnings about signal >> > registration being ignored, as it attempted to do the rest of the >> > function (I tacked your code onto the beginning): >> >> > [Tue Feb 17 21:48:32 2009] [warn] mod_wsgi (pid=30467): Callback >> > registration for signal 15 ignored. >> >> It is a bit antisocial of library type component modules to think they >> control the process execution environment and register signals. :-( >> >> Was it only for signal 15? Is that SIGTERM on your operating system? >> >> Are you using embedded mode or daemon mode? There are currently issues >> with creating sub processes from daemon mode processes where sub >> processes need to handle signals. >> >> > So, it would appear that the child is being spawned/run, but only in >> > the simplest sense. The mod_wsgi rules seem to make it ignore >> > signals. If I add back WSGIRestrictSignal Off, the warnings go away, >> > but the function still doesn't *do* anything. The purpose of the >> > target function is to make some http requests (via Twisted) then parse >> > those requests, which involves running the Twisted reactor (the entire >> > reason I am spawning a new process; the reactor cannot run within a >> > mod_wsgi thread). It seems this is causing a problem for some reason. >> > Obviously the target function is being run, as evidenced by the output >> > in error_log, but Twisted is having trouble with signaling within it >> > (at least, that is my only guess). As I said, this only seems to be >> > affected when running w/ mod_wsgi; any other time the function works >> > fine. >> >> Any particular reason you are using Twisted just to do HTTP requests >> rather than simpler Python HTTP client modules. >> >> Anyway, let me have a think about it for a while. >> >> Graham >> >> > On Feb 17, 10:03 pm, Graham Dumpleton <[email protected]> >> > wrote: >> >> 2009/2/18 Tom Davis <[email protected]>: >> >> >> > I am trying to use multiprocessing within a modwsgi app and am having >> >> > an issue. At first I was getting errors regarding permissions to use >> >> > stdout, etc. I fixed these by setting WSGIRestrictStdout, >> >> > WSGIRestrictStdin, and WSGIRestrictSignal to Off in my conf file. Now >> >> > I get no errors, but the process doesn't actually run. My usage is >> >> > quite simple: on a page request, I try something along the lines of: >> >> >> > p = multiprocessing.Process(target=fn, args...) >> >> > p.start() >> >> > p.join() >> >> >> > The target doesn't output anything, but if run properly it should add >> >> > some records to the database. I tested that it works normally by >> >> > running the app under the django development server, which doesn't >> >> > seem to have an issue with multiprocessing. Has anyone encountered >> >> > this before? >> >> >> What does the target function do? >> >> >> Try following WSGI script file. Note that for this script you will >> >> need to look in main Apache error log for output, even if WSGI script >> >> defined in VirtualHost context and that VirtualHost has its own error >> >> log. This is because am reseting stdin/stdout/stderr back to original >> >> values. Take out the WSGIRestrictStdin, WSGIRestrictStdout and >> >> WSGIRestrictSignal directives you already added. >> >> >> from multiprocessing import Process >> >> >> import sys, os >> >> >> sys.stdin = sys.__stdin__ >> >> sys.stdout = sys.__stdout__ >> >> sys.stderr = sys.__stderr__ >> >> >> def f(name): >> >> sys.stderr.write('subprocess pid %d\n' % os.getpid()) >> >> sys.stderr.write('hello %s\n' % name) >> >> sys.stderr.flush() >> >> >> def application(environ, start_response): >> >> status = '200 OK' >> >> output = 'Hello World!' >> >> >> sys.stderr.write('parent pid %d\n' % os.getpid()) >> >> sys.stderr.flush() >> >> >> p = Process(target=f, args=('grumpy',)) >> >> p.start() >> >> p.join() >> >> >> response_headers = [('Content-type', 'text/plain'), >> >> ('Content-Length', str(len(output)))] >> >> start_response(status, response_headers) >> >> >> return [output] > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "modwsgi" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en -~----------~----~----~----~------~----~------~--~---
