Lawrence Oluyede wrote: > On Wed, Mar 4, 2009 at 11:56 PM, Kochhar <[email protected]> wrote: >> I am also leaning towards using processes instead of threads (using the >> multiprocessing module) but to play devil's advocate for moment why do you >> prefer processes to threads? > > The answer is kind of easy. I do not have computations which would > benefit more from a threading model than from a processing model. Our > Apache frontend uses multiprocesses and our async computations are > done in processes. We do not need to share anything and if we do, we > just copy the data to the process. With an API such as pyprocessing's > or subprocess is kind of easy. > > If we do have to go to the reason why I generally don't like threading > with shared state there's a great resource which I suggest to read > from cover to cover: > <http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-1.html> > > It's also useful if you do want to know about pro and cons of the > usual way threads are used. There's nothing bad intrinsically, it's > the common style of threading some environments taught us which is > bad. > > By the way you can combine the two techniques if you have to
I lean towards using processes and skipping shared memory entirely as well. However, there's no reason that multi-threaded applications can't use message-passing. This might not be the forum for a processes vs. threads discussion, though. >> Secondly, and this is pertinent when spawning >> processes, how do you hook into the pylons shutdown process to get the >> external >> process to stop? > > Not sure how to respond to this. Pylons is a framework, it can't be > turned on or shutted down. You can start and stop the web server and > how to sync this process with some "daemon processes" largely depend > on the operating system, and so on. Sorry, I tend to say pylons when I mean the paste http sever. I'm still green on the terminology. > If you have worker process usually those worker processes are created > on demand. If you need some kind of process pool the processes are > killed by the pool, and so on. > > Take a look at: <http://pyprocessing.berlios.de/> > > If you use Python 2.6 use the builtin module "multiprocessing", > otherwise you can use the backport of that: > <http://pypi.python.org/pypi/multiprocessing/> > > I'm not entirely sure of the compatibility issues between the standard > library version of the package and the original one. I've successfully used multiprocessing before in other apps, and I'm fine spawning worker processes inside of pylons. I just need to be able to shut them down when pylons^H^H^H^H^H^H the web server shuts down. I guess I can just write the process pids to the filesystem and use supervisord to send signals. Though, I would really like that logic to be a part of the pylons application. I was hoping to find the converse of config/environment.py:load_environment which I could hook into to unload worker processes. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "pylons-discuss" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~----------~----~----~----~------~----~------~--~---
