On Thu, Mar 12, 2009 at 9:51 AM, Mark Larsen <larsen...@gmail.com> wrote: > I took your advice and implemented using "Parallel Python". Can you comment > on your choice of pyprocessing? It seems like a simple re-write to switch > over.
I use pyprocessing partly because it gave me the impression that it was more tuned in to the general python community... but mostly because it's just what I happened to bump into first. I'm sure they both work fine; whatever works for you... (I've also used IPython's distributed computing stuff, and it was nice too.) The fact that pyprocessing just forks has sometimes come in handy -- in one case I had a giant multi-gigabyte structure that I needed to pull data out of, so I loaded it once to a global variable in the parent process before spawning the workers, and they all shared the data structure via COW memory without any extra work on my part. (This is an ugly Unix-specific hack, of course, but an effective one!) And I've sometimes had trouble when calling into rpy2 with memory usage growing unboundedly (still need to track this down), and in that case it's really handy that it's cheap and easy to just restart the slaves. -- Nathaniel ------------------------------------------------------------------------------ Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are powering Web 2.0 with engaging, cross-platform capabilities. Quickly and easily build your RIAs with Flex Builder, the Eclipse(TM)based development software that enables intelligent coding and step-through debugging. Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com _______________________________________________ rpy-list mailing list rpy-list@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/rpy-list
