A Sunday 26 September 2010 19:24:30 Mario Juric escrigué: > Hi Francesc, > Thank you for the quick response (and the excellent code!). > > I thought the processes forked by multiprocessing.Process shared no > state, except explicitly through IPC (which is how they sidestep the > issues with GIL)? In principle those could even be on different > machines (using multiprocessing.Manager). So I'm not sure why > thread-safety is the issue.
You are right. Threading is not the issue. I was not thinking clearly. > Also, in the original testcase, even if I comment out the call to > _worker() from the parent process (so that pytables are used _only_ > from the child) the problem remains. > > Do either pytables or HDF5 do something internally that makes them > not work across a fork()? I've looked into this problem and tracked it down to the new multi- threading mode of Numexpr. It turns out that, when the multiprocess module creates subprocess, the threads are left in a stale state (I don't know exactly why yet). Anyway, I've opened a ticket for this: http://code.google.com/p/numexpr/issues/detail?id=33 and, until a more elegant solution is devised, I've implemented a workaround in: http://code.google.com/p/numexpr/source/detail?r=239 It seems to work for me. Please try it out and tell me if that works for your more complex setup too. -- Francesc Alted ------------------------------------------------------------------------------ Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev _______________________________________________ Pytables-users mailing list Pytables-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pytables-users
