Hi David, Apple's response here is somewhat confusing, but I will add that on the supercomputing side of things we rarely fork, as this is not well-supported from the vendors or the hardware (it's hard enough to performantly spawn 500,000 processes statically, doing this dynamically becomes even more challenging). This sounds like an issue in Python multiprocessing itself, as I guess many other Apple libraries will fail or crash with the fork-no-exec model.
My suggestion would be that numpy continue to integrate with Accelerate but prefer a macports or brew supplied blas, if available. This should probably also be filed as a wont-fix bug on the tracker so anybody who hits the same problem knows that it's on the system side and not us. A On Sat, Aug 4, 2012 at 2:04 PM, David Cournapeau <courn...@gmail.com> wrote: > Hi, > > During last PyCon, Olivier Grisel (from scikits-learn fame) and myself > looked into a nasty bug on mac os x: https://gist.github.com/2027412. > The short story is that I believe this means numpy cannot be used with > multiprocessing if linked against accelerate framework, and as such we > should think about giving up on accelerate, and use e.g. ATLAS on mac > for our official binaries. > > Long story: we recently received a answer where the engineers mention > that using blas on each 'side' of a fork is not supported. The meat of > the email is attached below > > thoughts ? > > David > > ---------- Forwarded message ---------- > From: <devb...@apple.com> > Date: 2012/8/2 > Subject: Bug ID 11036478: Segfault when calling dgemm with Accelerate > / GCD after in a forked process > To: olivier.gri...@gmail.com > > > Hi Olivier, > > Thank you for contacting us regarding Bug ID# 11036478. > > Thank you for filing this bug report. > > This usage of fork() is not supported on our platform. > > For API outside of POSIX, including GCD and technologies like > Accelerate, we do not support usage on both sides of a fork(). For > this reason among others, use of fork() without exec is discouraged in > general in processes that use layers above POSIX. > > We recommend that you either restrict usage of blas to the parent or > the child process but not both, or that you switch to using GCD or > pthreads rather than forking to create parallelism. > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion >
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
