Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-07-11 Thread Dr. Michael Lehn
Am 29.04.2014 um 02:01 schrieb Nathaniel Smith n...@pobox.com: On Tue, Apr 29, 2014 at 12:52 AM, Sturla Molden sturla.mol...@gmail.com wrote: On 29/04/14 01:30, Nathaniel Smith wrote: I finally read this paper: http://www.cs.utexas.edu/users/flame/pubs/blis2_toms_rev2.pdf and I

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-05-12 Thread Olivier Grisel
BLIS looks interesting. Besides threading and runtime configuration, adding support for building it as a shared library would also be required to be usable by python packages that have several extension modules that link against a BLAS implementation.

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-05-12 Thread Matthieu Brucher
Yes, they seem to be focused on HPC clusters with sometimes old rules (as no shared library). Also, they don't use a potable Makefile generator, not even autoconf, this may also play a role in Windows support. 2014-05-12 12:52 GMT+01:00 Olivier Grisel olivier.gri...@ensta.org: BLIS looks

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-05-12 Thread Carl Kleffner
Neither the numpy ATLAS build nor the MKL build on Windows makes use of shared libs. The latter due due licence restriction. Carl 2014-05-12 14:23 GMT+02:00 Matthieu Brucher matthieu.bruc...@gmail.com: Yes, they seem to be focused on HPC clusters with sometimes old rules (as no shared

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-05-12 Thread Matthieu Brucher
There is the issue of installing the shared library at the proper location as well IIRC? 2014-05-12 13:54 GMT+01:00 Carl Kleffner cmkleff...@gmail.com: Neither the numpy ATLAS build nor the MKL build on Windows makes use of shared libs. The latter due due licence restriction. Carl

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-05-12 Thread Matthew Brett
Hi, On Mon, May 12, 2014 at 6:01 AM, Matthieu Brucher matthieu.bruc...@gmail.com wrote: There is the issue of installing the shared library at the proper location as well IIRC? As Carl implies, the standard numpy installers do static linking to the BLAS lib, so we haven't (as far as I know)

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-05-12 Thread Carl Kleffner
2014-05-12 19:25 GMT+02:00 Matthew Brett matthew.br...@gmail.com: Hi, On Mon, May 12, 2014 at 6:01 AM, Matthieu Brucher matthieu.bruc...@gmail.com wrote: There is the issue of installing the shared library at the proper location as well IIRC? As Carl implies, the standard numpy

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-28 Thread Michael Lehn
Am 11 Apr 2014 um 19:05 schrieb Sturla Molden sturla.mol...@gmail.com: Sturla Molden sturla.mol...@gmail.com wrote: Making a totally new BLAS might seem like a crazy idea, but it might be the best solution in the long run. To see if this can be done, I'll try to re-implement cblas_dgemm

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-28 Thread Nathaniel Smith
On Mon, Apr 28, 2014 at 11:25 AM, Michael Lehn michael.l...@uni-ulm.de wrote: Am 11 Apr 2014 um 19:05 schrieb Sturla Molden sturla.mol...@gmail.com: Sturla Molden sturla.mol...@gmail.com wrote: Making a totally new BLAS might seem like a crazy idea, but it might be the best solution in the

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-28 Thread Sturla Molden
On 29/04/14 01:30, Nathaniel Smith wrote: I finally read this paper: http://www.cs.utexas.edu/users/flame/pubs/blis2_toms_rev2.pdf and I have to say that I'm no longer so convinced that OpenBLAS is the right starting point. I think OpenBLAS in the long run is doomed as an OSS project.

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-28 Thread Nathaniel Smith
On Tue, Apr 29, 2014 at 12:52 AM, Sturla Molden sturla.mol...@gmail.com wrote: On 29/04/14 01:30, Nathaniel Smith wrote: I finally read this paper: http://www.cs.utexas.edu/users/flame/pubs/blis2_toms_rev2.pdf and I have to say that I'm no longer so convinced that OpenBLAS is the right

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-28 Thread Matthew Brett
Hi, On Mon, Apr 28, 2014 at 4:30 PM, Nathaniel Smith n...@pobox.com wrote: On Mon, Apr 28, 2014 at 11:25 AM, Michael Lehn michael.l...@uni-ulm.de wrote: Am 11 Apr 2014 um 19:05 schrieb Sturla Molden sturla.mol...@gmail.com: Sturla Molden sturla.mol...@gmail.com wrote: Making a totally

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-28 Thread Julian Taylor
On 29.04.2014 02:05, Matthew Brett wrote: Hi, On Mon, Apr 28, 2014 at 4:30 PM, Nathaniel Smith n...@pobox.com wrote: On Mon, Apr 28, 2014 at 11:25 AM, Michael Lehn michael.l...@uni-ulm.de wrote: Am 11 Apr 2014 um 19:05 schrieb Sturla Molden sturla.mol...@gmail.com: Sturla Molden

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-28 Thread Nathaniel Smith
On Tue, Apr 29, 2014 at 1:10 AM, Julian Taylor jtaylor.deb...@googlemail.com wrote: On 29.04.2014 02:05, Matthew Brett wrote: Hi, On Mon, Apr 28, 2014 at 4:30 PM, Nathaniel Smith n...@pobox.com wrote: It would be really interesting if someone were to try hacking simple runtime CPU detection

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-28 Thread Matthew Brett
Hi, On Mon, Apr 28, 2014 at 5:10 PM, Julian Taylor jtaylor.deb...@googlemail.com wrote: On 29.04.2014 02:05, Matthew Brett wrote: Hi, On Mon, Apr 28, 2014 at 4:30 PM, Nathaniel Smith n...@pobox.com wrote: On Mon, Apr 28, 2014 at 11:25 AM, Michael Lehn michael.l...@uni-ulm.de wrote: Am 11

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-28 Thread Nathaniel Smith
On Tue, Apr 29, 2014 at 1:05 AM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Apr 28, 2014 at 4:30 PM, Nathaniel Smith n...@pobox.com wrote: On Mon, Apr 28, 2014 at 11:25 AM, Michael Lehn michael.l...@uni-ulm.de wrote: Am 11 Apr 2014 um 19:05 schrieb Sturla Molden

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-28 Thread Matthew Brett
Hi, On Mon, Apr 28, 2014 at 5:50 PM, Nathaniel Smith n...@pobox.com wrote: On Tue, Apr 29, 2014 at 1:05 AM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Apr 28, 2014 at 4:30 PM, Nathaniel Smith n...@pobox.com wrote: On Mon, Apr 28, 2014 at 11:25 AM, Michael Lehn

Re: [Numpy-discussion] The BLAS problem

2014-04-12 Thread Dinesh Vadhia
Agree that OpenBLAS is the most favorable route instead of starting from scratch. Btw, why is sparse BLAS not included as I've always been under the impression that scipy sparse supports BLAS - no? ___ NumPy-Discussion mailing list

[Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-11 Thread Nathaniel Smith
On Fri, Apr 11, 2014 at 12:21 PM, Carl Kleffner cmkleff...@gmail.com wrote: a discussion about OpenBLAS on the octave maintainer list: http://article.gmane.org/gmane.comp.gnu.octave.maintainers/38746 I'm getting the impression that OpenBLAS is being both a tantalizing opportunity and a

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-11 Thread Sturla Molden
Nathaniel Smith n...@pobox.com wrote: I unfortunately don't have the skills to actually lead such an effort (I've never written a line of asm in my life...), but surely our collective communities have people who do? The assembly part in OpenBLAS/GotoBLAS is the major problem. Not just that

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-11 Thread Sturla Molden
Sturla Molden sturla.mol...@gmail.com wrote: Making a totally new BLAS might seem like a crazy idea, but it might be the best solution in the long run. To see if this can be done, I'll try to re-implement cblas_dgemm and then benchmark against MKL, Accelerate and OpenBLAS. If I can get the

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-11 Thread Julian Taylor
On 11.04.2014 18:03, Nathaniel Smith wrote: On Fri, Apr 11, 2014 at 12:21 PM, Carl Kleffner cmkleff...@gmail.com wrote: a discussion about OpenBLAS on the octave maintainer list: http://article.gmane.org/gmane.comp.gnu.octave.maintainers/38746 I'm getting the impression that OpenBLAS is

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-11 Thread Matthew Brett
Hi, On Fri, Apr 11, 2014 at 9:03 AM, Nathaniel Smith n...@pobox.com wrote: On Fri, Apr 11, 2014 at 12:21 PM, Carl Kleffner cmkleff...@gmail.com wrote: a discussion about OpenBLAS on the octave maintainer list: http://article.gmane.org/gmane.comp.gnu.octave.maintainers/38746 I'm getting the

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-11 Thread Julian Taylor
On 11.04.2014 19:05, Sturla Molden wrote: Sturla Molden sturla.mol...@gmail.com wrote: Making a totally new BLAS might seem like a crazy idea, but it might be the best solution in the long run. To see if this can be done, I'll try to re-implement cblas_dgemm and then benchmark against

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-11 Thread Nathaniel Smith
On Fri, Apr 11, 2014 at 6:05 PM, Sturla Molden sturla.mol...@gmail.com wrote: Sturla Molden sturla.mol...@gmail.com wrote: Making a totally new BLAS might seem like a crazy idea, but it might be the best solution in the long run. To see if this can be done, I'll try to re-implement

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-11 Thread Julian Taylor
On 11.04.2014 18:03, Nathaniel Smith wrote: On Fri, Apr 11, 2014 at 12:21 PM, Carl Kleffner cmkleff...@gmail.com wrote: a discussion about OpenBLAS on the octave maintainer list: http://article.gmane.org/gmane.comp.gnu.octave.maintainers/38746 I'm getting the impression that OpenBLAS is

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-11 Thread Nathaniel Smith
On Fri, Apr 11, 2014 at 7:53 PM, Julian Taylor jtaylor.deb...@googlemail.com wrote: On 11.04.2014 18:03, Nathaniel Smith wrote: On Fri, Apr 11, 2014 at 12:21 PM, Carl Kleffner cmkleff...@gmail.com wrote: a discussion about OpenBLAS on the octave maintainer list:

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-11 Thread Matthew Brett
Hi, On Fri, Apr 11, 2014 at 10:05 AM, Sturla Molden sturla.mol...@gmail.com wrote: Sturla Molden sturla.mol...@gmail.com wrote: Making a totally new BLAS might seem like a crazy idea, but it might be the best solution in the long run. To see if this can be done, I'll try to re-implement

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-11 Thread Sturla Molden
On 11/04/14 23:11, Matthew Brett wrote: Are you sure that you can redistribute object code statically linked against icc runtimes? I am not a lawyer... ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-11 Thread Matthew Brett
On Fri, Apr 11, 2014 at 2:58 PM, Sturla Molden sturla.mol...@gmail.com wrote: On 11/04/14 23:11, Matthew Brett wrote: Are you sure that you can redistribute object code statically linked against icc runtimes? I am not a lawyer... No - sure - but it would be frustrating if you found yourself

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-11 Thread Sturla Molden
On 12/04/14 00:01, Matthew Brett wrote: No - sure - but it would be frustrating if you found yourself optimizing with a compiler that is useless for subsequent open-source builds. No, I think MSVC or gcc 4.8/4.9 will work too. It's just that I happen to have icc and clang on this computer :)

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-11 Thread Sturla Molden
On 11/04/14 20:47, Nathaniel Smith wrote: Also, while Windows is maybe in the worst shape, all platforms would seriously benefit from the existence of a reliable speed-competitive binary-distribution-compatible BLAS that doesn't break fork(). Windows is worst off, yes. I don't think fork

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-11 Thread Nathaniel Smith
On Fri, Apr 11, 2014 at 11:26 PM, Sturla Molden sturla.mol...@gmail.com wrote: On 11/04/14 20:47, Nathaniel Smith wrote: Also, while Windows is maybe in the worst shape, all platforms would seriously benefit from the existence of a reliable speed-competitive binary-distribution-compatible

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-11 Thread Sturla Molden
On 12/04/14 00:39, Nathaniel Smith wrote: The spawn mode is fine and all, but (a) the presence of something in 3.4 helps only a minority of users, (b) spawn is not a full replacement for fork; It basically does the same as on Windows. If you want portability to Windows, you must abide by

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-11 Thread Nathaniel Smith
On Sat, Apr 12, 2014 at 12:07 AM, Sturla Molden sturla.mol...@gmail.com wrote: On 12/04/14 00:39, Nathaniel Smith wrote: The spawn mode is fine and all, but (a) the presence of something in 3.4 helps only a minority of users, (b) spawn is not a full replacement for fork; It basically does

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-11 Thread Nathaniel Smith
On Fri, Apr 11, 2014 at 7:29 PM, Julian Taylor jtaylor.deb...@googlemail.com wrote: x86 cpus are backward compatible with almost all instructions they ever introduced, so one machine with the latest instruction set supported is sufficient to test almost everything. For that the runtime kernel

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-11 Thread Sturla Molden
On 12/04/14 01:07, Sturla Molden wrote: ATM the only other way to work with a data set that's larger than memory-divided-by-numcpus is to explicitly set up shared memory, and this is *really* hard for anything more complicated than a single flat array. Not difficult. You just go to my

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-11 Thread Nathaniel Smith
Okay, I started taking notes here: https://github.com/numpy/numpy/wiki/BLAS-desiderata Please add as appropriate... -n On Sat, Apr 12, 2014 at 12:19 AM, Nathaniel Smith n...@pobox.com wrote: On Fri, Apr 11, 2014 at 7:29 PM, Julian Taylor jtaylor.deb...@googlemail.com wrote: x86 cpus are