Re: [Numpy-discussion] Binary releases
On Mon, Sep 16, 2013 at 7:10 AM, Ralf Gommers wrote: > On Mon, Sep 16, 2013 at 2:45 AM, wrote: >> >> On Sun, Sep 15, 2013 at 9:04 PM, Charles R Harris >> wrote: >> > Hi All, >> > >> > Numpy 1.8 is about ready for an rc1, which brings up the question of >> > which >> > binary builds so put up on sourceforge. For Windows maybe >> > >> > 32 bit windows, python 2.6, 2.7, 3.2, 3.3, compiled with MSVC >> > 64 bit windows, python 2.6, 2.7, 3.2, 3.3, compiled with MSVC, linked >> > with >> > MKL >> >> Are we not running into problems with scipy? >> scipy would need to use the same libraries, AFAIU (given Fortran and >> maybe C compatibilities) > > Indeed. If numpy goes MSVC + MKL, then scipy should go the same way. Some > other options to go to MinGW 4.x are being discussed on > https://github.com/scipy/scipy/issues/2829. Is it actually legal to distribute scipy linked with MKL? Scipy still includes GPL code (umfpack), and shipping MKL+GPL code integrated into a single download is extremely dicey... (this goes also for any downstream users who might package precompiled numpy/scipy with other packages). (In either case I think we ought to just bite the bullet and get MinGW 4.x running as a supported option, even if we don't use it for the official binaries and even if this requires some unaesthetic hacks. I bet we'd have more windows developers if there was an accessible way to build on windows...) -n ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Binary releases
On Mon, Sep 16, 2013 at 2:04 AM, Charles R Harris wrote: > Hi All, > > Numpy 1.8 is about ready for an rc1, which brings up the question of which > binary builds so put up on sourceforge. For Windows maybe > > 32 bit windows, python 2.6, 2.7, 3.2, 3.3, compiled with MSVC > 64 bit windows, python 2.6, 2.7, 3.2, 3.3, compiled with MSVC, linked with > MKL Another question is, can we start distributing wheels yet? That would make 'pip install' realistic on Windows, but I don't know how much trouble they are to build. -n ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Binary releases
On Mon, Sep 16, 2013 at 10:49 AM, Nathaniel Smith wrote: > On Mon, Sep 16, 2013 at 7:10 AM, Ralf Gommers > wrote: > > On Mon, Sep 16, 2013 at 2:45 AM, wrote: > >> > >> On Sun, Sep 15, 2013 at 9:04 PM, Charles R Harris > >> wrote: > >> > Hi All, > >> > > >> > Numpy 1.8 is about ready for an rc1, which brings up the question of > >> > which > >> > binary builds so put up on sourceforge. For Windows maybe > >> > > >> > 32 bit windows, python 2.6, 2.7, 3.2, 3.3, compiled with MSVC > >> > 64 bit windows, python 2.6, 2.7, 3.2, 3.3, compiled with MSVC, linked > >> > with > >> > MKL > >> > >> Are we not running into problems with scipy? > >> scipy would need to use the same libraries, AFAIU (given Fortran and > >> maybe C compatibilities) > > > > Indeed. If numpy goes MSVC + MKL, then scipy should go the same way. Some > > other options to go to MinGW 4.x are being discussed on > > https://github.com/scipy/scipy/issues/2829. > > Is it actually legal to distribute scipy linked with MKL? Scipy still > includes GPL code (umfpack), and shipping MKL+GPL code integrated into > a single download is extremely dicey... (this goes also for any > downstream users who might package precompiled numpy/scipy with other > packages). > Wait, we don't includes UMFPACK. We can optionally link against it, but that's not done for any of our binary (unless this was changed recently ?) > > (In either case I think we ought to just bite the bullet and get MinGW > 4.x running as a supported option, even if we don't use it for the > official binaries and even if this requires some unaesthetic hacks. I > bet we'd have more windows developers if there was an accessible way > to build on windows...) > Mingw 4 already works for compilation, no ? If not, that's definitely something to fix. The discussion around binary distribution should not preclude supporting it for people who want it. David > > -n > ___ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Binary releases
On Mon, Sep 16, 2013 at 10:56 AM, Nathaniel Smith wrote: > On Mon, Sep 16, 2013 at 2:04 AM, Charles R Harris > wrote: > > Hi All, > > > > Numpy 1.8 is about ready for an rc1, which brings up the question of > which > > binary builds so put up on sourceforge. For Windows maybe > > > > 32 bit windows, python 2.6, 2.7, 3.2, 3.3, compiled with MSVC > > 64 bit windows, python 2.6, 2.7, 3.2, 3.3, compiled with MSVC, linked > with > > MKL > > Another question is, can we start distributing wheels yet? That would > make 'pip install' realistic on Windows, but I don't know how much > trouble they are to build. > I made a proof of concept with Olivier Grisel from scikit learn at EuroScipy. I submitted a talk to pycon.fr to show how to automate the whole thing with vagrant/packer, and if the talk is accepted, I will work on making all the scripts available. David > > -n > ___ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Binary releases
On Sep 15, 2013, at 9:04 PM, Charles R Harris wrote: > Hi All, > > Numpy 1.8 is about ready for an rc1, which brings up the question of which > binary builds so put up on sourceforge. For Windows maybe [byte] > For Mac there is first the question of OS X versions, (10.5?), 10.6, > 10.7, 10.8. I don't know if some builds work on more than one OS X version. > The 10.5 version is a bit harder to come by than 10.6 and up. It looks like > 10.9 is coming up, but it isn't out yet. I have no idea what Python version > to match up these, but assuming all of them, then > • OS X 10.6 python 2.6, 2.7, 3.2, 3.3, compiled with native compiler, > linked with Accelerate. > • OS X 10.7 python 2.6, 2.7, 3.2, 3.3, compiled with native compiler, > linked with Accelerate. > • OS X 10.8 python 2.6, 2.7, 3.2, 3.3, compiled with native compiler, > linked with Accelerate. > That seems like a lot. It is fairly easy to compile from source on the mac > these days, are all those binary packages really needed? > > I don't know what I am doing with the binary stuff, so any suggestions are > welcome. If you will forgive an observation from a Mac user and (amateur) developer. I have twice tried to build Numpy from source and both times failed. The problem was that I couldn't find a single comprehensive set of directions that started from a virgin system (nothing but Apple's python and Xcode) and proceed to working copies of Numpy (and of course Matplotlib). Long time users know all about the differences between SourceForge, Github, and such. But bootstrapping pip, homebrew, macports, and similar was totally opaque to me. Sorry for the rant, but what I'm trying to say is that if there were such a recipe and it was clearly pointed to, then the need for a lengthy list of binaries would be pretty much moot. Thanks for listening, Bill ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Binary releases
On Mon, Sep 16, 2013 at 4:36 PM, Charles R Harris wrote: > New summary > >1. 32 bit windows, python 2.6, 2.7, 3.2, 3.3, compiled with MSVC >2. 64 bit windows, python 2.6, 2.7, 3.2, 3.3, compiled with MSVC, >linked with MKL > > These should be good for both windows 7 and window 8. > > Wait, when was it decided to move to MSVC for the official binaries ? Especially using ifort/MKL on windows means it will be difficult for other projects to produce packages on top of it. For Mac there is first the question of OS X versions, (10.5?), 10.6, 10.7, > 10.8. If 10.5 is omitted, packages built on 10.6 should be good for 10.7 > and 10.8, so > >1. OS X 10.6 python 2.6, 2.7, 3.2, 3.3, compiled with native >compiler, linked with Accelerate. > > The main question seems to be distribution and coordination with scipy. I > was thinking we would link in MKL statically, which I think should be OK. > Christoph does that and it should decouple Numpy from Scipy. It may not be > the most efficient way to do things, but it would work. My impression is > that if we wanted to distribute a dynamic library then every user would > need an MKL license to use it. > > It would be good to get this settled soon as we can't afford to futz > around with this forever waiting to release Numpy 1.8 and Scipy 0.13. > > Chuck > > > ___ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > > ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Unexpected casting result
An unexpected casting result was just reported on stackoverflow: http://stackoverflow.com/questions/18833639/attributeerror-in-python-numpy-when-constructing-function-for-certain-values The following show the essence of the issue: In [1]: np.__version__ Out[1]: '1.9.0.dev-6ce65d8' In [2]: type(np.array(1.) * (2**64-1)) Out[2]: numpy.float64 In [3]: type(np.array(1.) * (2**64)) Out[3]: float Note that the result of `np.array(1.0) * 2**64` is a Python float, not a numpy float64. Is this intentional? (As pointed out in the stackoverflow question, the issue https://github.com/numpy/numpy/issues/3409 is at least tangentially related.) Warren ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Binary releases
16.09.2013 16:31, William Ray Wing kirjoitti: [clip] > If you will forgive an observation from a Mac user and (amateur) developer. > I have twice tried to build Numpy from source and both times failed. > The problem was that I couldn't find a single comprehensive set of > directions that started from a virgin system (nothing but > Apple's python and Xcode) and proceed to working copies of > Numpy (and of course Matplotlib). The problem with OSX is that as a software development platform with open source tools, it is simply immature. You have homebrew, macports, fink, python.org binaries, Apple's own Python installation, proprietary BLAS/LAPACK with its own quirks, unified multiarch binaries, and whatnot in the mix. This makes everything more complicated to support, especially as the situation evolves with time and "recipes" that once worked stop working. -- Pauli Virtanen ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Binary releases
On Mon, Sep 16, 2013 at 12:12 PM, David Cournapeau wrote: > > > > On Mon, Sep 16, 2013 at 4:36 PM, Charles R Harris < > charlesr.har...@gmail.com> wrote: > >> New summary >> >>1. 32 bit windows, python 2.6, 2.7, 3.2, 3.3, compiled with MSVC >>2. 64 bit windows, python 2.6, 2.7, 3.2, 3.3, compiled with MSVC, >>linked with MKL >> >> These should be good for both windows 7 and window 8. >> >> > Wait, when was it decided to move to MSVC for the official binaries ? > Especially using ifort/MKL on windows means it will be difficult for other > projects to produce packages on top of it. > > It hasn't been decided, this is just a modified version of the initial post. Why not use MSVC? python.org does. What is the problem with statically linked MKL? Do other packages need to link to lapack? The windows build problem has hung around for years. I'm tired of it. > For Mac there is first the question of OS X versions, (10.5?), 10.6, 10.7, >> 10.8. If 10.5 is omitted, packages built on 10.6 should be good for 10.7 >> and 10.8, so >> >>1. OS X 10.6 python 2.6, 2.7, 3.2, 3.3, compiled with native >>compiler, linked with Accelerate. >> >> The main question seems to be distribution and coordination with scipy. I >> was thinking we would link in MKL statically, which I think should be OK. >> Christoph does that and it should decouple Numpy from Scipy. It may not be >> the most efficient way to do things, but it would work. My impression is >> that if we wanted to distribute a dynamic library then every user would >> need an MKL license to use it. >> >> It would be good to get this settled soon as we can't afford to futz >> around with this forever waiting to release Numpy 1.8 and Scipy 0.13. >> >> Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Binary releases
New summary 1. 32 bit windows, python 2.6, 2.7, 3.2, 3.3, compiled with MSVC 2. 64 bit windows, python 2.6, 2.7, 3.2, 3.3, compiled with MSVC, linked with MKL These should be good for both windows 7 and window 8. For Mac there is first the question of OS X versions, (10.5?), 10.6, 10.7, 10.8. If 10.5 is omitted, packages built on 10.6 should be good for 10.7 and 10.8, so 1. OS X 10.6 python 2.6, 2.7, 3.2, 3.3, compiled with native compiler, linked with Accelerate. The main question seems to be distribution and coordination with scipy. I was thinking we would link in MKL statically, which I think should be OK. Christoph does that and it should decouple Numpy from Scipy. It may not be the most efficient way to do things, but it would work. My impression is that if we wanted to distribute a dynamic library then every user would need an MKL license to use it. It would be good to get this settled soon as we can't afford to futz around with this forever waiting to release Numpy 1.8 and Scipy 0.13. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Binary releases
On Mon, Sep 16, 2013 at 4:36 PM, Charles R Harris wrote: > New summary > > 32 bit windows, python 2.6, 2.7, 3.2, 3.3, compiled with MSVC > 64 bit windows, python 2.6, 2.7, 3.2, 3.3, compiled with MSVC, linked with > MKL > > These should be good for both windows 7 and window 8. > > For Mac there is first the question of OS X versions, (10.5?), 10.6, 10.7, > 10.8. If 10.5 is omitted, packages built on 10.6 should be good for 10.7 and > 10.8, so > > OS X 10.6 python 2.6, 2.7, 3.2, 3.3, compiled with native compiler, linked > with Accelerate. > > The main question seems to be distribution and coordination with scipy. I > was thinking we would link in MKL statically, which I think should be OK. > Christoph does that and it should decouple Numpy from Scipy. It may not be > the most efficient way to do things, but it would work. My impression is > that if we wanted to distribute a dynamic library then every user would need > an MKL license to use it. > > It would be good to get this settled soon as we can't afford to futz around > with this forever waiting to release Numpy 1.8 and Scipy 0.13. Why not just release numpy 1.8 with the old and terrible system? As you know I'm 110% in favor of getting rid of it, but 1.8 is ready to go and 1.9 is coming soon enough, and the old and terrible system does work right now, today. None of the other options have this property. As you know, parallelization is the key to performance, and reducing serial data dependencies is the key to parallelization ;-). -n ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Binary releases
In article <8e95a257-3f06-43b7-8407-95916d284...@mac.com>, William Ray Wing wrote: > On Sep 15, 2013, at 9:04 PM, Charles R Harris > wrote: > > > Hi All, > > > > Numpy 1.8 is about ready for an rc1, which brings up the question of which > > binary builds so put up on sourceforge. For Windows maybe > > [byte] > > > For Mac there is first the question of OS X versions, (10.5?), 10.6, > > 10.7, 10.8. I don't know if some builds work on more than one OS X version. > > The 10.5 version is a bit harder to come by than 10.6 and up. It looks like > > 10.9 is coming up, but it isn't out yet. I have no idea what Python version > > to match up these, but assuming all of them, then > > OS X 10.6 python 2.6, 2.7, 3.2, 3.3, compiled with native compiler, > > linked with Accelerate. > > OS X 10.7 python 2.6, 2.7, 3.2, 3.3, compiled with native compiler, > > linked with Accelerate. > > OS X 10.8 python 2.6, 2.7, 3.2, 3.3, compiled with native compiler, > > linked with Accelerate. > > That seems like a lot. It is fairly easy to compile from source on the mac > > these days, are all those binary packages really needed? > > > > I don't know what I am doing with the binary stuff, so any suggestions are > > welcome. > > > If you will forgive an observation from a Mac user and (amateur) developer. > I have twice tried to build Numpy from source and both times failed. The > problem was that I couldn't find a single comprehensive set of directions > that started from a virgin system (nothing but Apple's python and Xcode) and > proceed to working copies of Numpy (and of course Matplotlib). > > Long time users know all about the differences between SourceForge, Github, > and such. But bootstrapping pip, homebrew, macports, and similar was totally > opaque to me. > > Sorry for the rant, but what I'm trying to say is that if there were such a > recipe and it was clearly pointed to, then the need for a lengthy list of > binaries would be pretty much moot. > > Thanks for listening, > Bill I sympathize. Unfortunately it changes all the time so it's hard to keep up to date. The usual suggestion is to either install a self-contained python distribution such as Anaconda, which contains all sorts of useful packages, or use the the binary installer (which requires python.org python). For the record: binary installers offer a very important additional benefit: the resulting package can be included in an application with some assurance about what versions of MacOS X it supports. If you build from source you probably have no idea what versions of MacOS X the package will support -- which is fine for personal use, but not for distribution. -- Russell ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Binary releases
On Mon, Sep 16, 2013 at 1:05 PM, Nathaniel Smith wrote: > On Mon, Sep 16, 2013 at 4:36 PM, Charles R Harris > wrote: > > New summary > > > > 32 bit windows, python 2.6, 2.7, 3.2, 3.3, compiled with MSVC > > 64 bit windows, python 2.6, 2.7, 3.2, 3.3, compiled with MSVC, linked > with > > MKL > > > > These should be good for both windows 7 and window 8. > > > > For Mac there is first the question of OS X versions, (10.5?), 10.6, > 10.7, > > 10.8. If 10.5 is omitted, packages built on 10.6 should be good for 10.7 > and > > 10.8, so > > > > OS X 10.6 python 2.6, 2.7, 3.2, 3.3, compiled with native compiler, > linked > > with Accelerate. > > > > The main question seems to be distribution and coordination with scipy. I > > was thinking we would link in MKL statically, which I think should be OK. > > Christoph does that and it should decouple Numpy from Scipy. It may not > be > > the most efficient way to do things, but it would work. My impression is > > that if we wanted to distribute a dynamic library then every user would > need > > an MKL license to use it. > > > > It would be good to get this settled soon as we can't afford to futz > around > > with this forever waiting to release Numpy 1.8 and Scipy 0.13. > > Why not just release numpy 1.8 with the old and terrible system? As > you know I'm 110% in favor of getting rid of it, but 1.8 is ready to > go and 1.9 is coming soon enough, and the old and terrible system does > work right now, today. None of the other options have this property. > > As you know, parallelization is the key to performance, and reducing > serial data dependencies is the key to parallelization ;-). > And necessity is the mother of invention ;) Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Binary releases
On Mon, Sep 16, 2013 at 8:05 PM, Nathaniel Smith wrote: > > Why not just release numpy 1.8 with the old and terrible system? As > you know I'm 110% in favor of getting rid of it, but 1.8 is ready to > go and 1.9 is coming soon enough, and the old and terrible system does > work right now, today. None of the other options have this property. On the down side, the "old and terrible system" does not cover providing pre-built binaries for 64 bit Windows. Doing that right is important not just for SciPy but for any other downstream package including C code compiled against the NumPy C API (and the people doing this probably will only have access to free compilers). Regards, Peter ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Binary releases
On Mon, Sep 16, 2013 at 12:31 PM, David Cournapeau wrote: > > > > On Mon, Sep 16, 2013 at 7:19 PM, Charles R Harris < > charlesr.har...@gmail.com> wrote: > >> >> >> >> On Mon, Sep 16, 2013 at 12:12 PM, David Cournapeau wrote: >> >>> >>> >>> >>> On Mon, Sep 16, 2013 at 4:36 PM, Charles R Harris < >>> charlesr.har...@gmail.com> wrote: >>> New summary 1. 32 bit windows, python 2.6, 2.7, 3.2, 3.3, compiled with MSVC 2. 64 bit windows, python 2.6, 2.7, 3.2, 3.3, compiled with MSVC, linked with MKL These should be good for both windows 7 and window 8. >>> Wait, when was it decided to move to MSVC for the official binaries ? >>> Especially using ifort/MKL on windows means it will be difficult for other >>> projects to produce packages on top of it. >>> >>> >> It hasn't been decided, this is just a modified version of the initial >> post. >> > > ok, sorry for the confusion > > >> Why not use MSVC? python.org does. What is the problem with statically >> linked MKL? Do other packages need to link to lapack? The windows build >> problem has hung around for years. I'm tired of it. >> > > Which build problem ? Being tired of it does not justify a decision in > particular, otherwise we fall into the fallacy "there is a problem, > therefore we must do something". There are multiple issues: > If it isn't a good reason, what is? ;) > > - moving away from gcc 3.x on 32 bits. Those compilers are old, but it > works well today. It is an untenable situation in the long term, though. I > will look again at building numpy/scipy with mingw-w64 in 32 bits to see if > the problems with -static-* are gone or not. > - 64 bits support: linked to first point. If the first point is solved, I > am relatively confident this one can too. > OK. What is the timeline? Days, weeks, months? - moving away from Atlas to MKL: this is much more problematic. First, MKL > is *huge*. For info, the MKL package we provide @ Enthought is 70 MB zip > compressed, and that's for the DLLs. The static version may be even bigger > I don't think static linkage would bring in the whole library, just the parts needed. Christolph's packages are < 10MB. Redistribution using the offered licenses is a potential problem that we need to get clarification on. No redistribution, no MKL. > - using ifort for fortran: by doing this, we impose on *every* package > downstream to use ifort as well (same for MKL BTW). > How does that work? > > There is also the issue of a blas/lapack for windows 64 bits. There the > situation has changed a lot since I last looked into those issues: cygwin > (required by atlas) now supports 64 bits natively, and openblas is > relatively well supported. I would certainly be happy to get rid of ATLAS > which is a PITA to maintain, and use openblas instead. > > Other packages *do* need to link into lapack (scikit learn, theano are > examples I am aware of). > So we need to distribute an LAPACK DLL? That sounds like a separate problem. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Binary releases
On Sep 16, 2013, at 1:54 PM, Pauli Virtanen wrote: > 16.09.2013 16:31, William Ray Wing kirjoitti: > [clip] >> If you will forgive an observation from a Mac user and (amateur) developer. >> I have twice tried to build Numpy from source and both times failed. >> The problem was that I couldn't find a single comprehensive set of >> directions that started from a virgin system (nothing but >> Apple's python and Xcode) and proceed to working copies of >> Numpy (and of course Matplotlib). > > The problem with OSX is that as a software development platform with > open source tools, it is simply immature. You have homebrew, macports, > fink, python.org binaries, Apple's own Python installation, proprietary > BLAS/LAPACK with its own quirks, unified multiarch binaries, and whatnot > in the mix. This makes everything more complicated to support, > especially as the situation evolves with time and "recipes" that once > worked stop working. > > -- > Pauli Virtanen > I think I could make a good case that maintaining a recipe is a lot less work than maintaining a binary distribution. Doing the second implicitly requires doing the first anyway. Thanks, Bill ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Binary releases
In article , Ralf Gommers wrote: > On Mon, Sep 16, 2013 at 2:45 AM, wrote: > > > On Sun, Sep 15, 2013 at 9:04 PM, Charles R Harris > > wrote: > > > Hi All, > > > > > > Numpy 1.8 is about ready for an rc1, which brings up the question of > > which > > > binary builds so put up on sourceforge. For Windows maybe > > >... > > > OS X 10.6 python 2.6, 2.7, 3.2, 3.3, compiled with native compiler, > > linked > > > with Accelerate. > > > OS X 10.7 python 2.6, 2.7, 3.2, 3.3, compiled with native compiler, > > linked > > > with Accelerate. > > > OS X 10.8 python 2.6, 2.7, 3.2, 3.3, compiled with native compiler, > > linked > > > with Accelerate. > > > > > > That seems like a lot. It is fairly easy to compile from source on the > > mac > > > these days, are all those binary packages really needed? > > > > That's not exactly the right list - the same installers built on 10.6 also > work on 10.7 and 10.8. I agree. I'll chime in and give my recommendations, though Ralf is the expert: For MacOS X I suggest building binary installers for python.org's python 2.7, 3.2 and 3.3 (the 64-bit versions). The result will run on 10.6 and later. It is safest to build these on MacOS X 10.6; it may work to build on a later MacOS X, but it sure doesn't for some packages. You will have to update to the latest bdist_mpkg to build Mac binary installers for python 3. I've not tried it yet. I don't think users expect a binary installer for Apple's python; I don't recall ever seeing these for numpy, scipy, matplotlib But if you do want to supply one, Apple provides Python 2.5, 2.6 and 2.7 but no 3.x (at least in MacOS X 10.8). -- Russell ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Binary releases
On Mon, Sep 16, 2013 at 7:19 PM, Charles R Harris wrote: > > > > On Mon, Sep 16, 2013 at 12:12 PM, David Cournapeau wrote: > >> >> >> >> On Mon, Sep 16, 2013 at 4:36 PM, Charles R Harris < >> charlesr.har...@gmail.com> wrote: >> >>> New summary >>> >>>1. 32 bit windows, python 2.6, 2.7, 3.2, 3.3, compiled with MSVC >>>2. 64 bit windows, python 2.6, 2.7, 3.2, 3.3, compiled with MSVC, >>>linked with MKL >>> >>> These should be good for both windows 7 and window 8. >>> >>> >> Wait, when was it decided to move to MSVC for the official binaries ? >> Especially using ifort/MKL on windows means it will be difficult for other >> projects to produce packages on top of it. >> >> > It hasn't been decided, this is just a modified version of the initial > post. > ok, sorry for the confusion > Why not use MSVC? python.org does. What is the problem with statically > linked MKL? Do other packages need to link to lapack? The windows build > problem has hung around for years. I'm tired of it. > Which build problem ? Being tired of it does not justify a decision in particular, otherwise we fall into the fallacy "there is a problem, therefore we must do something". There are multiple issues: - moving away from gcc 3.x on 32 bits. Those compilers are old, but it works well today. It is an untenable situation in the long term, though. I will look again at building numpy/scipy with mingw-w64 in 32 bits to see if the problems with -static-* are gone or not. - 64 bits support: linked to first point. If the first point is solved, I am relatively confident this one can too. - moving away from Atlas to MKL: this is much more problematic. First, MKL is *huge*. For info, the MKL package we provide @ Enthought is 70 MB zip compressed, and that's for the DLLs. The static version may be even bigger - using ifort for fortran: by doing this, we impose on *every* package downstream to use ifort as well (same for MKL BTW). There is also the issue of a blas/lapack for windows 64 bits. There the situation has changed a lot since I last looked into those issues: cygwin (required by atlas) now supports 64 bits natively, and openblas is relatively well supported. I would certainly be happy to get rid of ATLAS which is a PITA to maintain, and use openblas instead. Other packages *do* need to link into lapack (scikit learn, theano are examples I am aware of). David For Mac there is first the question of OS X versions, (10.5?), 10.6, 10.7, >>> 10.8. If 10.5 is omitted, packages built on 10.6 should be good for 10.7 >>> and 10.8, so >>> >>>1. OS X 10.6 python 2.6, 2.7, 3.2, 3.3, compiled with native >>>compiler, linked with Accelerate. >>> >>> The main question seems to be distribution and coordination with scipy. >>> I was thinking we would link in MKL statically, which I think should be OK. >>> Christoph does that and it should decouple Numpy from Scipy. It may not be >>> the most efficient way to do things, but it would work. My impression is >>> that if we wanted to distribute a dynamic library then every user would >>> need an MKL license to use it. >>> >>> It would be good to get this settled soon as we can't afford to futz >>> around with this forever waiting to release Numpy 1.8 and Scipy 0.13. >>> >>> > Chuck > > > ___ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > > ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Unexpected casting result
On Mon, Sep 16, 2013 at 1:54 PM, Warren Weckesser < warren.weckes...@gmail.com> wrote: > An unexpected casting result was just reported on stackoverflow: > > http://stackoverflow.com/questions/18833639/attributeerror-in-python-numpy-when-constructing-function-for-certain-values > > The following show the essence of the issue: > > In [1]: np.__version__ > Out[1]: '1.9.0.dev-6ce65d8' > > In [2]: type(np.array(1.) * (2**64-1)) > Out[2]: numpy.float64 > > In [3]: type(np.array(1.) * (2**64)) > Out[3]: float > > Note that the result of `np.array(1.0) * 2**64` is a Python float, not a > numpy float64. Is this intentional? > > (As pointed out in the stackoverflow question, the issue > https://github.com/numpy/numpy/issues/3409 is at least tangentially > related.) > > Warren > > The original poster of the stackoverflow question has reported the issue on github: https://github.com/numpy/numpy/issues/3756 Warren ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Binary releases
On Mon, Sep 16, 2013 at 8:15 PM, Peter Cock wrote: > On Mon, Sep 16, 2013 at 8:05 PM, Nathaniel Smith wrote: >> >> Why not just release numpy 1.8 with the old and terrible system? As >> you know I'm 110% in favor of getting rid of it, but 1.8 is ready to >> go and 1.9 is coming soon enough, and the old and terrible system does >> work right now, today. None of the other options have this property. > > On the down side, the "old and terrible system" does not > cover providing pre-built binaries for 64 bit Windows. > > Doing that right is important not just for SciPy but for any > other downstream package including C code compiled > against the NumPy C API (and the people doing this > probably will only have access to free compilers). That's not a downside -- that's the situation right now and will continue to be the situation for the immediate future, if we cut a 1.8rc1 tomorrow and also if we don't cut a 1.8rc1 tomorrow. Again, I'm absolutely behind getting this sorted out, but holding up the release on all platforms is not going to make win64 standalone binaries appear any faster, and in the mean time everyone seems to be getting along OK, either because they're using a distribution, are on another platform, or taking advantage of Cristoph's generosity (thank you Cristoph!). Worst case, if it all gets sorted out next week we could release an 1.8.1 to celebrate... -n ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Binary releases
On Mon, Sep 16, 2013 at 7:01 PM, Russell E. Owen wrote: > In article > , > Ralf Gommers wrote: > > > On Mon, Sep 16, 2013 at 2:45 AM, wrote: > > > > > On Sun, Sep 15, 2013 at 9:04 PM, Charles R Harris > > > wrote: > > > > Hi All, > > > > > > > > Numpy 1.8 is about ready for an rc1, which brings up the question of > > > which > > > > binary builds so put up on sourceforge. For Windows maybe > > > >... > > > > OS X 10.6 python 2.6, 2.7, 3.2, 3.3, compiled with native compiler, > > > linked > > > > with Accelerate. > > > > OS X 10.7 python 2.6, 2.7, 3.2, 3.3, compiled with native compiler, > > > linked > > > > with Accelerate. > > > > OS X 10.8 python 2.6, 2.7, 3.2, 3.3, compiled with native compiler, > > > linked > > > > with Accelerate. > > > > > > > > That seems like a lot. It is fairly easy to compile from source on > the > > > mac > > > > these days, are all those binary packages really needed? > > > > > > > That's not exactly the right list - the same installers built on 10.6 > also > > work on 10.7 and 10.8. > > I agree. I'll chime in and give my recommendations, though Ralf is the > expert: > David is our resident build/distribute guru actually. > For MacOS X I suggest building binary installers for python.org's python > 2.7, 3.2 and 3.3 (the 64-bit versions). The result will run on 10.6 and > later. It is safest to build these on MacOS X 10.6; it may work to build > on a later MacOS X, but it sure doesn't for some packages. > Agreed. I think we discussed before not providing OS X 10.5 and python 2.6 binaries, that would make sense imho. > > You will have to update to the latest bdist_mpkg to build Mac binary > installers for python 3. I've not tried it yet. > I just tried, it's now possible to build 3.x binaries with a simple ``paver dmg -p 3.3``. > > I don't think users expect a binary installer for Apple's python; I > don't recall ever seeing these for numpy, scipy, matplotlib But if > you do want to supply one, We don't I think. Just python.org, keeps it simple. Cheers, Ralf > Apple provides Python 2.5, 2.6 and 2.7 but no > 3.x (at least in MacOS X 10.8). > > -- Russell > > ___ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Binary releases
On Mon, Sep 16, 2013 at 2:56 AM, Nathaniel Smith wrote: > Another question is, can we start distributing wheels yet? > yes, yes, yes -- though maybe not for the beta testing. wheels will never catch on , or even get debugged if none of us use them. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Binary releases
On Mon, Sep 16, 2013 at 6:31 AM, William Ray Wing wrote: > If you will forgive an observation from a Mac user and (amateur) > developer. I have twice tried to build Numpy from source and both times > failed. The problem was that I couldn't find a single comprehensive set of > directions that started from a virgin system (nothing but Apple's python > and Xcode) and proceed to working copies of Numpy > This is odd -- it could build out of the box with plain old: setup.py install or, for that matter. pip install. > (and of course Matplotlib). > > MPL is a whole different kettle of fish -- I know most folks need more of the "stack" than numpy, but are you sure you had trouble building numpy? Sorry for the rant, but what I'm trying to say is that if there were such a > recipe and it was clearly pointed to, then the need for a lengthy list of > binaries would be pretty much moot. > Actually, numpy should be easy, but MPL. scipy, etc, -- there is no such think as a single easy recipe -- too many versions of python, OS-X, systems for the dependencies, etc. sad, but true. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Binary releases
On Mon, Sep 16, 2013 at 12:15 PM, Peter Cock wrote: > Doing that right is important not just for SciPy but for any > other downstream package including C code compiled > against the NumPy C API (and the people doing this > probably will only have access to free compilers). > > well, "free as in beer" anyway -- I"ve always used the MS "express" compilers for my work. And I'd like to continue. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Binary releases
On Mon, Sep 16, 2013 at 12:31 PM, David Cournapeau wrote: > > > > On Mon, Sep 16, 2013 at 7:19 PM, Charles R Harris < > charlesr.har...@gmail.com> wrote: > >> >> >> >> On Mon, Sep 16, 2013 at 12:12 PM, David Cournapeau wrote: >> >>> >>> >>> >>> On Mon, Sep 16, 2013 at 4:36 PM, Charles R Harris < >>> charlesr.har...@gmail.com> wrote: >>> New summary 1. 32 bit windows, python 2.6, 2.7, 3.2, 3.3, compiled with MSVC 2. 64 bit windows, python 2.6, 2.7, 3.2, 3.3, compiled with MSVC, linked with MKL These should be good for both windows 7 and window 8. >>> Wait, when was it decided to move to MSVC for the official binaries ? >>> Especially using ifort/MKL on windows means it will be difficult for other >>> projects to produce packages on top of it. >>> >>> >> It hasn't been decided, this is just a modified version of the initial >> post. >> > > ok, sorry for the confusion > > >> Why not use MSVC? python.org does. What is the problem with statically >> linked MKL? Do other packages need to link to lapack? The windows build >> problem has hung around for years. I'm tired of it. >> > > Which build problem ? Being tired of it does not justify a decision in > particular, otherwise we fall into the fallacy "there is a problem, > therefore we must do something". There are multiple issues: > > - moving away from gcc 3.x on 32 bits. Those compilers are old, but it > works well today. It is an untenable situation in the long term, though. I > will look again at building numpy/scipy with mingw-w64 in 32 bits to see if > the problems with -static-* are gone or not. > - 64 bits support: linked to first point. If the first point is solved, I > am relatively confident this one can too. > - moving away from Atlas to MKL: this is much more problematic. First, > MKL is *huge*. For info, the MKL package we provide @ Enthought is 70 MB > zip compressed, and that's for the DLLs. The static version may be even > bigger > - using ifort for fortran: by doing this, we impose on *every* package > downstream to use ifort as well (same for MKL BTW). > > There is also the issue of a blas/lapack for windows 64 bits. There the > situation has changed a lot since I last looked into those issues: cygwin > (required by atlas) now supports 64 bits natively, and openblas is > relatively well supported. I would certainly be happy to get rid of ATLAS > which is a PITA to maintain, and use openblas instead. > > Openblas includes some of lapack and is available for x86_64 on windows. It isn't clear what is included nor what compiler was used. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
