Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
On Thu, Feb 7, 2013 at 6:21 AM, Ondřej Čertík ondrej.cer...@gmail.com wrote: On Wed, Feb 6, 2013 at 9:20 PM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: On 02/07/2013 12:16 AM, Matthew Brett wrote: [...] Can you clarify the people you think will get stuck? I think I'm right in saying that anyone with a C extension should be able to build them against numpy, by installing the free (as-in-beer) MS tools? So do you just mean people needing a Fortran compiler? That's a small constituency, I think. Off the top of my head there's SciPy and pymc... Anyway, I'm butting in because I wish this discussion could separate between the user perspective and the developer perspective. FWIW, 1) From a user's perspective, I don't understand this either. If you are already using a closed source, not-free-as-in-beer operating system, why would you not use (or buy!) a closed source, not-free-as-in-beer Fortran compiler? Indeed. Though I really have no clue on the Windows use cases. Maybe most Windows users don't want to compile anything, just use numpy and scipy from Python? 2) BUT, the argument I've seen that I can at least understand is that the release manager should be able to do a release using only open source tools (even using Wine instead of Windows) and not rely on a limited number of licenses. And that the release manager must be able to perform all the official builds directly. As the release manager, I really only have two requirements: * I want to ssh in there from my Ubuntu * I want to automate the whole process For Mac, linux and Wine I can do that. So I have just spend few hours browsing the net and it looks like that the combination of Windows PowerShell 2.0: http://en.wikipedia.org/wiki/Windows_PowerShell and some SSH server, there are quite a few, one commercial but free for one user one connection (perfect for me!): http://www.powershellinside.com/powershell/ssh/ So if I understand the pages correctly, I can login there from linux, and then I use the PowerShell commands to script anything. It looks like I can even use my Fabric fabfiles with powershell: https://gist.github.com/diyan/2850866 I can also use git with PowerShell: http://windows.github.com/ http://haacked.com/archive/2011/12/13/better-git-with-powershell.aspx Ondrej, you may be interested in some hack I've done to use winrm from fabric: https://github.com/fabric/fabric/pull/872 It gives a new winrm_run function where you can put any batch command. While the code is a hack, it works pretty well in practice. This works from mac os x and linux, without the need for wine, or ssh on windows. David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
Hi Matthew, On Tue, Mar 26, 2013 at 5:48 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi Ondrej, On Thu, Feb 7, 2013 at 3:18 PM, Ondřej Čertík ondrej.cer...@gmail.com wrote: On Thu, Feb 7, 2013 at 12:29 PM, Chris Barker - NOAA Federal chris.bar...@noaa.gov wrote: On Thu, Feb 7, 2013 at 11:38 AM, Matthew Brett matthew.br...@gmail.com wrote: a) If we cannot build Scipy now, it may or may not be acceptable to release numpy now. I think it is, you (Ralf) think it isn't, we haven't discussed that. It may not come up. Is anyone suggesting we hold the whole release for this? I fnot, then Just to make it clear, I do not plan to hold the whole release because of this. Previous releases also didn't have this official 64bit Windows binary, so there is no regression. Once we figure out how to create 64bit binaries, then we'll start uploading them. Did you make any progress with this? Worth making some notes? Anything we can do to help? Unfortunately I've been too busy the last month to push this through, so right now I am just concentrating on getting 1.7.1 out of the door, as that is higher priority. I am starting a new job on Monday, so once things settle down, I should be able to get back to this. I will post notes once I get to this again. Ondrej ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
Hi Ondrej, On Thu, Feb 7, 2013 at 3:18 PM, Ondřej Čertík ondrej.cer...@gmail.com wrote: On Thu, Feb 7, 2013 at 12:29 PM, Chris Barker - NOAA Federal chris.bar...@noaa.gov wrote: On Thu, Feb 7, 2013 at 11:38 AM, Matthew Brett matthew.br...@gmail.com wrote: a) If we cannot build Scipy now, it may or may not be acceptable to release numpy now. I think it is, you (Ralf) think it isn't, we haven't discussed that. It may not come up. Is anyone suggesting we hold the whole release for this? I fnot, then Just to make it clear, I do not plan to hold the whole release because of this. Previous releases also didn't have this official 64bit Windows binary, so there is no regression. Once we figure out how to create 64bit binaries, then we'll start uploading them. Did you make any progress with this? Worth making some notes? Anything we can do to help? Cheers, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
Hi, On Wed, Feb 6, 2013 at 10:59 PM, Ondřej Čertík ondrej.cer...@gmail.com wrote: On Wed, Feb 6, 2013 at 10:41 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Wed, Feb 6, 2013 at 10:21 PM, Ondřej Čertík ondrej.cer...@gmail.com wrote: On Wed, Feb 6, 2013 at 9:20 PM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: On 02/07/2013 12:16 AM, Matthew Brett wrote: [...] Can you clarify the people you think will get stuck? I think I'm right in saying that anyone with a C extension should be able to build them against numpy, by installing the free (as-in-beer) MS tools? So do you just mean people needing a Fortran compiler? That's a small constituency, I think. Off the top of my head there's SciPy and pymc... Anyway, I'm butting in because I wish this discussion could separate between the user perspective and the developer perspective. FWIW, 1) From a user's perspective, I don't understand this either. If you are already using a closed source, not-free-as-in-beer operating system, why would you not use (or buy!) a closed source, not-free-as-in-beer Fortran compiler? Indeed. Though I really have no clue on the Windows use cases. Maybe most Windows users don't want to compile anything, just use numpy and scipy from Python? Well - yes - as a packager I really want to be able to provide a binary so my binary consumers don't have to have a C compiler installed. I imagine it's the same for all of us packagers out there. 2) BUT, the argument I've seen that I can at least understand is that the release manager should be able to do a release using only open source tools (even using Wine instead of Windows) and not rely on a limited number of licenses. And that the release manager must be able to perform all the official builds directly. As the release manager, I really only have two requirements: * I want to ssh in there from my Ubuntu * I want to automate the whole process For Mac, linux and Wine I can do that. So I have just spend few hours browsing the net and it looks like that the combination of Windows PowerShell 2.0: http://en.wikipedia.org/wiki/Windows_PowerShell and some SSH server, there are quite a few, one commercial but free for one user one connection (perfect for me!): http://www.powershellinside.com/powershell/ssh/ So if I understand the pages correctly, I can login there from linux, and then I use the PowerShell commands to script anything. It looks like I can even use my Fabric fabfiles with powershell: https://gist.github.com/diyan/2850866 I can also use git with PowerShell: http://windows.github.com/ http://haacked.com/archive/2011/12/13/better-git-with-powershell.aspx So the final problem is how to execute MSVC and Fortran from Power Shell on Windows. These links might help for MSVC: http://stackoverflow.com/questions/4398136/use-powershell-for-visual-studio-command-prompt http://geekswithblogs.net/dwdii/archive/2011/05/20/automating-a-visual-studio-build-with-powershell---part-1.aspx Finally, for Intel Fortran + powershell: http://software.intel.com/en-us/forums/topic/284425 So I think it is all possible. If somebody can provide a machine with Windows, MSVC, PowerShell2.0, SSH server and some Fortran compiler, it should be possible for me to automate everything from Ubuntu using my Fabric files (https://github.com/certik/numpy-vendor). Many many thanks for trying to solve this. I had really started to give up hope. I think you will need a developer's license for MKL for Numpy. Ralf - any ETA for those? I think I'm right in thinking you'll need a Fortran compiler for Scipy but not Numpy? Can we defer the Scipy build until after the Numpy build? I will try to get you set up with ssh on my Windows 7 machine in case you can use it. It has the MS tools. That would be amazing! If you can set me up with the Power Shell and some ssh server, I'll start playing with this right away. I've set up a Cygwin SSH server on the box, and powershell 2 comes with windows 7, I believe. At least, that's the version I'm getting. However, it's hard to run powershell scripts interactively via cygwin : http://hivearchive.com/2006/07/03/using-powershell-through-ssh/ http://cygwin.com/ml/cygwin/2008-10/msg00393.html so you might need to debug the scripts interactively via remote desktop protocol and then run them non-interactively. Could you send me your ssh public key off list or give me a call to get set up? Thanks again, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
On 2/6/2013 10:35 PM, Ondřej Čertík wrote: Christoph, On Tue, Feb 5, 2013 at 3:04 PM, Christoph Gohlke cgoh...@uci.edu wrote: [...] In order not to leave this discussion without a resolution: Christophe - would you allow us to distribute your numpy binaries for 1.7 from the numpy sourceforge page? Cheers, Matthew I am OK with providing 64 bit numpy-MKL binaries (that is numpy compiled with MSVC compilers and linked to Intel's MKL) for official numpy releases. However: 1) There seems to be no real consensus and urge for doing this. Using a free toolchain capable of building the whole scipy-stack would be much preferred. Several 64 bit Python distributions containing numpy-MKL are already available, some for free. 2) Releasing 64 bit numpy without matching scipy binaries would make little sense to me. 3) Please do not just redistribute the binaries from my website and declare them official. They might contain unreleased fixes from git master and pull requests that are needed for my work and other packages. 4) Numpy-MKL requires the Intel runtime DLLs (MKL is linked statically btw). I ship those with the installers and append the directory containing the DLLs to os.environ['PATH'] in numpy/__init__.py. This is a big no-no according to numpy developers. I don't agree. Anyway, those changes are not in the numpy source repositories. 5) My numpy-MKL installers are Python distutils bdist_wininst installers. That means if Python was installed for all users, installing numpy-MKL on Windows 6.0 will prompt for UAC elevation. Another no-no? I think that all these things should be possible to fix so that the binary is acceptable for the official NumPy binary. How exactly do you build the binaries? I wasn't able to find the info at: http://www.lfd.uci.edu/~gohlke/pythonlibs/ Do you have some scripts to do that? Do you use PowerShell? Or you do it by hand by mouse and clicks in Visual Studio somehow? If I can figure out how to do these builds, I'll be happy to figure out how to automate it and then we can try to figure out a solution that works for NumPy. Ondrej My development/build environment is listed at http://www.lfd.uci.edu/~gohlke/pythonlibs/#buildenv. Not that it helps much... Assuming that Windows 7|8 Pro 64 bit, Visual Studio 2008 Pro SP1 (with 64 bit compiler option), Visual Studio 2010 Pro, Intel Composer XE 2013, 64 bit CPython 2.6, 2.7, 3.2 and 3.3 are installed, the following batch script (no need for PowerShell or an IDE) should build 64 bit numpy-MKL installers when run from within the numpy source directory. I do not really use this script but the secrets are there. It can be extended for building eggs and MSIs, 32 bit, and automated testing. Probably not all the libraries listed in site.cfg are needed but this works for me also with scipy and other packages. @echo off setlocal set ICDIR=C:/Program Files (x86)/Intel/Composer XE rem Work around a bug in numpy distutils. Requires admin privileges fsutil hardlink create %ICDIR%/mkl/lib/intel64/libiomp5md.lib %ICDIR%/compiler/lib/intel64/libiomp5md.lib fsutil hardlink create %ICDIR%/mkl/lib/intel64/libifportmd.lib %ICDIR%/compiler/lib/intel64/libifportmd.lib rem Create site.cfg for static linking to 64 bit MKL echo [mkl] site.cfg echo include_dirs = %ICDIR%/mkl/include site.cfg echo library_dirs = %ICDIR%/mkl/lib/intel64;%ICDIR%/compiler/lib/intel64 site.cfg echo mkl_libs = mkl_lapack95_lp64,mkl_blas95_lp64,mkl_intel_lp64,mkl_intel_thread,mkl_core,libiomp5md,libifportmd site.cfg echo lapack_libs = mkl_lapack95_lp64,mkl_blas95_lp64,mkl_intel_lp64,mkl_intel_thread,mkl_core,libiomp5md,libifportmd site.cfg rem Build installers using distutils rd /q /s build call C:\Python26\python.exe setup.py bdist_wininst --user-access-control=auto rd /q /s build call C:\Python27\python.exe setup.py bdist_wininst --user-access-control=auto rd /q /s build call C:\Python32\python.exe setup.py bdist_wininst --user-access-control=auto copy /Y build\py3k\dist\*.exe dist\ rd /q /s build call C:\Python33\python.exe setup.py bdist_wininst --user-access-control=auto copy /Y build\py3k\dist\*.exe dist\ rd /q /s build endlocal -- Christoph ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
On Thu, Feb 7, 2013 at 7:41 AM, Matthew Brett matthew.br...@gmail.comwrote: I think you will need a developer's license for MKL for Numpy. Ralf - any ETA for those? No, I'll have to ask again. I think I'm right in thinking you'll need a Fortran compiler for Scipy but not Numpy? Correct. Can we defer the Scipy build until after the Numpy build? That doesn't sound like a good idea to me. I will try to get you set up with ssh on my Windows 7 machine in case you can use it. It has the MS tools. As pointed out before by Robert, sharing the machine between you requires multiple licenses. Intel has promised us some licenses, but those are given to specific people. Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
Hi, On Thu, Feb 7, 2013 at 10:47 AM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Thu, Feb 7, 2013 at 7:41 AM, Matthew Brett matthew.br...@gmail.com wrote: I think you will need a developer's license for MKL for Numpy. Ralf - any ETA for those? No, I'll have to ask again. I think I'm right in thinking you'll need a Fortran compiler for Scipy but not Numpy? Correct. Can we defer the Scipy build until after the Numpy build? That doesn't sound like a good idea to me. I will try to get you set up with ssh on my Windows 7 machine in case you can use it. It has the MS tools. As pointed out before by Robert, sharing the machine between you requires multiple licenses. Intel has promised us some licenses, but those are given to specific people. Right. Ondrej has his own account on my machine, and it will be Ondrej's license, when it is available. Cheers, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
Hi, On Thu, Feb 7, 2013 at 10:47 AM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Thu, Feb 7, 2013 at 7:41 AM, Matthew Brett matthew.br...@gmail.com wrote: I think you will need a developer's license for MKL for Numpy. Ralf - any ETA for those? No, I'll have to ask again. I think I'm right in thinking you'll need a Fortran compiler for Scipy but not Numpy? Correct. Can we defer the Scipy build until after the Numpy build? That doesn't sound like a good idea to me. I must say I'm a little confused as to how we're going to make the decisions here. I'm sure you agree that there's an opposite argument to be made, and I would make it if I thought it would make a difference, but I'm losing faith in my ability to keep the discussion on track, and I don't know what to do about that. Cheers, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
On Thu, Feb 7, 2013 at 10:59 AM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Thu, Feb 7, 2013 at 10:47 AM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Thu, Feb 7, 2013 at 7:41 AM, Matthew Brett matthew.br...@gmail.com wrote: I think you will need a developer's license for MKL for Numpy. Ralf - any ETA for those? No, I'll have to ask again. I think I'm right in thinking you'll need a Fortran compiler for Scipy but not Numpy? Correct. Can we defer the Scipy build until after the Numpy build? That doesn't sound like a good idea to me. I must say I'm a little confused as to how we're going to make the decisions here. I'm sure you agree that there's an opposite argument to be made, and I would make it if I thought it would make a difference, but I'm losing faith in my ability to keep the discussion on track, and I don't know what to do about that. Matthew I don't see any problem here. I agree with Ralf that we need to figure out how to build scipy with Fortran pretty much at the same time, to see if the solution is a viable solution. With Christoph help and experience, I am sure it can get done in a satisfactory way. Ondrej ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
On Thu, Feb 7, 2013 at 7:59 PM, Matthew Brett matthew.br...@gmail.comwrote: Can we defer the Scipy build until after the Numpy build? That doesn't sound like a good idea to me. I must say I'm a little confused as to how we're going to make the decisions here. How about: attempt to reach consensus? David's concern on DLLs hasn't been addressed yet, nor has mine on packages being unavailable. I was actually still answering another of your emails, but I can't seem to reply fast enough. I'm sure you agree that there's an opposite argument to be made, and I would make it if I thought it would make a difference, but I'm losing faith in my ability to keep the discussion on track, and I don't know what to do about that. I don't see the problem. Before you offered to put in work. Ondrej is willing to help, so is Christoph. So why is it impossible to do Scipy builds? I can see us getting to a solution here, but offering Numpy installers without Scipy ones is not a solution in my book. Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
On Thu, Feb 7, 2013 at 11:05 AM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Thu, Feb 7, 2013 at 7:59 PM, Matthew Brett matthew.br...@gmail.com wrote: Can we defer the Scipy build until after the Numpy build? That doesn't sound like a good idea to me. I must say I'm a little confused as to how we're going to make the decisions here. How about: attempt to reach consensus? David's concern on DLLs hasn't been addressed yet, nor has mine on packages being unavailable. I was actually still answering another of your emails, but I can't seem to reply fast enough. Yep, we will need to address those. I'm sure you agree that there's an opposite argument to be made, and I would make it if I thought it would make a difference, but I'm losing faith in my ability to keep the discussion on track, and I don't know what to do about that. I don't see the problem. Before you offered to put in work. Ondrej is willing to help, so is Christoph. So why is it impossible to do Scipy builds? I can see us getting to a solution here, but offering Numpy installers without Scipy ones is not a solution in my book. Exactly. There is no problem here. Fortran needs to be working as a first class citizen. I personally use modern Fortran a lot. I've setup this page: http://fortran90.org/ with a relevant FAQ about binary compatibility: http://fortran90.org/src/faq.html#are-fortran-compilers-abi-compatible and based on how things work on Windows, I'll be happy to extend the information there. Ondrej ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
Hi, On Thu, Feb 7, 2013 at 11:05 AM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Thu, Feb 7, 2013 at 7:59 PM, Matthew Brett matthew.br...@gmail.com wrote: Can we defer the Scipy build until after the Numpy build? That doesn't sound like a good idea to me. I must say I'm a little confused as to how we're going to make the decisions here. How about: attempt to reach consensus? David's concern on DLLs hasn't been addressed yet, nor has mine on packages being unavailable. I was actually still answering another of your emails, but I can't seem to reply fast enough. Right - consensus is good - but at the moment I keep getting lost because the arguments seem to shift and get lost, and sometimes they are not made. So, here is the summary as I understand it, please correct if I am wrong I think we agree that: 1) Having a binary installer for numpy Windows 64 bit is desirable 2) It is desirable to have a matching binary installer for Scipy as soon as possible 3) It is preferable to build with free tools 4) It is acceptable to use non-free tools 5) The build will need to do some run-time linking to MKL and / or mingw 6) It is preferable that the build should be fully automated 7) It is preferable that one person can build all numpy / scipy builds The points of potential disagreement are: a) If we cannot build Scipy now, it may or may not be acceptable to release numpy now. I think it is, you (Ralf) think it isn't, we haven't discussed that. It may not come up. b) It may or may not be acceptable for someone other than Ondrej to be responsible for the Windows 64-bit builds. I think it should be, if necessary, we haven't really discussed that, it may not come up. c) It may or may not be acceptable for the build to be only partially automated. Ditto. d) It may or may not be acceptable to add the DLL directory to the PATH on numpy import. David says not, Christophe disagrees, we haven't really discussed that. Is that right? Cheers, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
On Thu, Feb 7, 2013 at 8:38 PM, Matthew Brett matthew.br...@gmail.comwrote: Hi, On Thu, Feb 7, 2013 at 11:05 AM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Thu, Feb 7, 2013 at 7:59 PM, Matthew Brett matthew.br...@gmail.com wrote: Can we defer the Scipy build until after the Numpy build? That doesn't sound like a good idea to me. I must say I'm a little confused as to how we're going to make the decisions here. How about: attempt to reach consensus? David's concern on DLLs hasn't been addressed yet, nor has mine on packages being unavailable. I was actually still answering another of your emails, but I can't seem to reply fast enough. Right - consensus is good - but at the moment I keep getting lost because the arguments seem to shift and get lost, and sometimes they are not made. So, here is the summary as I understand it, please correct if I am wrong I think we agree that: 1) Having a binary installer for numpy Windows 64 bit is desirable 2) It is desirable to have a matching binary installer for Scipy as soon as possible 3) It is preferable to build with free tools 4) It is acceptable to use non-free tools 5) The build will need to do some run-time linking to MKL and / or mingw 6) It is preferable that the build should be fully automated 7) It is preferable that one person can build all numpy / scipy builds The points of potential disagreement are: a) If we cannot build Scipy now, it may or may not be acceptable to release numpy now. I think it is, you (Ralf) think it isn't, we haven't discussed that. It may not come up. b) It may or may not be acceptable for someone other than Ondrej to be responsible for the Windows 64-bit builds. I think it should be, if necessary, we haven't really discussed that, it may not come up. c) It may or may not be acceptable for the build to be only partially automated. Ditto. d) It may or may not be acceptable to add the DLL directory to the PATH on numpy import. David says not, Christophe disagrees, we haven't really discussed that. Is that right? Good summary, looks complete to me. Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
On Thu, Feb 7, 2013 at 11:38 AM, Matthew Brett matthew.br...@gmail.com wrote: a) If we cannot build Scipy now, it may or may not be acceptable to release numpy now. I think it is, you (Ralf) think it isn't, we haven't discussed that. It may not come up. Is anyone suggesting we hold the whole release for this? I fnot, then why would you say we shouldnt' put out a 64 bit Windows numpy binary because we dont' ahve a matching Scipy one? folks that need Scipy will have to find a way to built it themselves anyway, and folks that use numpy without scipy would have a nice binary to use -- That seems like value-added to me. d) It may or may not be acceptable to add the DLL directory to the PATH on numpy import. David says not, Christophe disagrees, we haven't really discussed that. I like the load them with ctypes approach better, but I don't know how much effort it would be to implement that. I'm still not totally clear if there is an issue with buildings other extensions tht rely on numpy/scipy -- can they be built with mingGW if numpy/scipy were built with MSVC (Or any other compiler for that matter). -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
Hi, On Thu, Feb 7, 2013 at 12:29 PM, Chris Barker - NOAA Federal chris.bar...@noaa.gov wrote: On Thu, Feb 7, 2013 at 11:38 AM, Matthew Brett matthew.br...@gmail.com wrote: a) If we cannot build Scipy now, it may or may not be acceptable to release numpy now. I think it is, you (Ralf) think it isn't, we haven't discussed that. It may not come up. Is anyone suggesting we hold the whole release for this? I fnot, then why would you say we shouldnt' put out a 64 bit Windows numpy binary because we dont' ahve a matching Scipy one? folks that need Scipy will have to find a way to built it themselves anyway, and folks that use numpy without scipy would have a nice binary to use -- That seems like value-added to me. Right - me too - but we could hold off that question until Ondrej has had a chance to build both I guess? d) It may or may not be acceptable to add the DLL directory to the PATH on numpy import. David says not, Christophe disagrees, we haven't really discussed that. I like the load them with ctypes approach better, but I don't know how much effort it would be to implement that. I'm still not totally clear if there is an issue with buildings other extensions tht rely on numpy/scipy -- can they be built with mingGW if numpy/scipy were built with MSVC (Or any other compiler for that matter). I had the impression that the problem with Mingw64 was the need to load Mingw DLLs at run time - if that's the same for MSVC / MKL, maybe there's no advantage to using the MS tools? But I don't understand the issues very well. Cheers, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
On Thu, Feb 7, 2013 at 7:38 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Thu, Feb 7, 2013 at 11:05 AM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Thu, Feb 7, 2013 at 7:59 PM, Matthew Brett matthew.br...@gmail.com wrote: Can we defer the Scipy build until after the Numpy build? That doesn't sound like a good idea to me. I must say I'm a little confused as to how we're going to make the decisions here. How about: attempt to reach consensus? David's concern on DLLs hasn't been addressed yet, nor has mine on packages being unavailable. I was actually still answering another of your emails, but I can't seem to reply fast enough. Right - consensus is good - but at the moment I keep getting lost because the arguments seem to shift and get lost, and sometimes they are not made. So, here is the summary as I understand it, please correct if I am wrong I think we agree that: 1) Having a binary installer for numpy Windows 64 bit is desirable 2) It is desirable to have a matching binary installer for Scipy as soon as possible 3) It is preferable to build with free tools 4) It is acceptable to use non-free tools 5) The build will need to do some run-time linking to MKL and / or mingw 6) It is preferable that the build should be fully automated 7) It is preferable that one person can build all numpy / scipy builds The points of potential disagreement are: a) If we cannot build Scipy now, it may or may not be acceptable to release numpy now. I think it is, you (Ralf) think it isn't, we haven't discussed that. It may not come up. b) It may or may not be acceptable for someone other than Ondrej to be responsible for the Windows 64-bit builds. I think it should be, if necessary, we haven't really discussed that, it may not come up. c) It may or may not be acceptable for the build to be only partially automated. Ditto. d) It may or may not be acceptable to add the DLL directory to the PATH on numpy import. David says not, Christophe disagrees, we haven't really discussed that. I assumed this was obvious, but looks like it isn't: modifying the process user environment when importing a python package is quite user hostile. os.environ is a global variable, and it could cause quite hard to diagnose issues when something goes wrong. I would see a library doing this kind of things, especially at import time, quite badly. David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
On Thu, Feb 7, 2013 at 12:29 PM, Chris Barker - NOAA Federal chris.bar...@noaa.gov wrote: On Thu, Feb 7, 2013 at 11:38 AM, Matthew Brett matthew.br...@gmail.com wrote: a) If we cannot build Scipy now, it may or may not be acceptable to release numpy now. I think it is, you (Ralf) think it isn't, we haven't discussed that. It may not come up. Is anyone suggesting we hold the whole release for this? I fnot, then Just to make it clear, I do not plan to hold the whole release because of this. Previous releases also didn't have this official 64bit Windows binary, so there is no regression. Once we figure out how to create 64bit binaries, then we'll start uploading them. Ondrej ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
On Tue, Feb 5, 2013 at 5:01 PM, Matthew Brett matthew.br...@gmail.com wrote: easy_install can install into virtualenvs from bdist_wininst installers, at least the ones I have built... really? cool! I never thought to try that! Thanks, -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
On Wed, Feb 6, 2013 at 1:32 AM, Matthew Brett matthew.br...@gmail.comwrote: Hi, On Tue, Feb 5, 2013 at 3:04 PM, Christoph Gohlke cgoh...@uci.edu wrote: On 2/5/2013 10:51 AM, Matthew Brett wrote: Hi, On Mon, Feb 4, 2013 at 5:09 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Feb 4, 2013 at 3:46 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Mon, Feb 4, 2013 at 4:04 PM, Robert Kern robert.k...@gmail.com wrote: On Mon, Feb 4, 2013 at 10:38 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Feb 4, 2013 at 1:15 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: MSVC + Intel Fortran + MKL, yes. But those aren't free. So how can you provide an Amazon image for those? You can make an image that is not public, I guess. I suppose anyone who uses the image would have to have their own licenses for the Intel stuff? Does anyone have experience of this? You need to purchase one license per developer: http://software.intel.com/en-us/articles/intel-math-kernel-library-licensing-faq#eula1 I think 64 bits on windows is best pushed off to 1.7.1 or 1.8. It would be a bit much to get it implemented in the next week or two. The problem with not providing these binaries is that they are at the bottom of everyone's stack, so a delay in numpy holds everyone back. I can't find completely convincing stats, but it looks as though 64 bit windows 7 is now the most common version of Windows, at least for Gamers [1] around now, and it was getting that way for everyone in 2010 [2]. It don't think it reflects well on on us that we don't appear to support 64 bits out of the box; just for example, R already has a 32 bit / 64 bit installer. If I understand correctly, the options for doing this right now are: 1) Minimal cost in time : ask Christophe nicely whether we can distribute his binaries via the Numpy page 2) Small cost in time / money : pay for licenses for Ondrej or me or someone to install the dependencies on my Berkeley machine / an Amazon image. In order not to leave this discussion without a resolution: Christophe - would you allow us to distribute your numpy binaries for 1.7 from the numpy sourceforge page? Cheers, Matthew I am OK with providing 64 bit numpy-MKL binaries (that is numpy compiled with MSVC compilers and linked to Intel's MKL) for official numpy releases. Thank you - that is great. However: 1) There seems to be no real consensus and urge for doing this. I certainly feel the urge and feel it strongly. As a packager for two or three projects myself, it's a royal pain having to tell someone to go to two different places for binaries depending on the number of bits of their Windows. If you're relying on .exe installers on SF, then you have to send your users to more places than that probably. Really the separate installers are a poor alternative to the available scientific distributions. And the more packages you need as a user, the more annoying these separate installers are. I think Chuck was worried about the time it would take to do it, and I think you've already solved this problem. Ralf was worried about Scipy - see below. Not just Scipy - that would be my worry number one, but the same holds for all packages based on Numpy. You double the number of Windows installers that every single project needs to provide. Using a free toolchain capable of building the whole scipy-stack would be much preferred. That's true, but there seems general agreement this is not practical in the very near future. Several 64 bit Python distributions containing numpy-MKL are already available, some for free. You mean EPD and AnacondaCE? I don't think we should withhold easily available vanilla builds because they are also available in company-sponsored projects. Python.org provides windows builds even though ActiveState is free-as-in-beer. If the company-sponsored bit bothers you, there's also a 64-bit Python(x,y) now. Ralf 2) Releasing 64 bit numpy without matching scipy binaries would make little sense to me. Would you consider also releasing your scipy binaries? 3) Please do not just redistribute the binaries from my website and declare them official. They might contain unreleased fixes from git master and pull requests that are needed for my work and other packages. Right - would you consider then being the release provider for numpy / scipy binaries on windows, much as it appears that Martin v Lowis supplies Windows builds for Python? 4) Numpy-MKL requires the Intel runtime DLLs (MKL is linked statically btw). I ship those with the installers and append the directory containing the DLLs to os.environ['PATH'] in numpy/__init__.py. This is a big no-no according to numpy developers. I don't agree. Anyway, those changes are not in the numpy source repositories. 5) My
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
On 6 February 2013 01:55, Chris Barker - NOAA Federal wrote: On Tue, Feb 5, 2013 at 4:32 PM, Matthew Brett matthew.br...@gmail.com wrote: 4) Numpy-MKL requires the Intel runtime DLLs (MKL is linked statically btw). I ship those with the installers and append the directory containing the DLLs to os.environ['PATH'] in numpy/__init__.py. This is a big no-no according to numpy developers. I don't agree. Anyway, those changes are not in the numpy source repositories. I think you pointed out that another option is to load the dlls with ctypes -- is it much work to make that change? 5) My numpy-MKL installers are Python distutils bdist_wininst installers. That means if Python was installed for all users, installing numpy-MKL on Windows 6.0 will prompt for UAC elevation. Another no-no? not sure about the UAC elevation -- but: 1) most folks use bdist_wininst for Windows binaries -- including the current numpy builds, and python.org python -- yes? Even the current approach is off-limits for the few haggards out there (like me at work), who can not update numpy/scipy/anything else that requires an installation procedure (like pretty much all the bdist_wininst distributions for Windows 64 bits/Python 64 bits). The only (partial) solution I found is to install at home and bring the site-packages folder on a USB stick with me at work. Which is a bit sad overall: if I can make dozens of installers of my applications for my colleagues with InnoSetup/NSIS which do *not* require any UAC crap/elevation, what's stopping the bdist_wininst to do the same? The same holds (unfortunately) for the excellent distributions from Christoph Gohlke. The fact that Windows 64 bits/Python 64 bits UAC-free installers are pretty much non-existent in the Python world does not play very nicely with the fact that 64 bits architectures have been available for 783648660236729 years. I'd rather prefer if someone would upload to sourceforge/scipy.org/whatever a zipped folder containing numpy as it was in their site-packages directory. Now that would be a huge plus :-) Andrea. Imagination Is The Only Weapon In The War Against Reality. http://www.infinity77.net # - # def ask_mailing_list_support(email): if mention_platform_and_version() and include_sample_app(): send_message(email) else: install_malware() erase_hard_drives() # - # ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
Hi, On Wed, Feb 6, 2013 at 10:48 AM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Wed, Feb 6, 2013 at 1:32 AM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Tue, Feb 5, 2013 at 3:04 PM, Christoph Gohlke cgoh...@uci.edu wrote: On 2/5/2013 10:51 AM, Matthew Brett wrote: Hi, On Mon, Feb 4, 2013 at 5:09 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Feb 4, 2013 at 3:46 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Mon, Feb 4, 2013 at 4:04 PM, Robert Kern robert.k...@gmail.com wrote: On Mon, Feb 4, 2013 at 10:38 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Feb 4, 2013 at 1:15 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: MSVC + Intel Fortran + MKL, yes. But those aren't free. So how can you provide an Amazon image for those? You can make an image that is not public, I guess. I suppose anyone who uses the image would have to have their own licenses for the Intel stuff? Does anyone have experience of this? You need to purchase one license per developer: http://software.intel.com/en-us/articles/intel-math-kernel-library-licensing-faq#eula1 I think 64 bits on windows is best pushed off to 1.7.1 or 1.8. It would be a bit much to get it implemented in the next week or two. The problem with not providing these binaries is that they are at the bottom of everyone's stack, so a delay in numpy holds everyone back. I can't find completely convincing stats, but it looks as though 64 bit windows 7 is now the most common version of Windows, at least for Gamers [1] around now, and it was getting that way for everyone in 2010 [2]. It don't think it reflects well on on us that we don't appear to support 64 bits out of the box; just for example, R already has a 32 bit / 64 bit installer. If I understand correctly, the options for doing this right now are: 1) Minimal cost in time : ask Christophe nicely whether we can distribute his binaries via the Numpy page 2) Small cost in time / money : pay for licenses for Ondrej or me or someone to install the dependencies on my Berkeley machine / an Amazon image. In order not to leave this discussion without a resolution: Christophe - would you allow us to distribute your numpy binaries for 1.7 from the numpy sourceforge page? Cheers, Matthew I am OK with providing 64 bit numpy-MKL binaries (that is numpy compiled with MSVC compilers and linked to Intel's MKL) for official numpy releases. Thank you - that is great. However: 1) There seems to be no real consensus and urge for doing this. I certainly feel the urge and feel it strongly. As a packager for two or three projects myself, it's a royal pain having to tell someone to go to two different places for binaries depending on the number of bits of their Windows. If you're relying on .exe installers on SF, then you have to send your users to more places than that probably. Really the separate installers are a poor alternative to the available scientific distributions. And the more packages you need as a user, the more annoying these separate installers are. I think Chuck was worried about the time it would take to do it, and I think you've already solved this problem. Ralf was worried about Scipy - see below. Not just Scipy - that would be my worry number one, but the same holds for all packages based on Numpy. You double the number of Windows installers that every single project needs to provide. Using a free toolchain capable of building the whole scipy-stack would be much preferred. That's true, but there seems general agreement this is not practical in the very near future. Several 64 bit Python distributions containing numpy-MKL are already available, some for free. You mean EPD and AnacondaCE? I don't think we should withhold easily available vanilla builds because they are also available in company-sponsored projects. Python.org provides windows builds even though ActiveState is free-as-in-beer. If the company-sponsored bit bothers you, there's also a 64-bit Python(x,y) now. I'm sure you've seen that the question 'where are the 64-bit installers' comes up often for Numpy. It seems to me that we'd have to have a good reason not provide them. There will always be some number of people like me who like to install the various parts by hand, and don't like using large packages, free or or open or not. For example, I don't use macports. At the moment, the reasons I see you are giving are: 1) Then everyone would have to provide 64-bit binaries 2) You should use a super-package instead On those arguments, we should withdraw all binary installers. Or withdraw the 32-bit ones, on the basis that 64-bit is likely the more common now. Of course we shouldn't do that, because it will put off some measurable number of users, and convey the impression that the numpy
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
On Wed, Feb 6, 2013 at 8:46 PM, Matthew Brett matthew.br...@gmail.comwrote: Hi, On Wed, Feb 6, 2013 at 10:48 AM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Wed, Feb 6, 2013 at 1:32 AM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Tue, Feb 5, 2013 at 3:04 PM, Christoph Gohlke cgoh...@uci.edu wrote: On 2/5/2013 10:51 AM, Matthew Brett wrote: Hi, On Mon, Feb 4, 2013 at 5:09 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Feb 4, 2013 at 3:46 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Mon, Feb 4, 2013 at 4:04 PM, Robert Kern robert.k...@gmail.com wrote: On Mon, Feb 4, 2013 at 10:38 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Feb 4, 2013 at 1:15 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: MSVC + Intel Fortran + MKL, yes. But those aren't free. So how can you provide an Amazon image for those? You can make an image that is not public, I guess. I suppose anyone who uses the image would have to have their own licenses for the Intel stuff? Does anyone have experience of this? You need to purchase one license per developer: http://software.intel.com/en-us/articles/intel-math-kernel-library-licensing-faq#eula1 I think 64 bits on windows is best pushed off to 1.7.1 or 1.8. It would be a bit much to get it implemented in the next week or two. The problem with not providing these binaries is that they are at the bottom of everyone's stack, so a delay in numpy holds everyone back. I can't find completely convincing stats, but it looks as though 64 bit windows 7 is now the most common version of Windows, at least for Gamers [1] around now, and it was getting that way for everyone in 2010 [2]. It don't think it reflects well on on us that we don't appear to support 64 bits out of the box; just for example, R already has a 32 bit / 64 bit installer. If I understand correctly, the options for doing this right now are: 1) Minimal cost in time : ask Christophe nicely whether we can distribute his binaries via the Numpy page 2) Small cost in time / money : pay for licenses for Ondrej or me or someone to install the dependencies on my Berkeley machine / an Amazon image. In order not to leave this discussion without a resolution: Christophe - would you allow us to distribute your numpy binaries for 1.7 from the numpy sourceforge page? Cheers, Matthew I am OK with providing 64 bit numpy-MKL binaries (that is numpy compiled with MSVC compilers and linked to Intel's MKL) for official numpy releases. Thank you - that is great. However: 1) There seems to be no real consensus and urge for doing this. I certainly feel the urge and feel it strongly. As a packager for two or three projects myself, it's a royal pain having to tell someone to go to two different places for binaries depending on the number of bits of their Windows. If you're relying on .exe installers on SF, then you have to send your users to more places than that probably. Really the separate installers are a poor alternative to the available scientific distributions. And the more packages you need as a user, the more annoying these separate installers are. I think Chuck was worried about the time it would take to do it, and I think you've already solved this problem. Ralf was worried about Scipy - see below. Not just Scipy - that would be my worry number one, but the same holds for all packages based on Numpy. You double the number of Windows installers that every single project needs to provide. Using a free toolchain capable of building the whole scipy-stack would be much preferred. That's true, but there seems general agreement this is not practical in the very near future. Several 64 bit Python distributions containing numpy-MKL are already available, some for free. You mean EPD and AnacondaCE? I don't think we should withhold easily available vanilla builds because they are also available in company-sponsored projects. Python.org provides windows builds even though ActiveState is free-as-in-beer. If the company-sponsored bit bothers you, there's also a 64-bit Python(x,y) now. I'm sure you've seen that the question 'where are the 64-bit installers' comes up often for Numpy. It seems to me that we'd have to have a good reason not provide them. There will always be some number of people like me who like to install the various parts by hand, and don't like using large packages, free or or open or not. For example, I don't use macports. At the moment, the reasons I see you are giving are: 1) Then everyone would have to provide 64-bit binaries Indeed. And since many packages won't do that because there's no free compilers, users
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
I'm trying to weed out the issues here: 1) what should the binary installer for Windows look like: * bdist_wininst is the obvious choice but apparently has issues with the newer Windows security stuff -- the real solution is for distutils to be fixed/use something else/ etc -- but is there anyone here that has expertise, interest or time to get in and add-to or fix distutils? Do binary eggs solve any of this ? maybe, but pip doesn't support that last I saw, and setuptools easy-install is kind-of sort-of broken. But coming up with the best binary installer tool is orthogonal to the other questions: 2) Should we distribute binaries at all? - No: most people need a whole stack anyway, so they should get all that from the same place: - Enthought - Python(x.y) - Chris Gohlke's repository... - Yes: Some folks just want numpy damn it! Those big ol' distributions are a mess that don't work for everyone. (I'm in that camp.) But if we distribute binaries, we really should: - distribute them for 32 64 bit - Clearly define what the official binaries are, so that third-party packagers can build against that. Historically, this has been a huge mess on the Mac, as there are many options for Python itself: Apple's builds, Python,org builds, macports, fink, homebrew, .. Over the years, we in teh MacPython community have tried hard to establish that the python.org builds are the official builds that folks should support with binaries -- this has kind-of, sort-of worked, but I still don't see a better solution. (macports et al aren't really the issue, they aren't based on binaries anyway) On Windows, this has been pretty much a non-issue: MS doesn't provide a build, and the pyton.org builds are well accepted as the default binaries. AFAIU, third party distributions are compatible as well (Active State, Enthought) The parallel here is that we can establish in the scientific python community (that is, folks building packages based on numpy) that the binaries distributed by numpy are the official ones that should be supported by third part binaries. If we succeed in that, then folks can get the rest of the stack from any number of sources. the python.org python builds are built with the MS compilers, is there any reason we HAVE to build with a open-source stack? Can you build third party packages against an MS-built binary numpy with open-source compilers? -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
Hi, On Wed, Feb 6, 2013 at 1:36 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Wed, Feb 6, 2013 at 8:46 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Wed, Feb 6, 2013 at 10:48 AM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Wed, Feb 6, 2013 at 1:32 AM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Tue, Feb 5, 2013 at 3:04 PM, Christoph Gohlke cgoh...@uci.edu wrote: On 2/5/2013 10:51 AM, Matthew Brett wrote: Hi, On Mon, Feb 4, 2013 at 5:09 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Feb 4, 2013 at 3:46 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Mon, Feb 4, 2013 at 4:04 PM, Robert Kern robert.k...@gmail.com wrote: On Mon, Feb 4, 2013 at 10:38 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Feb 4, 2013 at 1:15 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: MSVC + Intel Fortran + MKL, yes. But those aren't free. So how can you provide an Amazon image for those? You can make an image that is not public, I guess. I suppose anyone who uses the image would have to have their own licenses for the Intel stuff? Does anyone have experience of this? You need to purchase one license per developer: http://software.intel.com/en-us/articles/intel-math-kernel-library-licensing-faq#eula1 I think 64 bits on windows is best pushed off to 1.7.1 or 1.8. It would be a bit much to get it implemented in the next week or two. The problem with not providing these binaries is that they are at the bottom of everyone's stack, so a delay in numpy holds everyone back. I can't find completely convincing stats, but it looks as though 64 bit windows 7 is now the most common version of Windows, at least for Gamers [1] around now, and it was getting that way for everyone in 2010 [2]. It don't think it reflects well on on us that we don't appear to support 64 bits out of the box; just for example, R already has a 32 bit / 64 bit installer. If I understand correctly, the options for doing this right now are: 1) Minimal cost in time : ask Christophe nicely whether we can distribute his binaries via the Numpy page 2) Small cost in time / money : pay for licenses for Ondrej or me or someone to install the dependencies on my Berkeley machine / an Amazon image. In order not to leave this discussion without a resolution: Christophe - would you allow us to distribute your numpy binaries for 1.7 from the numpy sourceforge page? Cheers, Matthew I am OK with providing 64 bit numpy-MKL binaries (that is numpy compiled with MSVC compilers and linked to Intel's MKL) for official numpy releases. Thank you - that is great. However: 1) There seems to be no real consensus and urge for doing this. I certainly feel the urge and feel it strongly. As a packager for two or three projects myself, it's a royal pain having to tell someone to go to two different places for binaries depending on the number of bits of their Windows. If you're relying on .exe installers on SF, then you have to send your users to more places than that probably. Really the separate installers are a poor alternative to the available scientific distributions. And the more packages you need as a user, the more annoying these separate installers are. I think Chuck was worried about the time it would take to do it, and I think you've already solved this problem. Ralf was worried about Scipy - see below. Not just Scipy - that would be my worry number one, but the same holds for all packages based on Numpy. You double the number of Windows installers that every single project needs to provide. Using a free toolchain capable of building the whole scipy-stack would be much preferred. That's true, but there seems general agreement this is not practical in the very near future. Several 64 bit Python distributions containing numpy-MKL are already available, some for free. You mean EPD and AnacondaCE? I don't think we should withhold easily available vanilla builds because they are also available in company-sponsored projects. Python.org provides windows builds even though ActiveState is free-as-in-beer. If the company-sponsored bit bothers you, there's also a 64-bit Python(x,y) now. I'm sure you've seen that the question 'where are the 64-bit installers' comes up often for Numpy. It seems to me that we'd have to have a good reason not provide them. There will always be some number of people like me who like to install the various parts by hand, and don't like using large packages, free or or open or not. For example, I don't use macports. At the moment, the reasons I see you are giving are: 1) Then everyone
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
On Wed, Feb 6, 2013 at 3:16 PM, Matthew Brett matthew.br...@gmail.com wrote: Can you clarify the people you think will get stuck? I think I'm right in saying that anyone with a C extension should be able to build them against numpy, by installing the free (as-in-beer) MS tools? yup -- and that's the recommended (easiest) way to do it against the the python.org python. and you can use mingw to compile extensions that will run with the python.org python (built with MSVC), can you not use mingw to build extensions that will work with a MSVC-build numpy? So do you just mean people needing a Fortran compiler? That's a small constituency, I think. particularly small overlap between needing fortran (and knowing how to deal with building it) and needing binaries of numpy if someone has a fortran-extension-building solution -- is it likeley not to work with a MSVC-built numpy? And, if we start providing installers, I suspect we'll find that people start solving the remaining issues much more quickly than would happen if we don't provide them. +1 -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
On 02/07/2013 12:16 AM, Matthew Brett wrote: Hi, On Wed, Feb 6, 2013 at 1:36 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Wed, Feb 6, 2013 at 8:46 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Wed, Feb 6, 2013 at 10:48 AM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Wed, Feb 6, 2013 at 1:32 AM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Tue, Feb 5, 2013 at 3:04 PM, Christoph Gohlke cgoh...@uci.edu wrote: On 2/5/2013 10:51 AM, Matthew Brett wrote: Hi, On Mon, Feb 4, 2013 at 5:09 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Feb 4, 2013 at 3:46 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Mon, Feb 4, 2013 at 4:04 PM, Robert Kern robert.k...@gmail.com wrote: On Mon, Feb 4, 2013 at 10:38 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Feb 4, 2013 at 1:15 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: MSVC + Intel Fortran + MKL, yes. But those aren't free. So how can you provide an Amazon image for those? You can make an image that is not public, I guess. I suppose anyone who uses the image would have to have their own licenses for the Intel stuff? Does anyone have experience of this? You need to purchase one license per developer: http://software.intel.com/en-us/articles/intel-math-kernel-library-licensing-faq#eula1 I think 64 bits on windows is best pushed off to 1.7.1 or 1.8. It would be a bit much to get it implemented in the next week or two. The problem with not providing these binaries is that they are at the bottom of everyone's stack, so a delay in numpy holds everyone back. I can't find completely convincing stats, but it looks as though 64 bit windows 7 is now the most common version of Windows, at least for Gamers [1] around now, and it was getting that way for everyone in 2010 [2]. It don't think it reflects well on on us that we don't appear to support 64 bits out of the box; just for example, R already has a 32 bit / 64 bit installer. If I understand correctly, the options for doing this right now are: 1) Minimal cost in time : ask Christophe nicely whether we can distribute his binaries via the Numpy page 2) Small cost in time / money : pay for licenses for Ondrej or me or someone to install the dependencies on my Berkeley machine / an Amazon image. In order not to leave this discussion without a resolution: Christophe - would you allow us to distribute your numpy binaries for 1.7 from the numpy sourceforge page? Cheers, Matthew I am OK with providing 64 bit numpy-MKL binaries (that is numpy compiled with MSVC compilers and linked to Intel's MKL) for official numpy releases. Thank you - that is great. However: 1) There seems to be no real consensus and urge for doing this. I certainly feel the urge and feel it strongly. As a packager for two or three projects myself, it's a royal pain having to tell someone to go to two different places for binaries depending on the number of bits of their Windows. If you're relying on .exe installers on SF, then you have to send your users to more places than that probably. Really the separate installers are a poor alternative to the available scientific distributions. And the more packages you need as a user, the more annoying these separate installers are. I think Chuck was worried about the time it would take to do it, and I think you've already solved this problem. Ralf was worried about Scipy - see below. Not just Scipy - that would be my worry number one, but the same holds for all packages based on Numpy. You double the number of Windows installers that every single project needs to provide. Using a free toolchain capable of building the whole scipy-stack would be much preferred. That's true, but there seems general agreement this is not practical in the very near future. Several 64 bit Python distributions containing numpy-MKL are already available, some for free. You mean EPD and AnacondaCE? I don't think we should withhold easily available vanilla builds because they are also available in company-sponsored projects. Python.org provides windows builds even though ActiveState is free-as-in-beer. If the company-sponsored bit bothers you, there's also a 64-bit Python(x,y) now. I'm sure you've seen that the question 'where are the 64-bit installers' comes up often for Numpy. It seems to me that we'd have to have a good reason not provide them. There will always be some number of people like me who like to install the various parts by hand, and don't like using large packages, free or or open or not. For example, I don't use macports. At the moment, the reasons I see you are giving are: 1) Then everyone would have to provide 64-bit binaries Indeed. And since many packages won't do that because there's no free compilers, users just get stuck a bit later in the installing the stack process. I'm sure
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
Hi, On Wed, Feb 6, 2013 at 9:20 PM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: On 02/07/2013 12:16 AM, Matthew Brett wrote: Hi, On Wed, Feb 6, 2013 at 1:36 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Wed, Feb 6, 2013 at 8:46 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Wed, Feb 6, 2013 at 10:48 AM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Wed, Feb 6, 2013 at 1:32 AM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Tue, Feb 5, 2013 at 3:04 PM, Christoph Gohlke cgoh...@uci.edu wrote: On 2/5/2013 10:51 AM, Matthew Brett wrote: Hi, On Mon, Feb 4, 2013 at 5:09 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Feb 4, 2013 at 3:46 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Mon, Feb 4, 2013 at 4:04 PM, Robert Kern robert.k...@gmail.com wrote: On Mon, Feb 4, 2013 at 10:38 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Feb 4, 2013 at 1:15 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: MSVC + Intel Fortran + MKL, yes. But those aren't free. So how can you provide an Amazon image for those? You can make an image that is not public, I guess. I suppose anyone who uses the image would have to have their own licenses for the Intel stuff? Does anyone have experience of this? You need to purchase one license per developer: http://software.intel.com/en-us/articles/intel-math-kernel-library-licensing-faq#eula1 I think 64 bits on windows is best pushed off to 1.7.1 or 1.8. It would be a bit much to get it implemented in the next week or two. The problem with not providing these binaries is that they are at the bottom of everyone's stack, so a delay in numpy holds everyone back. I can't find completely convincing stats, but it looks as though 64 bit windows 7 is now the most common version of Windows, at least for Gamers [1] around now, and it was getting that way for everyone in 2010 [2]. It don't think it reflects well on on us that we don't appear to support 64 bits out of the box; just for example, R already has a 32 bit / 64 bit installer. If I understand correctly, the options for doing this right now are: 1) Minimal cost in time : ask Christophe nicely whether we can distribute his binaries via the Numpy page 2) Small cost in time / money : pay for licenses for Ondrej or me or someone to install the dependencies on my Berkeley machine / an Amazon image. In order not to leave this discussion without a resolution: Christophe - would you allow us to distribute your numpy binaries for 1.7 from the numpy sourceforge page? Cheers, Matthew I am OK with providing 64 bit numpy-MKL binaries (that is numpy compiled with MSVC compilers and linked to Intel's MKL) for official numpy releases. Thank you - that is great. However: 1) There seems to be no real consensus and urge for doing this. I certainly feel the urge and feel it strongly. As a packager for two or three projects myself, it's a royal pain having to tell someone to go to two different places for binaries depending on the number of bits of their Windows. If you're relying on .exe installers on SF, then you have to send your users to more places than that probably. Really the separate installers are a poor alternative to the available scientific distributions. And the more packages you need as a user, the more annoying these separate installers are. I think Chuck was worried about the time it would take to do it, and I think you've already solved this problem. Ralf was worried about Scipy - see below. Not just Scipy - that would be my worry number one, but the same holds for all packages based on Numpy. You double the number of Windows installers that every single project needs to provide. Using a free toolchain capable of building the whole scipy-stack would be much preferred. That's true, but there seems general agreement this is not practical in the very near future. Several 64 bit Python distributions containing numpy-MKL are already available, some for free. You mean EPD and AnacondaCE? I don't think we should withhold easily available vanilla builds because they are also available in company-sponsored projects. Python.org provides windows builds even though ActiveState is free-as-in-beer. If the company-sponsored bit bothers you, there's also a 64-bit Python(x,y) now. I'm sure you've seen that the question 'where are the 64-bit installers' comes up often for Numpy. It seems to me that we'd have to have a good reason not provide them. There will always be some number of people like me who like to install the various parts by hand, and don't like using large packages, free or or open or not. For example, I don't use macports. At the moment, the reasons I see you are giving are: 1) Then everyone would have to provide 64-bit binaries Indeed. And since many packages won't do that because there's no free
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
On Wed, Feb 6, 2013 at 9:20 PM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: On 02/07/2013 12:16 AM, Matthew Brett wrote: [...] Can you clarify the people you think will get stuck? I think I'm right in saying that anyone with a C extension should be able to build them against numpy, by installing the free (as-in-beer) MS tools? So do you just mean people needing a Fortran compiler? That's a small constituency, I think. Off the top of my head there's SciPy and pymc... Anyway, I'm butting in because I wish this discussion could separate between the user perspective and the developer perspective. FWIW, 1) From a user's perspective, I don't understand this either. If you are already using a closed source, not-free-as-in-beer operating system, why would you not use (or buy!) a closed source, not-free-as-in-beer Fortran compiler? Indeed. Though I really have no clue on the Windows use cases. Maybe most Windows users don't want to compile anything, just use numpy and scipy from Python? 2) BUT, the argument I've seen that I can at least understand is that the release manager should be able to do a release using only open source tools (even using Wine instead of Windows) and not rely on a limited number of licenses. And that the release manager must be able to perform all the official builds directly. As the release manager, I really only have two requirements: * I want to ssh in there from my Ubuntu * I want to automate the whole process For Mac, linux and Wine I can do that. So I have just spend few hours browsing the net and it looks like that the combination of Windows PowerShell 2.0: http://en.wikipedia.org/wiki/Windows_PowerShell and some SSH server, there are quite a few, one commercial but free for one user one connection (perfect for me!): http://www.powershellinside.com/powershell/ssh/ So if I understand the pages correctly, I can login there from linux, and then I use the PowerShell commands to script anything. It looks like I can even use my Fabric fabfiles with powershell: https://gist.github.com/diyan/2850866 I can also use git with PowerShell: http://windows.github.com/ http://haacked.com/archive/2011/12/13/better-git-with-powershell.aspx So the final problem is how to execute MSVC and Fortran from Power Shell on Windows. These links might help for MSVC: http://stackoverflow.com/questions/4398136/use-powershell-for-visual-studio-command-prompt http://geekswithblogs.net/dwdii/archive/2011/05/20/automating-a-visual-studio-build-with-powershell---part-1.aspx Finally, for Intel Fortran + powershell: http://software.intel.com/en-us/forums/topic/284425 So I think it is all possible. If somebody can provide a machine with Windows, MSVC, PowerShell2.0, SSH server and some Fortran compiler, it should be possible for me to automate everything from Ubuntu using my Fabric files (https://github.com/certik/numpy-vendor). Ondrej ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
Christoph, On Tue, Feb 5, 2013 at 3:04 PM, Christoph Gohlke cgoh...@uci.edu wrote: [...] In order not to leave this discussion without a resolution: Christophe - would you allow us to distribute your numpy binaries for 1.7 from the numpy sourceforge page? Cheers, Matthew I am OK with providing 64 bit numpy-MKL binaries (that is numpy compiled with MSVC compilers and linked to Intel's MKL) for official numpy releases. However: 1) There seems to be no real consensus and urge for doing this. Using a free toolchain capable of building the whole scipy-stack would be much preferred. Several 64 bit Python distributions containing numpy-MKL are already available, some for free. 2) Releasing 64 bit numpy without matching scipy binaries would make little sense to me. 3) Please do not just redistribute the binaries from my website and declare them official. They might contain unreleased fixes from git master and pull requests that are needed for my work and other packages. 4) Numpy-MKL requires the Intel runtime DLLs (MKL is linked statically btw). I ship those with the installers and append the directory containing the DLLs to os.environ['PATH'] in numpy/__init__.py. This is a big no-no according to numpy developers. I don't agree. Anyway, those changes are not in the numpy source repositories. 5) My numpy-MKL installers are Python distutils bdist_wininst installers. That means if Python was installed for all users, installing numpy-MKL on Windows 6.0 will prompt for UAC elevation. Another no-no? I think that all these things should be possible to fix so that the binary is acceptable for the official NumPy binary. How exactly do you build the binaries? I wasn't able to find the info at: http://www.lfd.uci.edu/~gohlke/pythonlibs/ Do you have some scripts to do that? Do you use PowerShell? Or you do it by hand by mouse and clicks in Visual Studio somehow? If I can figure out how to do these builds, I'll be happy to figure out how to automate it and then we can try to figure out a solution that works for NumPy. Ondrej ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
Hi, On Wed, Feb 6, 2013 at 10:21 PM, Ondřej Čertík ondrej.cer...@gmail.com wrote: On Wed, Feb 6, 2013 at 9:20 PM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: On 02/07/2013 12:16 AM, Matthew Brett wrote: [...] Can you clarify the people you think will get stuck? I think I'm right in saying that anyone with a C extension should be able to build them against numpy, by installing the free (as-in-beer) MS tools? So do you just mean people needing a Fortran compiler? That's a small constituency, I think. Off the top of my head there's SciPy and pymc... Anyway, I'm butting in because I wish this discussion could separate between the user perspective and the developer perspective. FWIW, 1) From a user's perspective, I don't understand this either. If you are already using a closed source, not-free-as-in-beer operating system, why would you not use (or buy!) a closed source, not-free-as-in-beer Fortran compiler? Indeed. Though I really have no clue on the Windows use cases. Maybe most Windows users don't want to compile anything, just use numpy and scipy from Python? Well - yes - as a packager I really want to be able to provide a binary so my binary consumers don't have to have a C compiler installed. I imagine it's the same for all of us packagers out there. 2) BUT, the argument I've seen that I can at least understand is that the release manager should be able to do a release using only open source tools (even using Wine instead of Windows) and not rely on a limited number of licenses. And that the release manager must be able to perform all the official builds directly. As the release manager, I really only have two requirements: * I want to ssh in there from my Ubuntu * I want to automate the whole process For Mac, linux and Wine I can do that. So I have just spend few hours browsing the net and it looks like that the combination of Windows PowerShell 2.0: http://en.wikipedia.org/wiki/Windows_PowerShell and some SSH server, there are quite a few, one commercial but free for one user one connection (perfect for me!): http://www.powershellinside.com/powershell/ssh/ So if I understand the pages correctly, I can login there from linux, and then I use the PowerShell commands to script anything. It looks like I can even use my Fabric fabfiles with powershell: https://gist.github.com/diyan/2850866 I can also use git with PowerShell: http://windows.github.com/ http://haacked.com/archive/2011/12/13/better-git-with-powershell.aspx So the final problem is how to execute MSVC and Fortran from Power Shell on Windows. These links might help for MSVC: http://stackoverflow.com/questions/4398136/use-powershell-for-visual-studio-command-prompt http://geekswithblogs.net/dwdii/archive/2011/05/20/automating-a-visual-studio-build-with-powershell---part-1.aspx Finally, for Intel Fortran + powershell: http://software.intel.com/en-us/forums/topic/284425 So I think it is all possible. If somebody can provide a machine with Windows, MSVC, PowerShell2.0, SSH server and some Fortran compiler, it should be possible for me to automate everything from Ubuntu using my Fabric files (https://github.com/certik/numpy-vendor). Many many thanks for trying to solve this. I had really started to give up hope. I think you will need a developer's license for MKL for Numpy. Ralf - any ETA for those? I think I'm right in thinking you'll need a Fortran compiler for Scipy but not Numpy? Can we defer the Scipy build until after the Numpy build? I will try to get you set up with ssh on my Windows 7 machine in case you can use it. It has the MS tools. Thanks again, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
On 02/04/2013 06:09 PM, Matthew Brett wrote: The problem with not providing these binaries is that they are at the bottom of everyone's stack, so a delay in numpy holds everyone back. OTOH, so far it's been an *excellent* excuse for those of us further up the stack not to make a 64-bit binary. It sounds like we're completely out of luck on 64-bit Windows with free tools? So the rest of the community is going to face shelling out for compiler licenses as well? -- Jonathan Niehof ISR-3 Space Data Systems Los Alamos National Laboratory MS-D466 Los Alamos, NM 87545 Phone: 505-667-9595 email: jnie...@lanl.gov Correspondence / Technical data or Software Publicly Available ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
Hi, On Tue, Feb 5, 2013 at 6:03 AM, Jonathan T. Niehof jnie...@lanl.gov wrote: On 02/04/2013 06:09 PM, Matthew Brett wrote: The problem with not providing these binaries is that they are at the bottom of everyone's stack, so a delay in numpy holds everyone back. OTOH, so far it's been an *excellent* excuse for those of us further up the stack not to make a 64-bit binary. It sounds like we're completely out of luck on 64-bit Windows with free tools? So the rest of the community is going to face shelling out for compiler licenses as well? Normally you'd only need the free (as in beer) MS C compilers. You'd need a (possibly free) MKL license if you wanted to link against an optimized blas / lapack (it seems), and you'd need to use the Intel ifort compiler if you had fortran code in there (I think). Cheers, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
Hi, On Mon, Feb 4, 2013 at 5:09 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Feb 4, 2013 at 3:46 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Mon, Feb 4, 2013 at 4:04 PM, Robert Kern robert.k...@gmail.com wrote: On Mon, Feb 4, 2013 at 10:38 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Feb 4, 2013 at 1:15 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: MSVC + Intel Fortran + MKL, yes. But those aren't free. So how can you provide an Amazon image for those? You can make an image that is not public, I guess. I suppose anyone who uses the image would have to have their own licenses for the Intel stuff? Does anyone have experience of this? You need to purchase one license per developer: http://software.intel.com/en-us/articles/intel-math-kernel-library-licensing-faq#eula1 I think 64 bits on windows is best pushed off to 1.7.1 or 1.8. It would be a bit much to get it implemented in the next week or two. The problem with not providing these binaries is that they are at the bottom of everyone's stack, so a delay in numpy holds everyone back. I can't find completely convincing stats, but it looks as though 64 bit windows 7 is now the most common version of Windows, at least for Gamers [1] around now, and it was getting that way for everyone in 2010 [2]. It don't think it reflects well on on us that we don't appear to support 64 bits out of the box; just for example, R already has a 32 bit / 64 bit installer. If I understand correctly, the options for doing this right now are: 1) Minimal cost in time : ask Christophe nicely whether we can distribute his binaries via the Numpy page 2) Small cost in time / money : pay for licenses for Ondrej or me or someone to install the dependencies on my Berkeley machine / an Amazon image. In order not to leave this discussion without a resolution: Christophe - would you allow us to distribute your numpy binaries for 1.7 from the numpy sourceforge page? Cheers, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
On 2/5/2013 10:51 AM, Matthew Brett wrote: Hi, On Mon, Feb 4, 2013 at 5:09 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Feb 4, 2013 at 3:46 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Mon, Feb 4, 2013 at 4:04 PM, Robert Kern robert.k...@gmail.com wrote: On Mon, Feb 4, 2013 at 10:38 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Feb 4, 2013 at 1:15 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: MSVC + Intel Fortran + MKL, yes. But those aren't free. So how can you provide an Amazon image for those? You can make an image that is not public, I guess. I suppose anyone who uses the image would have to have their own licenses for the Intel stuff? Does anyone have experience of this? You need to purchase one license per developer: http://software.intel.com/en-us/articles/intel-math-kernel-library-licensing-faq#eula1 I think 64 bits on windows is best pushed off to 1.7.1 or 1.8. It would be a bit much to get it implemented in the next week or two. The problem with not providing these binaries is that they are at the bottom of everyone's stack, so a delay in numpy holds everyone back. I can't find completely convincing stats, but it looks as though 64 bit windows 7 is now the most common version of Windows, at least for Gamers [1] around now, and it was getting that way for everyone in 2010 [2]. It don't think it reflects well on on us that we don't appear to support 64 bits out of the box; just for example, R already has a 32 bit / 64 bit installer. If I understand correctly, the options for doing this right now are: 1) Minimal cost in time : ask Christophe nicely whether we can distribute his binaries via the Numpy page 2) Small cost in time / money : pay for licenses for Ondrej or me or someone to install the dependencies on my Berkeley machine / an Amazon image. In order not to leave this discussion without a resolution: Christophe - would you allow us to distribute your numpy binaries for 1.7 from the numpy sourceforge page? Cheers, Matthew I am OK with providing 64 bit numpy-MKL binaries (that is numpy compiled with MSVC compilers and linked to Intel's MKL) for official numpy releases. However: 1) There seems to be no real consensus and urge for doing this. Using a free toolchain capable of building the whole scipy-stack would be much preferred. Several 64 bit Python distributions containing numpy-MKL are already available, some for free. 2) Releasing 64 bit numpy without matching scipy binaries would make little sense to me. 3) Please do not just redistribute the binaries from my website and declare them official. They might contain unreleased fixes from git master and pull requests that are needed for my work and other packages. 4) Numpy-MKL requires the Intel runtime DLLs (MKL is linked statically btw). I ship those with the installers and append the directory containing the DLLs to os.environ['PATH'] in numpy/__init__.py. This is a big no-no according to numpy developers. I don't agree. Anyway, those changes are not in the numpy source repositories. 5) My numpy-MKL installers are Python distutils bdist_wininst installers. That means if Python was installed for all users, installing numpy-MKL on Windows 6.0 will prompt for UAC elevation. Another no-no? Christoph ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
Hi, On Tue, Feb 5, 2013 at 3:04 PM, Christoph Gohlke cgoh...@uci.edu wrote: On 2/5/2013 10:51 AM, Matthew Brett wrote: Hi, On Mon, Feb 4, 2013 at 5:09 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Feb 4, 2013 at 3:46 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Mon, Feb 4, 2013 at 4:04 PM, Robert Kern robert.k...@gmail.com wrote: On Mon, Feb 4, 2013 at 10:38 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Feb 4, 2013 at 1:15 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: MSVC + Intel Fortran + MKL, yes. But those aren't free. So how can you provide an Amazon image for those? You can make an image that is not public, I guess. I suppose anyone who uses the image would have to have their own licenses for the Intel stuff? Does anyone have experience of this? You need to purchase one license per developer: http://software.intel.com/en-us/articles/intel-math-kernel-library-licensing-faq#eula1 I think 64 bits on windows is best pushed off to 1.7.1 or 1.8. It would be a bit much to get it implemented in the next week or two. The problem with not providing these binaries is that they are at the bottom of everyone's stack, so a delay in numpy holds everyone back. I can't find completely convincing stats, but it looks as though 64 bit windows 7 is now the most common version of Windows, at least for Gamers [1] around now, and it was getting that way for everyone in 2010 [2]. It don't think it reflects well on on us that we don't appear to support 64 bits out of the box; just for example, R already has a 32 bit / 64 bit installer. If I understand correctly, the options for doing this right now are: 1) Minimal cost in time : ask Christophe nicely whether we can distribute his binaries via the Numpy page 2) Small cost in time / money : pay for licenses for Ondrej or me or someone to install the dependencies on my Berkeley machine / an Amazon image. In order not to leave this discussion without a resolution: Christophe - would you allow us to distribute your numpy binaries for 1.7 from the numpy sourceforge page? Cheers, Matthew I am OK with providing 64 bit numpy-MKL binaries (that is numpy compiled with MSVC compilers and linked to Intel's MKL) for official numpy releases. Thank you - that is great. However: 1) There seems to be no real consensus and urge for doing this. I certainly feel the urge and feel it strongly. As a packager for two or three projects myself, it's a royal pain having to tell someone to go to two different places for binaries depending on the number of bits of their Windows. I think Chuck was worried about the time it would take to do it, and I think you've already solved this problem. Ralf was worried about Scipy - see below. Using a free toolchain capable of building the whole scipy-stack would be much preferred. That's true, but there seems general agreement this is not practical in the very near future. Several 64 bit Python distributions containing numpy-MKL are already available, some for free. You mean EPD and AnacondaCE? I don't think we should withhold easily available vanilla builds because they are also available in company-sponsored projects. Python.org provides windows builds even though ActiveState is free-as-in-beer. 2) Releasing 64 bit numpy without matching scipy binaries would make little sense to me. Would you consider also releasing your scipy binaries? 3) Please do not just redistribute the binaries from my website and declare them official. They might contain unreleased fixes from git master and pull requests that are needed for my work and other packages. Right - would you consider then being the release provider for numpy / scipy binaries on windows, much as it appears that Martin v Lowis supplies Windows builds for Python? 4) Numpy-MKL requires the Intel runtime DLLs (MKL is linked statically btw). I ship those with the installers and append the directory containing the DLLs to os.environ['PATH'] in numpy/__init__.py. This is a big no-no according to numpy developers. I don't agree. Anyway, those changes are not in the numpy source repositories. 5) My numpy-MKL installers are Python distutils bdist_wininst installers. That means if Python was installed for all users, installing numpy-MKL on Windows 6.0 will prompt for UAC elevation. Another no-no? I defer to others on these ones, Thanks a lot, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
On Tue, Feb 5, 2013 at 4:32 PM, Matthew Brett matthew.br...@gmail.com wrote: 4) Numpy-MKL requires the Intel runtime DLLs (MKL is linked statically btw). I ship those with the installers and append the directory containing the DLLs to os.environ['PATH'] in numpy/__init__.py. This is a big no-no according to numpy developers. I don't agree. Anyway, those changes are not in the numpy source repositories. I think you pointed out that another option is to load the dlls with ctypes -- is it much work to make that change? 5) My numpy-MKL installers are Python distutils bdist_wininst installers. That means if Python was installed for all users, installing numpy-MKL on Windows 6.0 will prompt for UAC elevation. Another no-no? not sure about the UAC elevation -- but: 1) most folks use bdist_wininst for Windows binaries -- including the current numpy builds, and python.org python -- yes? 2) UAC aside, It would be great to have binaries that could be used with virtualenv -- binary eggs? -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
Hi, On Tue, Feb 5, 2013 at 4:55 PM, Chris Barker - NOAA Federal chris.bar...@noaa.gov wrote: On Tue, Feb 5, 2013 at 4:32 PM, Matthew Brett matthew.br...@gmail.com wrote: 4) Numpy-MKL requires the Intel runtime DLLs (MKL is linked statically btw). I ship those with the installers and append the directory containing the DLLs to os.environ['PATH'] in numpy/__init__.py. This is a big no-no according to numpy developers. I don't agree. Anyway, those changes are not in the numpy source repositories. I think you pointed out that another option is to load the dlls with ctypes -- is it much work to make that change? 5) My numpy-MKL installers are Python distutils bdist_wininst installers. That means if Python was installed for all users, installing numpy-MKL on Windows 6.0 will prompt for UAC elevation. Another no-no? not sure about the UAC elevation -- but: 1) most folks use bdist_wininst for Windows binaries -- including the current numpy builds, and python.org python -- yes? 2) UAC aside, It would be great to have binaries that could be used with virtualenv -- binary eggs? easy_install can install into virtualenvs from bdist_wininst installers, at least the ones I have built... See you, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
On Sun, Feb 3, 2013 at 2:57 AM, David Cournapeau courn...@gmail.com wrote: On Sun, Feb 3, 2013 at 12:28 AM, josef.p...@gmail.com wrote: On Sat, Feb 2, 2013 at 6:14 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, I see there is no Windows 64 bit installer for the 1.7 rc1. related: Is there any chance to get newer mingw or mingw-w64 support soonish? The problem has no solution until we can restrict support to windows 7 and above. Otherwise, any acceptable solution would require user to be an admin. The installer is built with this VM/scripts: https://github.com/certik/numpy-vendor currently the VM itself is 32 bit. I think that might be upgraded to 64bit, and maybe it's possible to use 64 bit Wine: http://wiki.winehq.org/Wine64 but then we would need to figure out how to use Mingw with 64 bits. I would be very happy to accept patches to the above repository. Alternatively, if the actual Windows 64bit machine would have to be used, is there any way to automate the process? Would you compile it from command line (cmd.exe), just like I do in Wine? I would much prefer if we can figure out how to do this in Wine, so that the process can be automated and other people can easily reproduce it. Ondrej ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
Hi, On Mon, Feb 4, 2013 at 12:27 PM, Ondřej Čertík ondrej.cer...@gmail.com wrote: On Sun, Feb 3, 2013 at 2:57 AM, David Cournapeau courn...@gmail.com wrote: On Sun, Feb 3, 2013 at 12:28 AM, josef.p...@gmail.com wrote: On Sat, Feb 2, 2013 at 6:14 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, I see there is no Windows 64 bit installer for the 1.7 rc1. related: Is there any chance to get newer mingw or mingw-w64 support soonish? The problem has no solution until we can restrict support to windows 7 and above. Otherwise, any acceptable solution would require user to be an admin. The installer is built with this VM/scripts: https://github.com/certik/numpy-vendor currently the VM itself is 32 bit. I think that might be upgraded to 64bit, and maybe it's possible to use 64 bit Wine: http://wiki.winehq.org/Wine64 but then we would need to figure out how to use Mingw with 64 bits. I would be very happy to accept patches to the above repository. Alternatively, if the actual Windows 64bit machine would have to be used, is there any way to automate the process? Would you compile it from command line (cmd.exe), just like I do in Wine? I would much prefer if we can figure out how to do this in Wine, so that the process can be automated and other people can easily reproduce it. I wonder whether getting ming64 to work on 64 bit Wine is too hard to get working before the release? I often can't get 32-bit Wine working, and we've apparently got problems with mingw64 on native windows. As a short term fix, how about an Amazon image with the Windows 64 bit compilers on it? Or can Christophe Gohlke help us out here? It seems a shame not to provide these builds. Cheers, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
On Mon, Feb 4, 2013 at 12:36 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Feb 4, 2013 at 12:27 PM, Ondřej Čertík ondrej.cer...@gmail.com wrote: On Sun, Feb 3, 2013 at 2:57 AM, David Cournapeau courn...@gmail.com wrote: On Sun, Feb 3, 2013 at 12:28 AM, josef.p...@gmail.com wrote: On Sat, Feb 2, 2013 at 6:14 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, I see there is no Windows 64 bit installer for the 1.7 rc1. related: Is there any chance to get newer mingw or mingw-w64 support soonish? The problem has no solution until we can restrict support to windows 7 and above. Otherwise, any acceptable solution would require user to be an admin. The installer is built with this VM/scripts: https://github.com/certik/numpy-vendor currently the VM itself is 32 bit. I think that might be upgraded to 64bit, and maybe it's possible to use 64 bit Wine: http://wiki.winehq.org/Wine64 but then we would need to figure out how to use Mingw with 64 bits. I would be very happy to accept patches to the above repository. Alternatively, if the actual Windows 64bit machine would have to be used, is there any way to automate the process? Would you compile it from command line (cmd.exe), just like I do in Wine? I would much prefer if we can figure out how to do this in Wine, so that the process can be automated and other people can easily reproduce it. I wonder whether getting ming64 to work on 64 bit Wine is too hard to get working before the release? I often can't get 32-bit Wine working, and we've apparently got problems with mingw64 on native windows. As a short term fix, how about an Amazon image with the Windows 64 bit compilers on it? Or can Christophe Gohlke help us out here? As a temporary measure it might make sense to just deem the cgolke builds official and upload them to the usual places -- or at least, it seems like it might make sense to me, but it depends on what Christophe thinks :-). -n ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
On Mon, Feb 4, 2013 at 9:39 PM, Nathaniel Smith n...@pobox.com wrote: On Mon, Feb 4, 2013 at 12:36 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Feb 4, 2013 at 12:27 PM, Ondřej Čertík ondrej.cer...@gmail.com wrote: On Sun, Feb 3, 2013 at 2:57 AM, David Cournapeau courn...@gmail.com wrote: On Sun, Feb 3, 2013 at 12:28 AM, josef.p...@gmail.com wrote: On Sat, Feb 2, 2013 at 6:14 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, I see there is no Windows 64 bit installer for the 1.7 rc1. related: Is there any chance to get newer mingw or mingw-w64 support soonish? The problem has no solution until we can restrict support to windows 7 and above. Otherwise, any acceptable solution would require user to be an admin. The installer is built with this VM/scripts: https://github.com/certik/numpy-vendor currently the VM itself is 32 bit. I think that might be upgraded to 64bit, and maybe it's possible to use 64 bit Wine: http://wiki.winehq.org/Wine64 but then we would need to figure out how to use Mingw with 64 bits. I would be very happy to accept patches to the above repository. Alternatively, if the actual Windows 64bit machine would have to be used, is there any way to automate the process? Would you compile it from command line (cmd.exe), just like I do in Wine? I would much prefer if we can figure out how to do this in Wine, so that the process can be automated and other people can easily reproduce it. I wonder whether getting ming64 to work on 64 bit Wine is too hard to get working before the release? I often can't get 32-bit Wine working, and we've apparently got problems with mingw64 on native windows. As a short term fix, how about an Amazon image with the Windows 64 bit compilers on it? -1 on providing an official solution which will require admin rights for the produced installer and not work for scipy. Or can Christophe Gohlke help us out here? As a temporary measure it might make sense to just deem the cgolke builds official and upload them to the usual places -- or at least, it seems like it might make sense to me, but it depends on what Christophe thinks :-). I'm +0 on that. MSVC + MKL is currently the only real option for 64-bit binaries, so providing Christoph's installers as official could make sense. On the other hand, Christoph has a matching set of other packages on his site, so users will be better off being redirected there than just grabbing only a numpy installer from SF and not finding a scipy one there. Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
On Mon, Feb 4, 2013 at 12:36 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Feb 4, 2013 at 12:27 PM, Ondřej Čertík ondrej.cer...@gmail.com wrote: On Sun, Feb 3, 2013 at 2:57 AM, David Cournapeau courn...@gmail.com wrote: On Sun, Feb 3, 2013 at 12:28 AM, josef.p...@gmail.com wrote: On Sat, Feb 2, 2013 at 6:14 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, I see there is no Windows 64 bit installer for the 1.7 rc1. related: Is there any chance to get newer mingw or mingw-w64 support soonish? The problem has no solution until we can restrict support to windows 7 and above. Otherwise, any acceptable solution would require user to be an admin. The installer is built with this VM/scripts: https://github.com/certik/numpy-vendor currently the VM itself is 32 bit. I think that might be upgraded to 64bit, and maybe it's possible to use 64 bit Wine: http://wiki.winehq.org/Wine64 but then we would need to figure out how to use Mingw with 64 bits. I would be very happy to accept patches to the above repository. Alternatively, if the actual Windows 64bit machine would have to be used, is there any way to automate the process? Would you compile it from command line (cmd.exe), just like I do in Wine? I would much prefer if we can figure out how to do this in Wine, so that the process can be automated and other people can easily reproduce it. I wonder whether getting ming64 to work on 64 bit Wine is too hard to get working before the release? I often can't get 32-bit Wine working, Yep, it took me over a week of work to figure out exactly how to get 32-bit Wine working with mingw and numpy. However, the work is done and you can either run my scripts in the VM, or you can easily reproduce it yourself by consulting the scripts: https://github.com/certik/numpy-vendor/blob/master/setup-wine.sh https://github.com/certik/numpy-vendor/blob/master/fabfile.py There are a lot of little tricks involved. However, as Ralf says, apparently we would need to use MSVC + MKL anyway on 64bits. Ondrej ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
On Mon, Feb 4, 2013 at 8:27 PM, Ondřej Čertík ondrej.cer...@gmail.com wrote: On Sun, Feb 3, 2013 at 2:57 AM, David Cournapeau courn...@gmail.com wrote: On Sun, Feb 3, 2013 at 12:28 AM, josef.p...@gmail.com wrote: On Sat, Feb 2, 2013 at 6:14 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, I see there is no Windows 64 bit installer for the 1.7 rc1. related: Is there any chance to get newer mingw or mingw-w64 support soonish? The problem has no solution until we can restrict support to windows 7 and above. Otherwise, any acceptable solution would require user to be an admin. The installer is built with this VM/scripts: I am not sure whether you're replying to my observation or just giving a status update: with mingw-w64 (or recent mingw), the built installer will depend on several .dll (libgcc_s_sjil.dll) that we can't easily distribute. The only place we can realistically put them is in C:\Python$VERSION (or wherever python happens to be installed), and I think it is a very bad idea to install dll from NumPy there. In Windows 2008 and above, one can refer in .pyd where to look for dlls in another directory which is private to numpy. David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
Hi, On Mon, Feb 4, 2013 at 12:55 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Mon, Feb 4, 2013 at 9:39 PM, Nathaniel Smith n...@pobox.com wrote: On Mon, Feb 4, 2013 at 12:36 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Feb 4, 2013 at 12:27 PM, Ondřej Čertík ondrej.cer...@gmail.com wrote: On Sun, Feb 3, 2013 at 2:57 AM, David Cournapeau courn...@gmail.com wrote: On Sun, Feb 3, 2013 at 12:28 AM, josef.p...@gmail.com wrote: On Sat, Feb 2, 2013 at 6:14 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, I see there is no Windows 64 bit installer for the 1.7 rc1. related: Is there any chance to get newer mingw or mingw-w64 support soonish? The problem has no solution until we can restrict support to windows 7 and above. Otherwise, any acceptable solution would require user to be an admin. The installer is built with this VM/scripts: https://github.com/certik/numpy-vendor currently the VM itself is 32 bit. I think that might be upgraded to 64bit, and maybe it's possible to use 64 bit Wine: http://wiki.winehq.org/Wine64 but then we would need to figure out how to use Mingw with 64 bits. I would be very happy to accept patches to the above repository. Alternatively, if the actual Windows 64bit machine would have to be used, is there any way to automate the process? Would you compile it from command line (cmd.exe), just like I do in Wine? I would much prefer if we can figure out how to do this in Wine, so that the process can be automated and other people can easily reproduce it. I wonder whether getting ming64 to work on 64 bit Wine is too hard to get working before the release? I often can't get 32-bit Wine working, and we've apparently got problems with mingw64 on native windows. As a short term fix, how about an Amazon image with the Windows 64 bit compilers on it? -1 on providing an official solution which will require admin rights for the produced installer and not work for scipy. Sorry if I am being slow, but I don't follow. Am I right in thinking that we can currently build numpy 64 bit installers with the Microsoft tools, and that these would be distributable without admin rights for windows =XP ? Why would this solution not work for scipy? Cheers, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
On Mon, Feb 4, 2013 at 10:06 PM, Matthew Brett matthew.br...@gmail.comwrote: Hi, On Mon, Feb 4, 2013 at 12:55 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Mon, Feb 4, 2013 at 9:39 PM, Nathaniel Smith n...@pobox.com wrote: On Mon, Feb 4, 2013 at 12:36 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Feb 4, 2013 at 12:27 PM, Ondřej Čertík ondrej.cer...@gmail.com wrote: On Sun, Feb 3, 2013 at 2:57 AM, David Cournapeau courn...@gmail.com wrote: On Sun, Feb 3, 2013 at 12:28 AM, josef.p...@gmail.com wrote: On Sat, Feb 2, 2013 at 6:14 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, I see there is no Windows 64 bit installer for the 1.7 rc1. related: Is there any chance to get newer mingw or mingw-w64 support soonish? The problem has no solution until we can restrict support to windows 7 and above. Otherwise, any acceptable solution would require user to be an admin. The installer is built with this VM/scripts: https://github.com/certik/numpy-vendor currently the VM itself is 32 bit. I think that might be upgraded to 64bit, and maybe it's possible to use 64 bit Wine: http://wiki.winehq.org/Wine64 but then we would need to figure out how to use Mingw with 64 bits. I would be very happy to accept patches to the above repository. Alternatively, if the actual Windows 64bit machine would have to be used, is there any way to automate the process? Would you compile it from command line (cmd.exe), just like I do in Wine? I would much prefer if we can figure out how to do this in Wine, so that the process can be automated and other people can easily reproduce it. I wonder whether getting ming64 to work on 64 bit Wine is too hard to get working before the release? I often can't get 32-bit Wine working, and we've apparently got problems with mingw64 on native windows. As a short term fix, how about an Amazon image with the Windows 64 bit compilers on it? -1 on providing an official solution which will require admin rights for the produced installer and not work for scipy. Sorry if I am being slow, but I don't follow. Am I right in thinking that we can currently build numpy 64 bit installers with the Microsoft tools, and that these would be distributable without admin rights for windows =XP ? MSVC + Intel Fortran + MKL, yes. But those aren't free. So how can you provide an Amazon image for those? Why would this solution not work for scipy? It would. gfortran doesn't. Looking at your mail, I still read it as providing an image with mingw64 + gfortran. Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
On Mon, Feb 4, 2013 at 4:15 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Mon, Feb 4, 2013 at 10:06 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Feb 4, 2013 at 12:55 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Mon, Feb 4, 2013 at 9:39 PM, Nathaniel Smith n...@pobox.com wrote: On Mon, Feb 4, 2013 at 12:36 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Feb 4, 2013 at 12:27 PM, Ondřej Čertík ondrej.cer...@gmail.com wrote: On Sun, Feb 3, 2013 at 2:57 AM, David Cournapeau courn...@gmail.com wrote: On Sun, Feb 3, 2013 at 12:28 AM, josef.p...@gmail.com wrote: On Sat, Feb 2, 2013 at 6:14 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, I see there is no Windows 64 bit installer for the 1.7 rc1. related: Is there any chance to get newer mingw or mingw-w64 support soonish? The problem has no solution until we can restrict support to windows 7 and above. Otherwise, any acceptable solution would require user to be an admin. The installer is built with this VM/scripts: https://github.com/certik/numpy-vendor currently the VM itself is 32 bit. I think that might be upgraded to 64bit, and maybe it's possible to use 64 bit Wine: http://wiki.winehq.org/Wine64 but then we would need to figure out how to use Mingw with 64 bits. I would be very happy to accept patches to the above repository. Alternatively, if the actual Windows 64bit machine would have to be used, is there any way to automate the process? Would you compile it from command line (cmd.exe), just like I do in Wine? I would much prefer if we can figure out how to do this in Wine, so that the process can be automated and other people can easily reproduce it. I wonder whether getting ming64 to work on 64 bit Wine is too hard to get working before the release? I often can't get 32-bit Wine working, and we've apparently got problems with mingw64 on native windows. As a short term fix, how about an Amazon image with the Windows 64 bit compilers on it? -1 on providing an official solution which will require admin rights for the produced installer and not work for scipy. Sorry if I am being slow, but I don't follow. Am I right in thinking that we can currently build numpy 64 bit installers with the Microsoft tools, and that these would be distributable without admin rights for windows =XP ? MSVC + Intel Fortran + MKL, yes. But those aren't free. So how can you provide an Amazon image for those? Related question Would scipy and similar packages complain when we try to build them without MKL against an MKL numpy. Gohlke has the compatibility notes to warn users. If incompatible files are on numpy sourceforge for download, then users might install by accident incompatible versions, or not? Josef Why would this solution not work for scipy? It would. gfortran doesn't. Looking at your mail, I still read it as providing an image with mingw64 + gfortran. Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
Hi, On Mon, Feb 4, 2013 at 1:15 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Mon, Feb 4, 2013 at 10:06 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Feb 4, 2013 at 12:55 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Mon, Feb 4, 2013 at 9:39 PM, Nathaniel Smith n...@pobox.com wrote: On Mon, Feb 4, 2013 at 12:36 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Feb 4, 2013 at 12:27 PM, Ondřej Čertík ondrej.cer...@gmail.com wrote: On Sun, Feb 3, 2013 at 2:57 AM, David Cournapeau courn...@gmail.com wrote: On Sun, Feb 3, 2013 at 12:28 AM, josef.p...@gmail.com wrote: On Sat, Feb 2, 2013 at 6:14 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, I see there is no Windows 64 bit installer for the 1.7 rc1. related: Is there any chance to get newer mingw or mingw-w64 support soonish? The problem has no solution until we can restrict support to windows 7 and above. Otherwise, any acceptable solution would require user to be an admin. The installer is built with this VM/scripts: https://github.com/certik/numpy-vendor currently the VM itself is 32 bit. I think that might be upgraded to 64bit, and maybe it's possible to use 64 bit Wine: http://wiki.winehq.org/Wine64 but then we would need to figure out how to use Mingw with 64 bits. I would be very happy to accept patches to the above repository. Alternatively, if the actual Windows 64bit machine would have to be used, is there any way to automate the process? Would you compile it from command line (cmd.exe), just like I do in Wine? I would much prefer if we can figure out how to do this in Wine, so that the process can be automated and other people can easily reproduce it. I wonder whether getting ming64 to work on 64 bit Wine is too hard to get working before the release? I often can't get 32-bit Wine working, and we've apparently got problems with mingw64 on native windows. As a short term fix, how about an Amazon image with the Windows 64 bit compilers on it? -1 on providing an official solution which will require admin rights for the produced installer and not work for scipy. Sorry if I am being slow, but I don't follow. Am I right in thinking that we can currently build numpy 64 bit installers with the Microsoft tools, and that these would be distributable without admin rights for windows =XP ? MSVC + Intel Fortran + MKL, yes. But those aren't free. So how can you provide an Amazon image for those? You can make an image that is not public, I guess. I suppose anyone who uses the image would have to have their own licenses for the Intel stuff? Does anyone have experience of this? Does ATLAS / openBLAS not build for windows? Sorry for my ignorance. Cheers, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
On Mon, Feb 4, 2013 at 10:38 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Feb 4, 2013 at 1:15 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: MSVC + Intel Fortran + MKL, yes. But those aren't free. So how can you provide an Amazon image for those? You can make an image that is not public, I guess. I suppose anyone who uses the image would have to have their own licenses for the Intel stuff? Does anyone have experience of this? You need to purchase one license per developer: http://software.intel.com/en-us/articles/intel-math-kernel-library-licensing-faq#eula1 -- Robert Kern ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
On Mon, Feb 4, 2013 at 4:04 PM, Robert Kern robert.k...@gmail.com wrote: On Mon, Feb 4, 2013 at 10:38 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Feb 4, 2013 at 1:15 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: MSVC + Intel Fortran + MKL, yes. But those aren't free. So how can you provide an Amazon image for those? You can make an image that is not public, I guess. I suppose anyone who uses the image would have to have their own licenses for the Intel stuff? Does anyone have experience of this? You need to purchase one license per developer: http://software.intel.com/en-us/articles/intel-math-kernel-library-licensing-faq#eula1 I think 64 bits on windows is best pushed off to 1.7.1 or 1.8. It would be a bit much to get it implemented in the next week or two. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
On 2/4/2013 12:59 PM, David Cournapeau wrote: On Mon, Feb 4, 2013 at 8:27 PM, Ondřej Čertík ondrej.cer...@gmail.com wrote: On Sun, Feb 3, 2013 at 2:57 AM, David Cournapeau courn...@gmail.com wrote: On Sun, Feb 3, 2013 at 12:28 AM, josef.p...@gmail.com wrote: On Sat, Feb 2, 2013 at 6:14 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, I see there is no Windows 64 bit installer for the 1.7 rc1. related: Is there any chance to get newer mingw or mingw-w64 support soonish? The problem has no solution until we can restrict support to windows 7 and above. Otherwise, any acceptable solution would require user to be an admin. The installer is built with this VM/scripts: I am not sure whether you're replying to my observation or just giving a status update: with mingw-w64 (or recent mingw), the built installer will depend on several .dll (libgcc_s_sjil.dll) that we can't easily distribute. The only place we can realistically put them is in C:\Python$VERSION (or wherever python happens to be installed), and I think it is a very bad idea to install dll from NumPy there. In Windows 2008 and above, one can refer in .pyd where to look for dlls in another directory which is private to numpy. David If I understand correctly the problem is distributing dependency/runtime DLLs with a package and ensuring the DLLs are found by Windows when the pyd extensions are imported? For numpy-MKL and other packages I include/install the extra DLLs in the package directories and, if necessary, (i) append the package directory to os.environ['PATH'] or (ii) pre-load the DLLs into the process using Ctypes, both early in the package's main __init__.py. No admin rights are required. Christoph ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
On Mon, Feb 4, 2013 at 3:49 PM, Christoph Gohlke cgoh...@uci.edu wrote: On 2/4/2013 12:59 PM, David Cournapeau wrote: On Mon, Feb 4, 2013 at 8:27 PM, Ondřej Čertík ondrej.cer...@gmail.com wrote: On Sun, Feb 3, 2013 at 2:57 AM, David Cournapeau courn...@gmail.com wrote: On Sun, Feb 3, 2013 at 12:28 AM, josef.p...@gmail.com wrote: On Sat, Feb 2, 2013 at 6:14 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, I see there is no Windows 64 bit installer for the 1.7 rc1. related: Is there any chance to get newer mingw or mingw-w64 support soonish? The problem has no solution until we can restrict support to windows 7 and above. Otherwise, any acceptable solution would require user to be an admin. The installer is built with this VM/scripts: I am not sure whether you're replying to my observation or just giving a status update: with mingw-w64 (or recent mingw), the built installer I was just giving a general status, sorry about not being clear. will depend on several .dll (libgcc_s_sjil.dll) that we can't easily distribute. The only place we can realistically put them is in C:\Python$VERSION (or wherever python happens to be installed), and I think it is a very bad idea to install dll from NumPy there. In Windows 2008 and above, one can refer in .pyd where to look for dlls in another directory which is private to numpy. Yes. David If I understand correctly the problem is distributing dependency/runtime DLLs with a package and ensuring the DLLs are found by Windows when the pyd extensions are imported? For numpy-MKL and other packages I include/install the extra DLLs in the package directories and, if necessary, (i) append the package directory to os.environ['PATH'] or (ii) pre-load the DLLs into the process using Ctypes, both early in the package's main __init__.py. No admin rights are required. So that seems to be the only option. Is there any other solution? Ondrej ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
On Tue, Feb 5, 2013 at 12:27 AM, Ondřej Čertík ondrej.cer...@gmail.com wrote: On Mon, Feb 4, 2013 at 3:49 PM, Christoph Gohlke cgoh...@uci.edu wrote: On 2/4/2013 12:59 PM, David Cournapeau wrote: On Mon, Feb 4, 2013 at 8:27 PM, Ondřej Čertík ondrej.cer...@gmail.com wrote: On Sun, Feb 3, 2013 at 2:57 AM, David Cournapeau courn...@gmail.com wrote: On Sun, Feb 3, 2013 at 12:28 AM, josef.p...@gmail.com wrote: On Sat, Feb 2, 2013 at 6:14 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, I see there is no Windows 64 bit installer for the 1.7 rc1. related: Is there any chance to get newer mingw or mingw-w64 support soonish? The problem has no solution until we can restrict support to windows 7 and above. Otherwise, any acceptable solution would require user to be an admin. The installer is built with this VM/scripts: I am not sure whether you're replying to my observation or just giving a status update: with mingw-w64 (or recent mingw), the built installer I was just giving a general status, sorry about not being clear. will depend on several .dll (libgcc_s_sjil.dll) that we can't easily distribute. The only place we can realistically put them is in C:\Python$VERSION (or wherever python happens to be installed), and I think it is a very bad idea to install dll from NumPy there. In Windows 2008 and above, one can refer in .pyd where to look for dlls in another directory which is private to numpy. Yes. David If I understand correctly the problem is distributing dependency/runtime DLLs with a package and ensuring the DLLs are found by Windows when the pyd extensions are imported? For numpy-MKL and other packages I include/install the extra DLLs in the package directories and, if necessary, (i) append the package directory to os.environ['PATH'] or (ii) pre-load the DLLs into the process using Ctypes, both early in the package's main __init__.py. No admin rights are required. So that seems to be the only option. Is there any other solution? I don't think it is an acceptable solution in general: modifying the PATH in a package is a big no-no, even worse than adding the dll in $prefix. I have not thought about pre-loading, but if it works, that may be a workaround. That's a very ugly workaround, though... David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
Hi, On Mon, Feb 4, 2013 at 3:46 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Mon, Feb 4, 2013 at 4:04 PM, Robert Kern robert.k...@gmail.com wrote: On Mon, Feb 4, 2013 at 10:38 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Feb 4, 2013 at 1:15 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: MSVC + Intel Fortran + MKL, yes. But those aren't free. So how can you provide an Amazon image for those? You can make an image that is not public, I guess. I suppose anyone who uses the image would have to have their own licenses for the Intel stuff? Does anyone have experience of this? You need to purchase one license per developer: http://software.intel.com/en-us/articles/intel-math-kernel-library-licensing-faq#eula1 I think 64 bits on windows is best pushed off to 1.7.1 or 1.8. It would be a bit much to get it implemented in the next week or two. The problem with not providing these binaries is that they are at the bottom of everyone's stack, so a delay in numpy holds everyone back. I can't find completely convincing stats, but it looks as though 64 bit windows 7 is now the most common version of Windows, at least for Gamers [1] around now, and it was getting that way for everyone in 2010 [2]. It don't think it reflects well on on us that we don't appear to support 64 bits out of the box; just for example, R already has a 32 bit / 64 bit installer. If I understand correctly, the options for doing this right now are: 1) Minimal cost in time : ask Christophe nicely whether we can distribute his binaries via the Numpy page 2) Small cost in time / money : pay for licenses for Ondrej or me or someone to install the dependencies on my Berkeley machine / an Amazon image. Ralf : I suppose we qualify for the free licenses you referred to earlier ? [3] . I guess that covers us for the Numpy build? Then it's only a question of paying for ifort licenses when it comes to do the Scipy build? So, if the cost of option 2 is too high, how about option 1? Cheers, Matthew [1] http://store.steampowered.com/hwsurvey [2] http://blogs.windows.com/windows/b/bloggingwindows/archive/2010/07/08/64-bit-momentum-surges-with-windows-7.aspx [3] http://numpy-discussion.10968.n7.nabble.com/MKL-licenses-for-core-scientific-Python-projects-td32530.html ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
Hi, On Mon, Feb 4, 2013 at 5:09 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Feb 4, 2013 at 3:46 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Mon, Feb 4, 2013 at 4:04 PM, Robert Kern robert.k...@gmail.com wrote: On Mon, Feb 4, 2013 at 10:38 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Feb 4, 2013 at 1:15 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: MSVC + Intel Fortran + MKL, yes. But those aren't free. So how can you provide an Amazon image for those? You can make an image that is not public, I guess. I suppose anyone who uses the image would have to have their own licenses for the Intel stuff? Does anyone have experience of this? You need to purchase one license per developer: http://software.intel.com/en-us/articles/intel-math-kernel-library-licensing-faq#eula1 I think 64 bits on windows is best pushed off to 1.7.1 or 1.8. It would be a bit much to get it implemented in the next week or two. The problem with not providing these binaries is that they are at the bottom of everyone's stack, so a delay in numpy holds everyone back. I can't find completely convincing stats, but it looks as though 64 bit windows 7 is now the most common version of Windows, at least for Gamers [1] around now, and it was getting that way for everyone in 2010 [2]. It don't think it reflects well on on us that we don't appear to support 64 bits out of the box; just for example, R already has a 32 bit / 64 bit installer. If I understand correctly, the options for doing this right now are: 1) Minimal cost in time : ask Christophe nicely whether we can distribute his binaries via the Numpy page 2) Small cost in time / money : pay for licenses for Ondrej or me or someone to install the dependencies on my Berkeley machine / an Amazon image. David - obviously if you were willing to do this - that would be far preferable to me stumbling along. Can provide machine and beer IOUs. Cheers, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Any plans for windows 64-bit installer for 1.7?
On Sun, Feb 3, 2013 at 12:28 AM, josef.p...@gmail.com wrote: On Sat, Feb 2, 2013 at 6:14 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, I see there is no Windows 64 bit installer for the 1.7 rc1. related: Is there any chance to get newer mingw or mingw-w64 support soonish? The problem has no solution until we can restrict support to windows 7 and above. Otherwise, any acceptable solution would require user to be an admin. David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
