Re: [Numpy-discussion] Should I use pip install numpy in linux?

Hi all, Just as a heads up: Nathaniel and I wrote a draft PEP on binary linux wheels that is now being discussed on distutils-sig, so you can check that out and participate in the conversation if you're interested. - PEP on python.org: https://www.python.org/dev/peps/pep-0513/ - PEP on github

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On Tue, Jan 19, 2016 at 5:57 PM, Chris Barker - NOAA Federal < chris.bar...@noaa.gov> wrote: > > > 2) continue to support those users fairly poorly, and at substantial > > ongoing cost > > I'm curious what the cost is for this poor support -- throw the source > up on PyPi, and we're done. The

Re: [Numpy-discussion] Should I use pip install numpy in linux?

> Last month, numpy had ~740,000 downloads from PyPI, Hm, given that Windows and Linux wheels have not been available, then that's mostly source installs anyway. Or failed installs -- no shortage of folks trying to pip install numpy on Windows and then having questions about why it doesn't work.

Re: [Numpy-discussion] Should I use pip install numpy in linux?

hmm -- didn't mean to rev this up quite so much -- sorry! But it's a good conversation to have, so... On Fri, Jan 15, 2016 at 1:08 PM, Benjamin Root wrote: > That being said... I take exception to your assertion that anaconda is > *the* solution to the packaging problem.

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On Thu, Jan 14, 2016 at 10:58 AM, Matthew Brett wrote: > > but neither the culture nor the tooling support that > > approach now, so I'm not very confident you could gather adoption. > > I don't think there's a very large amount of cultural work - but some > to be sure.

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On Thu, Jan 14, 2016 at 12:58 PM, Matthew Brett wrote: > On Thu, Jan 14, 2016 at 9:14 AM, Chris Barker - NOAA Federal > wrote: > >>> Also, you have the problem that there is one PyPi -- so where do you > put > >>> your nifty wheels that depend on

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On Jan 15, 2016 10:23 AM, "Chris Barker" wrote: > [...] > So here is the problem I want to solve: > > > pip install numpy scipy matplotlib scikit-learn scikit-image pandas h5py > > last I checked, each of those is self-contained, except for python-level dependencies, most

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On Fri, Jan 15, 2016 at 1:21 PM, Nathaniel Smith wrote: > On Jan 15, 2016 10:23 AM, "Chris Barker" wrote: > > > [...] > > So here is the problem I want to solve: > > > > > pip install numpy scipy matplotlib scikit-learn scikit-image pandas > h5py > > > >

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On Fri, Jan 15, 2016 at 11:21 AM, Nathaniel Smith wrote: > Sure. Someone's already packaged those for conda, and no one has packaged > them for pypi, so it makes sense that conda is more convenient for you. If > someone does the work of packaging them for pypi, then that

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On Fri, Jan 15, 2016 at 12:09 PM, Chris Barker wrote: > On Fri, Jan 15, 2016 at 11:21 AM, Nathaniel Smith wrote: >> >> Sure. Someone's already packaged those for conda, and no one has packaged >> them for pypi, so it makes sense that conda is more

Re: [Numpy-discussion] Should I use pip install numpy in linux?

Travis - I will preface the following by pointing out how valuable miniconda and anaconda has been for our workplace because we were running into issues with ensuring that everyone in our mixed platform office had access to all the same tools, particularly GDAL, NetCDF4 and such. For the longest

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On Fri, Jan 15, 2016 at 11:56 AM, Travis Oliphant wrote: > > > I still submit that this is not the best use of time. Conda *already* solves the problem.My sadness is that people keep working to create an ultimately inferior solution rather than just help make a better

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On 01/15/2016 04:08 PM, Benjamin Root wrote: So, again, I love conda for what it can do when it works well. I only take exception to the notion that it can address *all* problems, because there are some problems that it just simply isn't properly situated for. Actually, I would say you didn't

Re: [Numpy-discussion] Should I use pip install numpy in linux?

Hi, On Fri, Jan 15, 2016 at 1:56 PM, Steve Waterbury wrote: > On 01/15/2016 04:08 PM, Benjamin Root wrote: >> >> So, again, I love conda for what it can do when it works well. I only >> take exception to the notion that it can address *all* problems, because >> there are

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On 01/15/2016 05:07 PM, Matthew Brett wrote: I attribute some of the conda-ignoring to "NIH" and, to some extent, possibly defensiveness (I would be defensive too if I had been working on pip as long as they had when conda came along ;). I must say, I don't personally recognize those reasons.

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On Fri, Jan 15, 2016 at 2:15 PM, Steve Waterbury wrote: > On 01/15/2016 05:07 PM, Matthew Brett wrote: >>> >>> I attribute >>> some of the conda-ignoring to "NIH" and, to some extent, >>> possibly defensiveness (I would be defensive too if I had been >>> working on pip as

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On 01/15/2016 05:19 PM, Matthew Brett wrote: On Fri, Jan 15, 2016 at 2:15 PM, Steve Waterbury wrote: On 01/15/2016 05:07 PM, Matthew Brett wrote: I attribute some of the conda-ignoring to "NIH" and, to some extent, possibly defensiveness (I would be defensive too if

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On Fri, Jan 15, 2016 at 9:56 PM, Steve Waterbury wrote: > On 01/15/2016 04:08 PM, Benjamin Root wrote: > >> So, again, I love conda for what it can do when it works well. I only >> take exception to the notion that it can address *all* problems, because >> there are some

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On Fri, Jan 15, 2016 at 2:33 PM, Steve Waterbury wrote: > On 01/15/2016 05:19 PM, Matthew Brett wrote: >> >> On Fri, Jan 15, 2016 at 2:15 PM, Steve Waterbury >> wrote: >>> >>> On 01/15/2016 05:07 PM, Matthew Brett wrote: > > > I

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On Fri, Jan 15, 2016 at 2:38 PM, Matthew Brett wrote: > > I think there's a distinction between 'promote numpy as wheels' and > 'make numpy available as a wheel'. I don't think you'll see much > evidence of "promotion" here - it's not really the open-source way. >

Re: [Numpy-discussion] Should I use pip install numpy in linux?

> On Jan 15, 2016, at 21:22, Chris Barker wrote: > > Indeed, it's not uncommon for folks on the distutils list to say "go use > conda" in response to issues that pip does not address well. I was in the room at the very first proto-PyData conference when Guido told

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On Fri, Jan 15, 2016 at 11:56 AM, Travis Oliphant wrote: > [...] > I still submit that this is not the best use of time. Conda *already* > solves the problem.My sadness is that people keep working to create an > ultimately inferior solution rather than just help make a

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On 13 January 2016 at 22:23, Chris Barker wrote: > On Mon, Jan 11, 2016 at 5:29 PM, Nathaniel Smith wrote: >> >> I agree that talking about such things on distutils-sig tends to elicit a >> certain amount of puzzled incomprehension, but I don't think it

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On 09/01/16 00:13, Nathaniel Smith wrote: Right. There's a small problem which is that the base linux system isn't just "CentOS 5", it's "CentOS 5 and here's the list of libraries that you're allowed to link to: ...", where that list is empirically chosen to include only stuff that really is

Re: [Numpy-discussion] Should I use pip install numpy in linux?

>> Also, you have the problem that there is one PyPi -- so where do you put >> your nifty wheels that depend on other binary wheels? you may need to fork >> every package you want to build :-( > > Is this a real problem or a theoretical one? Do you know of some > situation where this wheel to

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On Thu, Jan 14, 2016 at 9:14 AM, Chris Barker - NOAA Federal wrote: >>> Also, you have the problem that there is one PyPi -- so where do you put >>> your nifty wheels that depend on other binary wheels? you may need to fork >>> every package you want to build :-( >> >> Is

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On Mon, Jan 11, 2016 at 5:29 PM, Nathaniel Smith wrote: > I agree that talking about such things on distutils-sig tends to elicit a > certain amount of puzzled incomprehension, but I don't think it matters -- > wheels already have everything you need to support this. > well,

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On Mon, Jan 11, 2016 at 11:02 AM, David Cournapeau wrote: > If we get all that worked out, we still haven't made any progress toward >> the non-standard libs that aren't python. This is the big "scipy problem" >> -- fortran, BLAS, hdf, ad infinitum. >> >> I argued for years

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On Jan 11, 2016 3:54 PM, "Chris Barker" wrote: > > On Mon, Jan 11, 2016 at 11:02 AM, David Cournapeau wrote: >>> >>> If we get all that worked out, we still haven't made any progress toward the non-standard libs that aren't python. This is the big

Re: [Numpy-discussion] Should I use pip install numpy in linux?

> And in any case we have lots of users who don't use conda and are thus doomed to support both ecosystems regardless, so we might as well make the best of it :-). Yes, this is the key. Conda is a great tool for a lot of users / use cases, but it's not for everyone. Anyways, I think I've made a

Re: [Numpy-discussion] Should I use pip install numpy in linux?

Anaconda "build environment" was setup by Ilan and me.Aaron helped to build packages while he was at Continuum but spent most of his time on the open-source conda project. It is important to understand the difference between Anaconda and conda in this respect. Anaconda is a particular

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On Fri, Jan 8, 2016 at 7:13 PM, Nathaniel Smith wrote: > > that this would potentially be able to let packages like numpy serve > their > > linux > > users better without risking too much junk being uploaded to PyPI. > > That will never fly. But like Matthew says, I think we can

Re: [Numpy-discussion] Should I use pip install numpy in linux?

The other half of the fun is how to deal with weird binary issues with libraries like libgeos_c, libhdf5 and such. You have to get all of the compile options right for your build of those libraries to get your build of GDAL and pyhdf working right. You also have packages like gdal and netcdf4

Re: [Numpy-discussion] Should I use pip install numpy in linux?

> Right. There's a small problem which is that the base linux system > isn't just "CentOS 5", it's "CentOS 5 and here's the list of libraries > that you're allowed to link to: ...", where that list is empirically > chosen to include only stuff that really is installed on ~all linux > machines and

Re: [Numpy-discussion] Should I use pip install numpy in linux?

> > Right. There's a small problem which is that the base linux system >> isn't just "CentOS 5", it's "CentOS 5 and here's the list of libraries > > that you're allowed to link to: ...", where that list is empirically > > chosen to include only stuff that really is installed on ~all linux >>

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On Sun, Jan 10, 2016 at 3:55 AM, Matthew Brett wrote: > I updated the page with more on reasons to prefer Debian packages over > installing with pip: > > https://matthew-brett.github.io/pydagogue/installing_on_debian.html > > Is that enough to get the message across?

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On Sun, Jan 10, 2016 at 3:40 AM, Sandro Tosi wrote: > On Sun, Jan 10, 2016 at 3:55 AM, Matthew Brett > wrote: >> I updated the page with more on reasons to prefer Debian packages over >> installing with pip: >> >>

Re: [Numpy-discussion] Should I use pip install numpy in linux?

> I wrote a page on using pip with Debian / Ubuntu here : > https://matthew-brett.github.io/pydagogue/installing_on_debian.html Speaking with my numpy debian maintainer hat on, I would really appreciate if you dont suggest to use pip to install packages in Debian, or at least not as the only

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On 09.01.2016 04:38, Nathaniel Smith wrote: > On Fri, Jan 8, 2016 at 7:17 PM, Nathan Goldbaum wrote: >> Doesn't building on CentOS 5 also mean using a quite old version of gcc? > > Yes. IIRC CentOS 5 ships with gcc 4.4, and you can bump that up to gcc > 4.8 by using the

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On 8 January 2016 at 23:27, Chris Barker wrote: > On Fri, Jan 8, 2016 at 1:58 PM, Robert McGibbon wrote: >> >> I'm not sure if this is the right path for numpy or not, > > > probably not -- AFAICT, the PyPa folks aren't interested in solving teh >

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On Sat, Jan 9, 2016 at 12:12 PM, Julian Taylor < jtaylor.deb...@googlemail.com> wrote: > On 09.01.2016 04:38, Nathaniel Smith wrote: > > On Fri, Jan 8, 2016 at 7:17 PM, Nathan Goldbaum > wrote: > >> Doesn't building on CentOS 5 also mean using a quite old version of gcc? >

Re: [Numpy-discussion] Should I use pip install numpy in linux?

Hi Sandro, On Sat, Jan 9, 2016 at 4:44 AM, Sandro Tosi wrote: >> I wrote a page on using pip with Debian / Ubuntu here : >> https://matthew-brett.github.io/pydagogue/installing_on_debian.html > > Speaking with my numpy debian maintainer hat on, I would really > appreciate if

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On Jan 9, 2016 04:12, "Julian Taylor" wrote: > > On 09.01.2016 04:38, Nathaniel Smith wrote: > > On Fri, Jan 8, 2016 at 7:17 PM, Nathan Goldbaum wrote: > >> Doesn't building on CentOS 5 also mean using a quite old version of gcc? > > > > Yes.

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On Sat, Jan 9, 2016 at 6:57 PM, Sandro Tosi wrote: > On Sat, Jan 9, 2016 at 6:08 PM, Matthew Brett wrote: >> Hi Sandro, >> >> On Sat, Jan 9, 2016 at 4:44 AM, Sandro Tosi wrote: I wrote a page on using pip with Debian / Ubuntu

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On Sat, Jan 9, 2016 at 6:08 PM, Matthew Brett wrote: > Hi Sandro, > > On Sat, Jan 9, 2016 at 4:44 AM, Sandro Tosi wrote: >>> I wrote a page on using pip with Debian / Ubuntu here : >>> https://matthew-brett.github.io/pydagogue/installing_on_debian.html

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On Jan 9, 2016 10:09, "Matthew Brett" wrote: > > Hi Sandro, > > On Sat, Jan 9, 2016 at 4:44 AM, Sandro Tosi wrote: > >> I wrote a page on using pip with Debian / Ubuntu here : > >> https://matthew-brett.github.io/pydagogue/installing_on_debian.html > >

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On Sat, Jan 9, 2016 at 8:49 PM, Nathaniel Smith wrote: > On Jan 9, 2016 10:09, "Matthew Brett" wrote: >> >> Hi Sandro, >> >> On Sat, Jan 9, 2016 at 4:44 AM, Sandro Tosi wrote: >> >> I wrote a page on using pip with Debian / Ubuntu here

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On Sat, Jan 9, 2016 at 8:58 PM, Matthew Brett wrote: > On Sat, Jan 9, 2016 at 8:49 PM, Nathaniel Smith wrote: >> On Jan 9, 2016 10:09, "Matthew Brett" wrote: >>> >>> Hi Sandro, >>> >>> On Sat, Jan 9, 2016 at 4:44 AM, Sandro Tosi

Re: [Numpy-discussion] Should I use pip install numpy in linux?

Does anyone know if there's been any movements with the PyPI folks on allowing linux wheels to be uploaded? I know you can never be certain what's provided by the distro, but it seems like if Anaconda can solve the cross-distro-binary-distribution-of-compiled-python-extensions problem, there

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On 8 Jan 2016 19:07, "Robert McGibbon" wrote: > > Does anyone know if there's been any movements with the PyPI folks on allowing linux wheels to be uploaded? > > I know you can never be certain what's provided by the distro, but it seems like if Anaconda can solve the

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On 8 Jan 2016 19:07, "Robert McGibbon" wrote: > > Does anyone know if there's been any movements with the PyPI folks on allowing linux wheels to be uploaded? > > I know you can never be certain what's provided by the distro, but it seems like if Anaconda can solve the

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On Fri, Jan 8, 2016 at 1:58 PM, Robert McGibbon wrote: > I'm not sure if this is the right path for numpy or not, > probably not -- AFAICT, the PyPa folks aren't interested in solving teh problems we have in the scipy community -- we can tweak around the edges, but we wont

Re: [Numpy-discussion] Should I use pip install numpy in linux?

Hi, On Fri, Jan 8, 2016 at 11:27 PM, Chris Barker wrote: > On Fri, Jan 8, 2016 at 1:58 PM, Robert McGibbon wrote: >> >> I'm not sure if this is the right path for numpy or not, > > > probably not -- AFAICT, the PyPa folks aren't interested in solving

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On Sat, Jan 9, 2016 at 12:31 AM, Robert McGibbon wrote: >> Both Anaconda and Canopy build on a base default Linux system so that >> the built binaries will work on many Linux systems. > > I think the base linux system is CentOS 5, and from my experience, it seems > like this

Re: [Numpy-discussion] Should I use pip install numpy in linux?

> Both Anaconda and Canopy build on a base default Linux system so that > the built binaries will work on many Linux systems. I think the base linux system is CentOS 5, and from my experience, it seems like this approach has worked very well. Those packages are compatible with all essentially all

Re: [Numpy-discussion] Should I use pip install numpy in linux?

Well, it's always possible to copy the dependencies like libopenblas.so into the wheel and fix up the RPATHs, similar to the way the Windows wheels work. I'm not sure if this is the right path for numpy or not, but it seems like something would be suitable for some projects with compiled

Re: [Numpy-discussion] Should I use pip install numpy in linux?

Doesn't building on CentOS 5 also mean using a quite old version of gcc? I've never tested this, but I've seen claims on the anaconda mailing list of ~25% slowdowns compared to building from source or using system packages, which was attributed to building using an older gcc that doesn't optimize

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On Fri, Jan 8, 2016 at 4:31 PM, Robert McGibbon wrote: >> Both Anaconda and Canopy build on a base default Linux system so that >> the built binaries will work on many Linux systems. > > I think the base linux system is CentOS 5, and from my experience, it seems > like this

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On Fri, Jan 8, 2016 at 7:17 PM, Nathan Goldbaum wrote: > Doesn't building on CentOS 5 also mean using a quite old version of gcc? Yes. IIRC CentOS 5 ships with gcc 4.4, and you can bump that up to gcc 4.8 by using the Redhat Developer Toolset release (which is gcc +

Re: [Numpy-discussion] Should I use pip install numpy in linux?

> Doesn't building on CentOS 5 also mean using a quite old version of gcc? I have had pretty good luck using the (awesomely named) Holy Build Box , which is a CentOS 5 docker image with a newer gcc version installed (but I guess the same old libc). I'm

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On Fri, Jan 8, 2016 at 7:41 PM, Robert McGibbon wrote: >> Doesn't building on CentOS 5 also mean using a quite old version of gcc? > > I have had pretty good luck using the (awesomely named) Holy Build Box, > which is a CentOS 5 docker image with a newer gcc version installed

Re: [Numpy-discussion] Should I use pip install numpy in linux?

> Continuum and Enthought both have a whole list of packages beyond glibc that are safe enough to link to, including a bunch of ones that would be big pains to statically link everywhere (libX11, etc.). That's the useful piece of information that goes beyond just CentOS5 + RH devtools + static

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On Jan 8, 2016 20:08, "Robert McGibbon" wrote: > > > Continuum and Enthought both have a whole list of packages beyond > glibc that are safe enough to link to, including a bunch of ones that > would be big pains to statically link everywhere (libX11, etc.). > That's the useful

Re: [Numpy-discussion] Should I use pip install numpy in linux?

Dear Nathaniel, Gotcha. That's very helpful. Thank you so much! Shawn On Thu, Jan 7, 2016 at 10:01 PM, Nathaniel Smith wrote: > On Thu, Jan 7, 2016 at 6:18 PM, Yuxiang Wang wrote: >> Dear all, >> >> I know that in Windows, we should use either Christoph's

Re: [Numpy-discussion] Should I use pip install numpy in linux?

Hi, On Fri, Jan 8, 2016 at 4:28 PM, Yuxiang Wang wrote: > Dear Nathaniel, > > Gotcha. That's very helpful. Thank you so much! > > Shawn > > On Thu, Jan 7, 2016 at 10:01 PM, Nathaniel Smith wrote: >> On Thu, Jan 7, 2016 at 6:18 PM, Yuxiang Wang

Re: [Numpy-discussion] Should I use pip install numpy in linux?

On Thu, Jan 7, 2016 at 6:18 PM, Yuxiang Wang wrote: > Dear all, > > I know that in Windows, we should use either Christoph's package or > Anaconda for MKL-optimized numpy. In Linux, the fortran compiler issue > is solved, so should I directly used pip install numpy to get