On Fri, Jan 15, 2016 at 1:21 PM, Nathaniel Smith <n...@pobox.com> wrote:
> On Jan 15, 2016 10:23 AM, "Chris Barker" <chris.bar...@noaa.gov> wrote: > > > [...] > > So here is the problem I want to solve: > > > > > pip install numpy scipy matplotlib scikit-learn scikit-image pandas > h5py > > > > last I checked, each of those is self-contained, except for python-level > dependencies, most notably on numpy. So it doesn't' help me solve my > problem. For instance, I have my own C/C++ code that I'm wrapping that > requires netcdf (https://github.com/NOAA-ORR-ERD/PyGnome), and another > that requires image libs like libpng, libjpeg, etc.( > https://github.com/NOAA-ORR-ERD/py_gd) > > > > netcdf is not too ugly itself, but it depends on hdf5, libcurl, zlib > (others?). So I have all these libs I need. As it happens, matplotlib > probably has the image libs I need, and h5py has hdf5 (and libcurl? and > zlib?). But even then, as far as I can tell, I need to build and provide > these libs myself for my code. Which is a pain in the @%$ and then I'm > shipping (and running) multiple copies of the same libs all over the place > -- will there be compatibility issues? apparently not, but it's still > wastes the advantage of shared libs, and makes things a pain for all of us. > > > > With conda, on the other hand, I get netcdf libs, hdf5 libs, libpng, > libjpeg, ibtiff, and I can build my stuff against those and depend on them > -- saves me a lot of pain, and my users get a better system. > > Sure. Someone's already packaged those for conda, and no one has packaged > them for pypi, so it makes sense that conda is more convenient for you. If > someone does the work of packaging them for pypi, then that difference goes > away. I'm not planning to do that work myself :-). My focus in these > discussions around pip/pypi is selfishly focused on the needs of numpy. > pip/pypi is clearly the weakest link in the space of packaging and > distribution systems that our users care about, so improvements there raise > the minimum baseline we can assume. But if/when we sort out the hard > problems blocking numpy wheels (Linux issues, windows issues, etc.) then I > suspect that we'll start seeing people packaging up those dependencies that > you're worrying about and putting them on pypi, just because there won't be > any big road blocks anymore to doing so. > I still submit that this is not the best use of time. Conda *already* solves the problem. My sadness is that people keep working to create an ultimately inferior solution rather than just help make a better solution more accessible. People mistakenly believe that wheels and conda packages are equivalent. They are not. If they were we would not have created conda. We could not do what was necessary with wheels and contorting wheels to become conda packages was and still is a lot more work. Now, obviously, it's just code and you can certainly spend effort and time to migrate wheels so that they functionally equivalently to conda packages --- but what is the point, really? Why don't we work together to make the open-source conda project and open-source conda packages more universally accessible? The other very real downside is that these efforts to promote numpy as wheels further encourages people to not use the better solution that already exists in conda. I have to deal with people that *think* pip will solve all their problems all the time. It causes a lot of difficulty when they end up with work-around after work-around that is actually all solved already with conda. It's a weird situation to be in. I'm really baffled by the resistance of this community to just help make conda *the* solution for the scientific python community. I think it would be better to spend time: 1) helping smooth out the pip/conda divide. There are many ways to do this that have my full support: * making sure pip install conda works well * creating "shadow packages" in conda for things that pip has installed * making it possible for pip to install conda packages directly (and ignore the extra features that conda makes possible that pip does not support). A pull-request to pip that did that would be far more useful than trying to cram things into the wheel concept. 2) creating a community conda packages to alleviate whatever concerns might exist about the "control" of the packages that people can install with conda. * Continuum has volunteered resources to Numfocus so that it can be the governing body of what goes into a "pycommunity" channel for conda that will be as easy to get as a "conda install pycommunity" I have not yet heard any really valid reason why this community should not just adopt conda packages and encourage others to do the same. The only thing I have heard are "chicken-and-egg" stories that come down to "we want people to be able to use pip." So, good, then let's make it so that pip can install conda packages and that conda packages with certain restrictions can be hosted on pypi or anywhere else that you have an "index". At least if there were valid reasons they could be addressed. But, this head-in-the-sand attitude towards a viable technology that is freely available is really puzzling to me. There are millions of downloads of Anaconda and many millions of downloads of conda packages each year. That is just with one company doing it. There could be many millions more with other companies and organizations hosting conda packages and indexes. The conda user-base is already very large. A great benefit to the Python ecosystem would be to allow pip users and conda users to share each other's work --- rather than to spend time reproducing work that is already done and freely available. -Travis > -n > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > https://mail.scipy.org/mailman/listinfo/numpy-discussion > > -- *Travis Oliphant* *Co-founder and CEO* @teoliphant 512-222-5440 http://www.continuum.io
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
