On 19 May 2015 at 17:28, Chris Barker <chris.bar...@noaa.gov> wrote: > On Tue, May 19, 2015 at 5:21 AM, Paul Moore <p.f.mo...@gmail.com> wrote: >> >> If conda did everything pip did (and that includes consuming wheels >> from PyPI, not just sdists, and it includes caching of downloads, >> autobuilding of wheels etc, etc.) > > hmm...what about half-way -- conda does everything pip does, but not > necessarily the same way -- i.e. you do a "conda install this_package", and > it works for every package ( OK -- almost every ;-) ) that pip install works > for.
Sure. Doesn't have to be the same way, but the user experience has to be the same. > But maybe that's not going to cut it -- in a way, we are headed there now, > with a contingent of people porting pypi packages to conda. So far it's > various subsets of the scientific community, but if we could get a few web > developers to join in... Unless project owners switch to providing conda packages, isn't there always going to be a lag? If a new version of lxml comes out, how long must I wait for "the conda folks" to release a package for it? >> then I'd certainly consider how to >> switch to conda (*not* Anaconda - I'd use a different package manager, >> but not a different Python distribution) rather than pip. > > hmm -- that's the interesting technical question -- conda works at a higher > level than pip -- it CAN manage python itself -- I'm not sure it is HAS to, > but that's how it is usually used, and the idea is to provide a complete > environment, which does include python itself. Yes. But I don't want to use Anaconda Python, Same reason - how long do I wait for the new release of Python to be available in Anaconda? There's currently no Python 3.5 alpha for example... >> But "considering switching" would include getting PyPI supporting >> conda packages, > > uhm, why? if there is a community supported repo of packages -- why does it > have to be PyPi? If conda/binstar is good enough to replace pip/PyPI, there's no reason for pip/PyPI to still exist. So in effect binstar *becomes* PyPI. There's an element of evangelisation going on here - you're (effectively) asking what it'd take to persuade me to use conda in place of pip. I'm playing hard to get, a little, because I see no specific benefits to me in using conda, so I don't see why I should accept any loss at all, in the absence of a benefit to justify it. My biggest worry is that at some point, "if you want numpy/scipy, you should use conda" becomes an explicit benefit of conda, and pip/PyPI users get abandoned by the scientific community. If that happens, I'd rather see the community rally behind conda than see a split. But I hope that's not the way things end up going. >> getting ensurepip replaced with ensureconda, etc. A >> total replacement for pip, in other words. This is the key point. The decision was made to "bless" pip as the official Python package manager. Should we revisit that decision? If not, then how do we ensure that pip (and the surrounding infrastructure) handles the needs of the *whole* Python community? If the authors of scientific extensions for Python abandon pip for conda, then pip isn't supporting that part of the community properly. But conversely, if the scientific community doesn't look to address their issues within the pip/wheel infrastructure, how can we do anything to avoid a rift? (end of doom and gloom section ;-)) >> As a pip maintainer I'm obviously biased, but if conda is intending to >> replace pip as the official packaging solution for Python, then it >> needs to do so completely. If it doesn't do that, then we (PyPA and >> the Python core developers) need to be able to credibly say that pip >> is the official solution, and that means that we need to make sure >> that pip/wheel provides the best user experience possible. That >> includes persuading parts of the Python community (e.g. Scientific >> users) not to abandon the standard solution in favour of a custom one. > > > I agree here. Though we do have a problem -- as Nick has pointed out, the > "full" scientific development process -- even if python-centered, requires > non-python parts, even beyond shared libs -- a fortran compiler is a big > one, but also maybe other languages, like R, or Julia, etc. Or LLVM (for > numba), or... > > This is why Continuum build conda -- they wanted to provide a way to manage > all that. > > So is it possible for PyPA to grow the features to mange all the python > bits, and then have things like conda use pip inside of Anaconda, maybe? I'd like to think so. The goal of pip is to be the baseline Python package manager. We're expecting Linux distributions to build their system packages via wheel, why can't Anaconda? Part of the problem here, to my mind, is that it's *very* hard for the outsider to separate out (Ana)conda-as-a-platform versus conda-as-a-tool, versus conda-as-a-distribution-format. > or > SOME transition where you can add conda if and only if you need its unique > features, and as a add-it-later-to-what-you-have solution, rather than, when > you need R or some such, you need to > > * Toss out your current setup > * Install Anaconda (or miniconda) > * Switch from virtualenv to conda environments > * re-install all your dependencies Yeah, that's hopeless. And worse still is the possibility that a pure Python user might have to do that just to gain access to a particular package. I keep saying this, and I ought to ask - is there *any* likelihood that a package would formally abandon any attempt to provide binary distributions for Windows (or OSX, or whatever) except in conda format? So PyPI users will be told "install from source yourself or switch to conda". If there's no intention for that to ever happen, a lot of the "conda vs pip" discussion is less relevant. At the moment, there are projects like numpy that don't distribute Windows wheels on PyPI, but Christoph Gohlke has most of them available, and in general such projects seem to be aiming to move to wheels. So there aren't any practical cases of "conda-only" packages. > And for even that to work, we need a way for everything installable by pip > to be installable within that conda environment -- which we could probably > achieve. See above - without some serious resource on the conda side, that "everything" is unlikely. >> My fear here is a split in the Python community, with some packages >> only being available via one ecosystem, and some via another. > > > Exactly. While I'm not at all sure that we could get to a "one way to do it" > that would meet every community's needs, I do think that we could push > pip+pypi+wheel a little further to better support at least the > python-centric stuff -- i.e. third party libs, which would get us a lot > farther. Agreed. Understanding the *actual* problem here is important though (see my other post about clarifying why dynamic linking is so important). > And again, it's not just the scipy stack -- there is stuff like image > manipulation packages, etc, that could be better handled. Well, (for example) Pillow provides wheels with no issue, so I'm not sure what you're thinking of here? > And the geospatial packages are a mess, too - is that "scientific"? -- I > don't know, but it's the "new hotness" in web development. I can't really comment on that one, as I've never used them. Does that make me untrendy? :-) Paul _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
