OK, I started this thread a while back, as I was getting confused and having issues with intermixing python, setuptools, pip, and Anaconda / conda.
Now I've figured out where i have my issue: I'm using an Anaconda distribution at the moment. I want conda to handle installing my dependencies, etc. for me. OK. However, I am also developing various python packages -- this means I can't / don't want o build and install a conda package for them, rather, I'd like to use setuptools "develop" mode. So here's the rub: When I call "setup.py develop", setuptools apparently looks for the "install_requires" packages. If it doesn't find them, it goes out and decided to apparently pip install them: gets the source form pypi, download, tries to compile, etc.... Even if it does find them already installed, it does some annoying adding to easy_install.pth magic for them. This is all why I've been thinking that dependencies do not belong in setup.py -- but rather outside of setup.py (requirements.txt), and more to the pint, dependency handling ius a pip (or conda, or...) issue - not one that should be handled by aw setuptools at build time. Note that conda build usually simply calls setup.py install as well, so this can be a problem even there (though I think it usually satisfies the requirements first, so not so bad) At the end of the day, however, I think the issue is not so much where you specify dependencies, but what setuptool develop mode is doing: it should NOT go an auto-install dependencies, particularly not install-dependencies (maybe build dependencies are required...) OK -- I just found the --no-deps option. So I can do what I want, but still, I don't think it belongs there and all, and certainly would be better to have the default be no-deps. Let pip (or conda, or...) handle that. Any one running these by hand are be definition doing things by hand, let them deal with the deps. OK, I suppose "casual" users may run setup.py install, so that mode _might_ want to auto install dependencies, if somethign has to. But develop mode is very much for developers, you really don't want this handled there. -Chris On Wed, Dec 31, 2014 at 9:41 AM, Chris Barker <chris.bar...@noaa.gov> wrote: > On Wed, Dec 31, 2014 at 9:10 AM, Nick Coghlan <ncogh...@gmail.com> wrote: > >> The problem always existed - it's the longstanding conflict between >> "platform independent, language specific" tooling and "platform specific, >> language independent" tooling. >> >> The former is often preferred on the developer side (since the tools are >> oriented towards building individual custom applications rather than >> maintaining a suite of applications written by different groups), while the >> latter is essential on the system integrator side (since you're dealing >> with inevitable technology sprawl in infrastructure that evolves over the >> course of years and decades). >> >> One of the best things coming out of the whole "DevOps" notion is the >> recognition that language independence and platform independence are aimed >> at solving *different problems*, and that what we need is suitable tools >> with different roles and responsibilities that interoperate effectively, >> rather than attempting to provide a single universal tool and having >> different groups of users yelling at each other for wanting to solve the >> "wrong" problem. >> >> Tools like conda and zc.buildout fit into that landscape by aiming to >> provide a platform & language independent approach to doing *application* >> level integration, which tends to have the effect of making them new >> platforms in their own right. >> > Indeed -- thanks for providing a clear way to talk/think about these > systems. > > I guess the trick here is that we want the different level tools to work > well with each-other. > > As conda started with python packages in mind, it does a pretty good job > with them. But I still find some conflicts between setuptools and conda -- > in particular, if you specify dependencies in setup.py (install_requires), > it can kind of make a mess of things. conda tries to ignore them, but > somehow I've had issues, even though I had specified it all in the conda's > meta.yaml. This is probably a conda bug/issue, but I'm still trying to > figure out how to best set up a python package so that it can be built > installed with the "regular" python tools, and also conda... > > Use case -- developing in a conda environment -- so I want to install > dependencies with conda, but the package under development with setuptools > "develop" mode. (conda does not (yet) have a develop mode that works...) > > Oh, and there does seem to be an odd (I think non-fatal) issue with > setuptools and conda: > > I have package A, with a bunch of stuff listed with "install_requires" > > I have all these dependencies installed with conda. > > When I run setup.py develop, setuptools goes and dumps all the > dependencies in easy_install.pth. > > I have no idea why that is, and it's probably only annoying, rather than a > problem, but I'm not sure what will happen when I upgrade one of those > dependencies with conda. > >> If you compare them to Linux distributions, then zc.buildout is a >> platform that practices the Gentoo style approach of building everything >> from source for each application, while conda uses the more common >> Debian/Fedora style of defining a binary packaging format that allows a >> redistributor to handle the build process on behalf of their users. >> > indeed -- and conda actually provides (to my disappointment) very little > in the way of build support -- you need to write platform dependent build > scripts to actually build the packages. > > But it does provide a nice way for me to provide a full "just install > this" distribution of my complex, ugly, hard to build packages.... > > -Chris > > > -- > > Christopher Barker, Ph.D. > Oceanographer > > Emergency Response Division > NOAA/NOS/OR&R (206) 526-6959 voice > 7600 Sand Point Way NE (206) 526-6329 fax > Seattle, WA 98115 (206) 526-6317 main reception > > chris.bar...@noaa.gov > -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig