Re: [Distutils] Handling the binary dependency management problem

2013-12-06 Thread Chris Barker
On Fri, Dec 6, 2013 at 5:16 AM, Thomas Heller wrote: > Am 06.12.2013 13:22, schrieb Nick Coghlan: > > Manipulation of __path__ at runtime usually makes it harder for > >> modulefinder to find all the required modules. >>> >> >> Not usually, always. That's why >> http://docs.python.org/2/library

Re: [Distutils] Handling the binary dependency management problem

2013-12-06 Thread Chris Barker
On Fri, Dec 6, 2013 at 5:06 AM, David Cournapeau wrote: > As Ralf, I think it is overkill. The problem of SSE vs non SSE is because > of one library, ATLAS, which as IMO the design flaw of being arch specific. > yup -- really designed for the end user to built it themselves > MKL does not

Re: [Distutils] Handling the binary dependency management problem

2013-12-06 Thread David Cournapeau
On Fri, Dec 6, 2013 at 5:50 PM, Chris Barker wrote: > On Fri, Dec 6, 2013 at 5:06 AM, David Cournapeau wrote: > >> As Ralf, I think it is overkill. The problem of SSE vs non SSE is because >> of one library, ATLAS, which as IMO the design flaw of being arch specific. >> > > yup -- really designed

Re: [Distutils] Handling the binary dependency management problem

2013-12-06 Thread Chris Barker
On Fri, Dec 6, 2013 at 4:33 AM, Nick Coghlan wrote: > In the absence of the perfect solution (i.e. picking the right variant > out of no SSE, SSE2, SSE3 automatically), would it be a reasonable > compromise to standardise on SSE2 as "lowest acceptable common > denominator"? > +1 > Users with n

Re: [Distutils] Handling the binary dependency management problem

2013-12-06 Thread Ralf Gommers
On Fri, Dec 6, 2013 at 2:48 PM, Oscar Benjamin wrote: > On 6 December 2013 13:06, David Cournapeau wrote: > > > > As Ralf, I think it is overkill. The problem of SSE vs non SSE is > because of > > one library, ATLAS, which as IMO the design flaw of being arch specific. > I > > always hoped we cou

Re: [Distutils] Handling the binary dependency management problem

2013-12-06 Thread Ralf Gommers
On Fri, Dec 6, 2013 at 1:33 PM, Nick Coghlan wrote: > On 6 December 2013 17:21, Ralf Gommers wrote: > > On Fri, Dec 6, 2013 at 6:47 AM, Nick Coghlan wrote: > >> With that approach, the existing wheel model would work (no need for a > >> variant system), and numpy installations could be freely m

Re: [Distutils] Handling the binary dependency management problem

2013-12-06 Thread Chris Barker
On Thu, Dec 5, 2013 at 11:21 PM, Ralf Gommers wrote: > Hmm, taking a compile flag and encoding it in the package layout seems > like a fundamentally wrong approach. > well, it's pretty ugly hack, but sometimes an ugly hack that does the job is better than nothing. IIUC, the Intel MKL libs do som

Re: [Distutils] Handling the binary dependency management problem

2013-12-06 Thread Oscar Benjamin
On 6 December 2013 13:06, David Cournapeau wrote: > > As Ralf, I think it is overkill. The problem of SSE vs non SSE is because of > one library, ATLAS, which as IMO the design flaw of being arch specific. I > always hoped we could get away from this when I built those special > installers for num

Re: [Distutils] Handling the binary dependency management problem

2013-12-06 Thread Thomas Heller
Am 06.12.2013 13:22, schrieb Nick Coghlan: On 6 December 2013 17:10, Thomas Heller wrote: Am 06.12.2013 06:47, schrieb Nick Coghlan: Hmm, I just had an idea for how to do the runtime selection thing. It actually shouldn't be that hard, so long as the numpy folks are OK with a bit of __path__ m

Re: [Distutils] Handling the binary dependency management problem

2013-12-06 Thread David Cournapeau
On Fri, Dec 6, 2013 at 12:44 PM, Donald Stufft wrote: > How does conda handle SSE vs SSE2 vs SSE3? I’m digging through it’s > source code and just installed numpy with it and I can’t seem to find any > handling of that? > I can't speak for conda, but @enthought, we solve it by using the MKL, whi

Re: [Distutils] Handling the binary dependency management problem

2013-12-06 Thread David Cournapeau
On Fri, Dec 6, 2013 at 5:47 AM, Nick Coghlan wrote: > On 6 December 2013 11:52, Donald Stufft wrote: > > > > On Dec 5, 2013, at 8:48 PM, Chris Barker - NOAA Federal < > chris.bar...@noaa.gov> wrote: > > > >> What would really be best is run-time selection of the appropriate lib > >> -- it would

Re: [Distutils] Handling the binary dependency management problem

2013-12-06 Thread Donald Stufft
How does conda handle SSE vs SSE2 vs SSE3? I’m digging through it’s source code and just installed numpy with it and I can’t seem to find any handling of that? On Dec 6, 2013, at 7:33 AM, Nick Coghlan wrote: > On 6 December 2013 17:21, Ralf Gommers wrote: >> On Fri, Dec 6, 2013 at 6:47 AM, Nick

Re: [Distutils] Handling the binary dependency management problem

2013-12-06 Thread Nick Coghlan
On 6 December 2013 17:21, Ralf Gommers wrote: > On Fri, Dec 6, 2013 at 6:47 AM, Nick Coghlan wrote: >> With that approach, the existing wheel model would work (no need for a >> variant system), and numpy installations could be freely moved between >> machines (or shared via a network directory).

Re: [Distutils] Handling the binary dependency management problem

2013-12-06 Thread Nick Coghlan
On 6 December 2013 17:10, Thomas Heller wrote: > Am 06.12.2013 06:47, schrieb Nick Coghlan: >> Hmm, I just had an idea for how to do the runtime selection thing. It >> actually shouldn't be that hard, so long as the numpy folks are OK >> with a bit of __path__ manipulation in package __init__ modu

Re: [Distutils] Handling the binary dependency management problem

2013-12-05 Thread Ralf Gommers
On Fri, Dec 6, 2013 at 6:47 AM, Nick Coghlan wrote: > On 6 December 2013 11:52, Donald Stufft wrote: > > > > On Dec 5, 2013, at 8:48 PM, Chris Barker - NOAA Federal < > chris.bar...@noaa.gov> wrote: > > > >> What would really be best is run-time selection of the appropriate lib > >> -- it would

Re: [Distutils] Handling the binary dependency management problem

2013-12-05 Thread Thomas Heller
Am 06.12.2013 06:47, schrieb Nick Coghlan: On 6 December 2013 11:52, Donald Stufft wrote: On Dec 5, 2013, at 8:48 PM, Chris Barker - NOAA Federal wrote: What would really be best is run-time selection of the appropriate lib -- it would solve this problem, and allow users to re-distribute w

Re: [Distutils] Handling the binary dependency management problem

2013-12-05 Thread Nick Coghlan
On 6 December 2013 11:52, Donald Stufft wrote: > > On Dec 5, 2013, at 8:48 PM, Chris Barker - NOAA Federal > wrote: > >> What would really be best is run-time selection of the appropriate lib >> -- it would solve this problem, and allow users to re-distribute >> working binaries via py2exe, etc.

Re: [Distutils] Handling the binary dependency management problem

2013-12-05 Thread Chris Barker
On Thu, Dec 5, 2013 at 5:52 PM, Donald Stufft wrote: > > On Dec 5, 2013, at 8:48 PM, Chris Barker - NOAA Federal < > chris.bar...@noaa.gov> wrote: > > > What would really be best is run-time selection of the appropriate lib > > -- it would solve this problem, and allow users to re-distribute > >

Re: [Distutils] Handling the binary dependency management problem

2013-12-05 Thread Donald Stufft
On Dec 5, 2013, at 8:48 PM, Chris Barker - NOAA Federal wrote: > What would really be best is run-time selection of the appropriate lib > -- it would solve this problem, and allow users to re-distribute > working binaries via py2exe, etc. And not require opening a security > hole in wheels... >

Re: [Distutils] Handling the binary dependency management problem

2013-12-05 Thread Chris Barker - NOAA Federal
On Dec 5, 2013, at 1:12 PM, Oscar Benjamin wrote: > > Yes, I wouldn't want that kind of bad PR getting around about > scientific Python "Python is slower than Matlab" etc. > Well, is that better or worse that 2% or less people finding they can't run it on their old machines > It seems as if

Re: [Distutils] Handling the binary dependency management problem

2013-12-05 Thread Ralf Gommers
On Thu, Dec 5, 2013 at 10:12 PM, Oscar Benjamin wrote: > On 4 December 2013 20:56, Ralf Gommers wrote: > > On Wed, Dec 4, 2013 at 5:05 PM, Chris Barker - NOAA Federal > > wrote: > >> > >> So a lowest common denominator wheel would be very, very, useful. > >> > >> As for what that would be: the s

Re: [Distutils] Handling the binary dependency management problem

2013-12-05 Thread Oscar Benjamin
On 4 December 2013 20:56, Ralf Gommers wrote: > On Wed, Dec 4, 2013 at 5:05 PM, Chris Barker - NOAA Federal > wrote: >> >> So a lowest common denominator wheel would be very, very, useful. >> >> As for what that would be: the superpack is great, but it's been around a >> while (long while in comp

Re: [Distutils] Handling the binary dependency management problem

2013-12-05 Thread Chris Barker - NOAA Federal
On Dec 4, 2013, at 11:35 PM, Ralf Gommers wrote I'm just wondering how much we are making this hard for very little return. I also don't know. I wonder if a poll on the relevant lists would be helpful... I'll start playing with wheels in the near future. Great! Thanks! There are multip

Re: [Distutils] Handling the binary dependency management problem

2013-12-05 Thread Chris Barker - NOAA Federal
On Dec 5, 2013, at 1:40 AM, Paul Moore wrote: > > I'm not sure how this would work - wheels don't seem to me to be > appropriate for installing "external dependencies", but as I'm not > 100% clear on what you mean by that term One of the key features of conda is that it is not specifically tied

Re: [Distutils] Handling the binary dependency management problem

2013-12-05 Thread Paul Moore
On 5 December 2013 09:52, Nick Coghlan wrote: >> I'm not sure how this would work - wheels don't seem to me to be >> appropriate for installing "external dependencies", but as I'm not >> 100% clear on what you mean by that term I may be misunderstanding. >> Can you provide a concrete example? > >

Re: [Distutils] Handling the binary dependency management problem

2013-12-05 Thread Nick Coghlan
On 5 December 2013 19:40, Paul Moore wrote: > On 4 December 2013 23:31, Nick Coghlan wrote: >> Hmm, rather than adding complexity most folks don't need directly to the >> base wheel spec, here's a possible "multiwheel" notion - embed multiple >> wheels with different names inside the multiwheel,

Re: [Distutils] Handling the binary dependency management problem

2013-12-05 Thread Paul Moore
On 4 December 2013 23:31, Nick Coghlan wrote: > Hmm, rather than adding complexity most folks don't need directly to the > base wheel spec, here's a possible "multiwheel" notion - embed multiple > wheels with different names inside the multiwheel, along with a > self-contained selector function fo

Re: [Distutils] Handling the binary dependency management problem

2013-12-05 Thread Nick Coghlan
On 5 December 2013 17:35, Ralf Gommers wrote: > > Namespace packages have been tried with scikits - there's a reason why > scikit-learn and statsmodels spent a lot of effort dropping them. They don't > work. Scipy, while monolithic, works for users. The namespace package emulation that was all th

Re: [Distutils] Handling the binary dependency management problem

2013-12-04 Thread Ralf Gommers
On Thu, Dec 5, 2013 at 1:09 AM, Chris Barker wrote: > On Wed, Dec 4, 2013 at 12:56 PM, Ralf Gommers wrote: > >> The problem is explaining to people what they want - no one reads docs >> before grabbing a binary. >> > > right -- so we want a default "pip install" install that will work for > most

Re: [Distutils] Handling the binary dependency management problem

2013-12-04 Thread Chris Barker
On Wed, Dec 4, 2013 at 12:56 PM, Ralf Gommers wrote: > The problem is explaining to people what they want - no one reads docs > before grabbing a binary. > right -- so we want a default "pip install" install that will work for most people. And I think "works for most people" is far more important

Re: [Distutils] Handling the binary dependency management problem

2013-12-04 Thread Nick Coghlan
On 5 Dec 2013 07:29, "Ralf Gommers" wrote: > > > > > On Wed, Dec 4, 2013 at 11:41 AM, Oscar Benjamin < oscar.j.benja...@gmail.com> wrote: >> >> On 4 December 2013 07:40, Ralf Gommers wrote: >> > On Wed, Dec 4, 2013 at 1:54 AM, Donald Stufft wrote: >> >> >> >> I’d love to get Wheels to the point

Re: [Distutils] Handling the binary dependency management problem

2013-12-04 Thread Ralf Gommers
On Wed, Dec 4, 2013 at 10:59 PM, Paul Moore wrote: > On 4 December 2013 21:13, Ralf Gommers wrote: > > Besides the issues you mention, the problem is that it creates a single > > point of failure. I really appreciate everything Christoph does, but it's > > not appropriate as the default way to p

Re: [Distutils] Handling the binary dependency management problem

2013-12-04 Thread Paul Moore
On 4 December 2013 21:13, Ralf Gommers wrote: > Besides the issues you mention, the problem is that it creates a single > point of failure. I really appreciate everything Christoph does, but it's > not appropriate as the default way to provide binary releases for a large > number of projects. Ther

Re: [Distutils] Handling the binary dependency management problem

2013-12-04 Thread Ralf Gommers
On Wed, Dec 4, 2013 at 11:41 AM, Oscar Benjamin wrote: > On 4 December 2013 07:40, Ralf Gommers wrote: > > On Wed, Dec 4, 2013 at 1:54 AM, Donald Stufft wrote: > >> > >> I’d love to get Wheels to the point they are more suitable then they are > >> for SciPy stuff, > > > > That would indeed be a

Re: [Distutils] Handling the binary dependency management problem

2013-12-04 Thread Ralf Gommers
On Wed, Dec 4, 2013 at 9:13 AM, Paul Moore wrote: > On 4 December 2013 07:40, Ralf Gommers wrote: > >> I’m not sure what the diff between the current state and what > >> they need to be are but if someone spells it out (I’ve only just skimmed > >> your last email so perhaps it’s contained in tha

Re: [Distutils] Handling the binary dependency management problem

2013-12-04 Thread Ralf Gommers
On Wed, Dec 4, 2013 at 5:05 PM, Chris Barker - NOAA Federal < chris.bar...@noaa.gov> wrote: > Ralf, > > Great to have you on this thread! > > Note: supporting "variants" on one way or another is a great idea, but for > right now, maybe we can get pretty far without it. > > There are options for "s

Re: [Distutils] Handling the binary dependency management problem

2013-12-04 Thread Chris Barker - NOAA Federal
Ralf, Great to have you on this thread! Note: supporting "variants" on one way or another is a great idea, but for right now, maybe we can get pretty far without it. There are options for "serious" scipy users that need optimum performance, and newbies that want the full stack. So our primary a

Re: [Distutils] Handling the binary dependency management problem

2013-12-04 Thread Oscar Benjamin
On 4 December 2013 12:10, Nick Coghlan wrote: > On 4 December 2013 20:41, Oscar Benjamin wrote: >> >> Another possibility is that the pip/wheel/PyPI/metadata system can be >> changed to allow a "variant" field for wheels/sdists. This was also >> suggested in the same thread by Nick Coghlan: >> ht

Re: [Distutils] Handling the binary dependency management problem

2013-12-04 Thread Thomas Heller
Am 04.12.2013 11:41, schrieb Oscar Benjamin: On 4 December 2013 07:40, Ralf Gommers wrote: How do I package those three builds into wheels and get the right one installed by ``pip install numpy``? This was discussed previously on this list: https://mail.python.org/pipermail/distutils-sig/2013

Re: [Distutils] Handling the binary dependency management problem

2013-12-04 Thread Nick Coghlan
On 4 December 2013 20:41, Oscar Benjamin wrote: > On 4 December 2013 07:40, Ralf Gommers wrote: >> On Wed, Dec 4, 2013 at 1:54 AM, Donald Stufft wrote: >>> >>> I’d love to get Wheels to the point they are more suitable then they are >>> for SciPy stuff, >> >> That would indeed be a good step for

Re: [Distutils] Handling the binary dependency management problem

2013-12-04 Thread Oscar Benjamin
On 4 December 2013 07:40, Ralf Gommers wrote: > On Wed, Dec 4, 2013 at 1:54 AM, Donald Stufft wrote: >> >> I’d love to get Wheels to the point they are more suitable then they are >> for SciPy stuff, > > That would indeed be a good step forward. I'm interested to try to help get > to that point f

Re: [Distutils] Handling the binary dependency management problem

2013-12-04 Thread Paul Moore
On 4 December 2013 08:13, Paul Moore wrote: >> If this is too difficult at the moment, an easier (but much less important >> one) would be to get the result of ``paver bdist_wininst_simple`` as a >> wheel. > > That I will certainly look into. Simple answer is "wheel convert > ". But maybe it would

Re: [Distutils] Handling the binary dependency management problem

2013-12-04 Thread Paul Moore
On 4 December 2013 07:40, Ralf Gommers wrote: >> I’m not sure what the diff between the current state and what >> they need to be are but if someone spells it out (I’ve only just skimmed >> your last email so perhaps it’s contained in that!) I’ll do the arguing >> for it. I >> just need someone wh

Re: [Distutils] Handling the binary dependency management problem

2013-12-04 Thread Paul Moore
On 3 December 2013 22:18, Chris Barker wrote: >> Looks like the conda stack is built around msvcr90, whereas python.org >> Python 3.3 is built around msvcr100. >> So conda is not interoperable *at all* with standard python.org Python >> 3.3 on Windows :-( > > again, Anaconda the distribution, is

Re: [Distutils] Handling the binary dependency management problem

2013-12-03 Thread Ralf Gommers
On Wed, Dec 4, 2013 at 1:54 AM, Donald Stufft wrote: > > On Dec 3, 2013, at 7:36 PM, Oscar Benjamin > wrote: > > > On 3 December 2013 21:13, Donald Stufft wrote: > >> I think Wheels are the way forward for Python dependencies. Perhaps not > for > >> things like fortran. I hope that the scientif

Re: [Distutils] Handling the binary dependency management problem

2013-12-03 Thread Donald Stufft
On Dec 3, 2013, at 7:36 PM, Oscar Benjamin wrote: > On 3 December 2013 21:13, Donald Stufft wrote: >> I think Wheels are the way forward for Python dependencies. Perhaps not for >> things like fortran. I hope that the scientific community can start >> publishing wheels at least in addition too.

Re: [Distutils] Handling the binary dependency management problem

2013-12-03 Thread Oscar Benjamin
On 3 December 2013 21:13, Donald Stufft wrote: > I think Wheels are the way forward for Python dependencies. Perhaps not for > things like fortran. I hope that the scientific community can start > publishing wheels at least in addition too. The Fortran issue is not that complicated. Very few pack

Re: [Distutils] Handling the binary dependency management problem

2013-12-03 Thread Oscar Benjamin
On 3 December 2013 22:18, Chris Barker wrote: > On Tue, Dec 3, 2013 at 12:48 AM, Nick Coghlan wrote: >> >> Because it already works for the scientific stack, and if we don't provide >> any explicit messaging around where conda fits into the distribution >> picture, users are going to remain confu

Re: [Distutils] Handling the binary dependency management problem

2013-12-03 Thread Chris Barker
On Tue, Dec 3, 2013 at 3:50 PM, Marcus Smith wrote: > >> umm, why not? you couldn't have a pySide wheel??? >> > > just saying that the anaconda index literally has packages for "qt" > itself, the c++ library. > http://repo.continuum.io/pkgs/free/linux-64/qt-4.8.5-0.tar.bz2 > > and it's pyside pac

Re: [Distutils] Handling the binary dependency management problem

2013-12-03 Thread Marcus Smith
> > well, except that the anaconda index covers non-python projects like "qt", >> which a private wheel index wouldn't cover (at least with the normal >> intended use of wheels) > > > umm, why not? you couldn't have a pySide wheel??? > just saying that the anaconda index literally has packages for

Re: [Distutils] Handling the binary dependency management problem

2013-12-03 Thread Chris Barker
On Tue, Dec 3, 2013 at 12:48 AM, Nick Coghlan wrote: > Because it already works for the scientific stack, and if we don't provide > any explicit messaging around where conda fits into the distribution > picture, users are going to remain confused about it for a long time. > Do we have to have exp

Re: [Distutils] Handling the binary dependency management problem

2013-12-03 Thread Marcus Smith
"conda init" isn't in the website docs. On Tue, Dec 3, 2013 at 2:00 PM, Marcus Smith wrote: > > >> The most striking difference may be that conda also installs and >> manages Python itself. For example, conda create -n py33 python=3.3 >> will download and install Python 3.3 into a new environme

Re: [Distutils] Handling the binary dependency management problem

2013-12-03 Thread Marcus Smith
> > The most striking difference may be that conda also installs and > manages Python itself. For example, conda create -n py33 python=3.3 > will download and install Python 3.3 into a new environment named > py33. This is completely different than pip which tends to run inside > the same Python en

Re: [Distutils] Handling the binary dependency management problem

2013-12-03 Thread Donald Stufft
Filed https://github.com/ContinuumIO/conda-recipes/issues/42 :( On Dec 3, 2013, at 4:48 PM, Donald Stufft wrote: > > On Dec 3, 2013, at 4:46 PM, Daniel Holth wrote: > >> In summary conda is very different than pip+virtualenv. > > Conda is a cross platform Homebrew. > > - > D

Re: [Distutils] Handling the binary dependency management problem

2013-12-03 Thread Donald Stufft
On Dec 3, 2013, at 4:46 PM, Daniel Holth wrote: > In summary conda is very different than pip+virtualenv. Conda is a cross platform Homebrew. - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signe

Re: [Distutils] Handling the binary dependency management problem

2013-12-03 Thread Paul Moore
On 3 December 2013 21:34, Marcus Smith wrote: >> The IPython web site makes it look like you really need to go get Anaconda >> or Canopy if you want iPython > > interesting... You really don't. I've been running IPython on Windows from wheels for ages. And you can actually build from source easil

Re: [Distutils] Handling the binary dependency management problem

2013-12-03 Thread Daniel Holth
On Tue, Dec 3, 2013 at 4:13 PM, Donald Stufft wrote: > I think Wheels are the way forward for Python dependencies. Perhaps not for > things like fortran. I hope that the scientific community can start > publishing wheels at least in addition too. > > I don't believe that Conda will gain the mindsh

Re: [Distutils] Handling the binary dependency management problem

2013-12-03 Thread Paul Moore
On 3 December 2013 21:34, Marcus Smith wrote: >> Anoconda (the distribution seems to address this by having conda packages >> that are essentially containers for the shared libs, and other packages that >> need those libs depend on them. I like this method, but it seems to me to be >> more a feat

Re: [Distutils] Handling the binary dependency management problem

2013-12-03 Thread Marcus Smith
> Anoconda (the distribution seems to address this by having conda packages > that are essentially containers for the shared libs, and other packages > that need those libs depend on them. I like this method, but it seems to me > to be more a feature of the Anoconda distribution than the conda pac

Re: [Distutils] Handling the binary dependency management problem

2013-12-03 Thread Donald Stufft
I think Wheels are the way forward for Python dependencies. Perhaps not for things like fortran. I hope that the scientific community can start publishing wheels at least in addition too. I don't believe that Conda will gain the mindshare that pip has outside of the scientific community so I h

Re: [Distutils] Handling the binary dependency management problem

2013-12-03 Thread Chris Barker
Side note about naming: I'm no expert, but I'm pretty sure "Anoconda" is a python distribution -- python itself and set of pre-build packages. "conda" is the package manager that is used by Anoconda -- kind of like rpm is used by RedHat -- conda is an open-source project, and thus could be used by

Re: [Distutils] Handling the binary dependency management problem

2013-12-03 Thread Nick Coghlan
On 4 Dec 2013 01:31, "Oscar Benjamin" wrote: > > On 3 December 2013 13:53, Nick Coghlan wrote: > > On 3 December 2013 22:49, Oscar Benjamin wrote: > > > > Hmm, I likely wouldn't build it into the core requirement system (that > > all operates at the distribution level), but the latest metadata >

Re: [Distutils] Handling the binary dependency management problem

2013-12-03 Thread Nick Coghlan
On 4 Dec 2013 05:54, "Marcus Smith" wrote: > > >> >> If conda doesn't offer such an internal consistency guarantee for >> published package sets, then I agree with the criticism that it's just >> an alternative to running a private PyPI index server hosting wheel >> files pre-built with particular

Re: [Distutils] Handling the binary dependency management problem

2013-12-03 Thread Marcus Smith
> If conda doesn't offer such an internal consistency guarantee for > published package sets, then I agree with the criticism that it's just > an alternative to running a private PyPI index server hosting wheel > files pre-built with particular options, and thus it becomes > substantially less inte

Re: [Distutils] Handling the binary dependency management problem

2013-12-03 Thread Oscar Benjamin
On 3 December 2013 13:53, Nick Coghlan wrote: > On 3 December 2013 22:49, Oscar Benjamin wrote: > > Hmm, I likely wouldn't build it into the core requirement system (that > all operates at the distribution level), but the latest metadata > updates split out a bunch of the optional stuff to extens

Re: [Distutils] Handling the binary dependency management problem

2013-12-03 Thread Nick Coghlan
On 3 December 2013 22:49, Oscar Benjamin wrote: > On 3 December 2013 11:54, Nick Coghlan wrote: >> I believe helping the conda devs to get it to play nice with virtual >> environments is still a worthwhile exercise though (even if just by >> pointing out areas where it *doesn't* currently interop

Re: [Distutils] Handling the binary dependency management problem

2013-12-03 Thread Oscar Benjamin
On 3 December 2013 11:54, Nick Coghlan wrote: > On 3 December 2013 21:22, Oscar Benjamin wrote: >> AFAICT conda/binstar are alternatives for pip/PyPI that happen to host >> binaries for some packages that don't have binaries on PyPI. (conda >> also provides a different - incompatible - take on vi

Re: [Distutils] Handling the binary dependency management problem

2013-12-03 Thread Nick Coghlan
On 3 December 2013 21:22, Oscar Benjamin wrote: > AFAICT conda/binstar are alternatives for pip/PyPI that happen to host > binaries for some packages that don't have binaries on PyPI. (conda > also provides a different - incompatible - take on virtualenvs but > that's not relevant to this proposal

Re: [Distutils] Handling the binary dependency management problem

2013-12-03 Thread Oscar Benjamin
On 1 December 2013 04:15, Nick Coghlan wrote: > > conda has its own binary distribution format, using hash based > dependencies. It's this mechanism which allows it to provide reliable > cross platform binary dependency management, but it's also the same > mechanism that prevents low impact securi

Re: [Distutils] Handling the binary dependency management problem

2013-12-03 Thread Nick Coghlan
On 3 December 2013 19:11, Paul Moore wrote: > On 3 December 2013 08:48, Nick Coghlan wrote: >>> And wouldn't it be better to make wheel a bit more robust in this regard >>> than add yet another recommended tool to the mix? >> >> Software that works today is generally more useful to end users than

Re: [Distutils] Handling the binary dependency management problem

2013-12-03 Thread Paul Moore
On 3 December 2013 10:36, Oscar Benjamin wrote: > Perhaps it would help if there were wheels for conda and its > dependencies. That may well be a good idea. One thing pip does is go to great lengths to *not* have any dependencies (by vendoring everything it needs, and relying only on pure Python

Re: [Distutils] Handling the binary dependency management problem - wording

2013-12-03 Thread Pachi
El 03/12/2013 10:22, Paul Moore escribió: On 3 December 2013 08:48, Nick Coghlan wrote: This means that one key reason I want to recommend it for the cases where it is a good fit (i.e. the scientific Python stack) is so we can explicitly advise *against* using it in other cases where it will ju

Re: [Distutils] Handling the binary dependency management problem

2013-12-03 Thread Oscar Benjamin
On 3 December 2013 10:19, Nick Coghlan wrote: >> Or >> how about a scientist that wants wxPython (to use Chris' example)? >> Apparently the conda repo doesn't include wxPython, so do they need to >> learn how to install pip into a conda environment? (Note that there's >> no wxPython wheel, so this

Re: [Distutils] Handling the binary dependency management problem

2013-12-03 Thread Nick Coghlan
On 3 December 2013 20:19, Nick Coghlan wrote: > Only folks that actually understand the difference between > static and dynamic linking and wrapper modules vs self-contained > accelerator modules are likely to understand what "shared external > binary dependency" means, so I agree it's not a usefu

Re: [Distutils] Handling the binary dependency management problem

2013-12-03 Thread Nick Coghlan
On 3 December 2013 19:22, Paul Moore wrote: > On 3 December 2013 08:48, Nick Coghlan wrote: >> This means that one key reason I want to recommend it for the cases where it >> is a good fit (i.e. the scientific Python stack) is so we can explicitly >> advise *against* using it in other cases where

Re: [Distutils] Handling the binary dependency management problem

2013-12-03 Thread Paul Moore
On 3 December 2013 09:11, Paul Moore wrote: > On 3 December 2013 08:48, Nick Coghlan wrote: >>> And wouldn't it be better to make wheel a bit more robust in this regard >>> than add yet another recommended tool to the mix? >> >> Software that works today is generally more useful to end users than

Re: [Distutils] Handling the binary dependency management problem

2013-12-03 Thread Paul Moore
On 3 December 2013 08:48, Nick Coghlan wrote: > This means that one key reason I want to recommend it for the cases where it > is a good fit (i.e. the scientific Python stack) is so we can explicitly > advise *against* using it in other cases where it will just add complexity > without adding valu

Re: [Distutils] Handling the binary dependency management problem

2013-12-03 Thread Paul Moore
On 3 December 2013 08:48, Nick Coghlan wrote: >> And wouldn't it be better to make wheel a bit more robust in this regard >> than add yet another recommended tool to the mix? > > Software that works today is generally more useful to end users than > software that might possibly handle their use ca

Re: [Distutils] Handling the binary dependency management problem

2013-12-03 Thread Nick Coghlan
Thanks for the robust feedback folks - it's really helping me to clarify what I think, and why I consider this an important topic :) On 3 Dec 2013 10:36, "Chris Barker" wrote: > > On Mon, Dec 2, 2013 at 5:22 AM, Nick Coghlan wrote: >> >> And the conda folks are working on playing nice with virtu

Re: [Distutils] Handling the binary dependency management problem

2013-12-02 Thread Chris Barker
On Mon, Dec 2, 2013 at 5:22 AM, Nick Coghlan wrote: > And the conda folks are working on playing nice with virtualenv - I don't > we'll see a similar offer from Microsoft for MSI any time soon :) > nice to know... > > > a single organisation. Pip (when used normally) communicates with PyPI > > >

Re: [Distutils] Handling the binary dependency management problem

2013-12-02 Thread Nick Coghlan
On 3 Dec 2013 09:03, "Paul Moore" wrote: > > On 2 December 2013 22:26, Nick Coghlan wrote: > >>Whether solving the Unix > >> issues is worth it is the Unix users' call - I'll help solve the > >> issues, if they choose to, but I won't support abandoning the existing > >> Windows solution just beca

Re: [Distutils] Handling the binary dependency management problem

2013-12-02 Thread Marcus Smith
> I'm not sure about the conda -> wheel direction, but "pip install conda && > conda init" mostly works already if you're in a virtualenv that owns its > copy of Python > ok, I just tried conda in a throw-away altinstall of py2.7. I was thinking I would have to "conda create" new isolated environme

Re: [Distutils] Handling the binary dependency management problem

2013-12-02 Thread Nick Coghlan
On 3 Dec 2013 08:17, "Marcus Smith" wrote: > > >>> >>> publication of curated stacks when the conda folks already have one, >> >> so, I see the index: http://repo.continuum.io/pkgs/index.html >> Is they a way to contribute to this index yet? or is that what would need to be worked out. > > > pro

Re: [Distutils] Handling the binary dependency management problem

2013-12-02 Thread Paul Moore
On 2 December 2013 22:26, Nick Coghlan wrote: >>Whether solving the Unix >> issues is worth it is the Unix users' call - I'll help solve the >> issues, if they choose to, but I won't support abandoning the existing >> Windows solution just because it can't be extended to cater for Unix >> as well.

Re: [Distutils] Handling the binary dependency management problem

2013-12-02 Thread Nick Coghlan
On 3 Dec 2013 00:19, "Paul Moore" wrote: > > On 2 December 2013 13:38, Tres Seaver wrote: > > On 12/01/2013 06:38 PM, Paul Moore wrote: > >> I understand that things are different in the Unix world, but to be > >> blunt why should Windows users care? > > > > You're kidding, right? 90% or more of

Re: [Distutils] Handling the binary dependency management problem

2013-12-02 Thread Marcus Smith
> publication of curated stacks when the conda folks already have one, >> > so, I see the index: http://repo.continuum.io/pkgs/index.html > Is they a way to contribute to this index yet? or is that what would need > to be worked out. > probably a dumb question, but would it be possible to conver

Re: [Distutils] Handling the binary dependency management problem

2013-12-02 Thread Marcus Smith
> In the conda build guide, the yaml spec files reference dependencies by > name/version (and the type of conda environment you're in will determine > the rest) > > http://docs.continuum.io/conda/build.html#specifying-versions-in-requirements > Where does the hash come in? what do you mean? > e.g

Re: [Distutils] Handling the binary dependency management problem

2013-12-02 Thread Marcus Smith
> hash based dependencies > In the conda build guide, the yaml spec files reference dependencies by name/version (and the type of conda environment you're in will determine the rest) http://docs.continuum.io/conda/build.html#specifying-versions-in-requirements Where does the hash come in? what do

Re: [Distutils] Handling the binary dependency management problem

2013-12-02 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/02/2013 12:23 PM, Vinay Sajip wrote: > On Mon, 2/12/13, Tres Seaver wrote: > >> The issue is combinatorial explosion in the compatibility tag >> space. There is basically zero chance that even Linux users (even >> RedHat users across RHEL ver

Re: [Distutils] Handling the binary dependency management problem

2013-12-02 Thread Vinay Sajip
On Mon, 2/12/13, Tres Seaver wrote: > The issue is combinatorial explosion in the compatibility tag space. > There is basically zero chance that even Linux users (even RedHat > users across RHEL version) would benefit from pre-built binary > wheels (as opposed to packages from their distribut

Re: [Distutils] Handling the binary dependency management problem

2013-12-02 Thread Paul Moore
On 2 December 2013 14:19, Oscar Benjamin wrote: > Basically if > 1) There is at least one single consistent set of built wheels for > Windows/OSX for any popular set of binary-interdependent packages. > 2) A way to automatically detect incompatibilities and to > automatically find compatible built

Re: [Distutils] Handling the binary dependency management problem

2013-12-02 Thread Oscar Benjamin
On 2 December 2013 13:54, Paul Moore wrote: > > If the named projects provided Windows binaries, then there would be > no issue with Christoph's stuff. But AFAIK, there is no place I can > get binary builds of matplotlib *except* from Christoph. And lxml > provides limited sets of binaries - there

Re: [Distutils] Handling the binary dependency management problem

2013-12-02 Thread Paul Moore
On 2 December 2013 13:38, Tres Seaver wrote: > On 12/01/2013 06:38 PM, Paul Moore wrote: >> I understand that things are different in the Unix world, but to be >> blunt why should Windows users care? > > You're kidding, right? 90% or more of the reason for wheels in the first > place is because W

Re: [Distutils] Handling the binary dependency management problem

2013-12-02 Thread Paul Moore
On 2 December 2013 13:22, Nick Coghlan wrote: >> As a quick sanity check question - what is the long-term advice for >> Christoph (and others like him)? Continue distributing wininst >> installers? Move to wheels? Move to conda packages? Do whatever you >> want, we don't care? We're supposedly pus

Re: [Distutils] Handling the binary dependency management problem

2013-12-02 Thread Ralf Gommers
On Mon, Dec 2, 2013 at 12:38 AM, Paul Moore wrote: > On 1 December 2013 22:17, Nick Coghlan wrote: > > > For example, I installed Nikola into a virtualenv last night. That > required > > installing the development headers for libxml2 and libxslt, but the error > > that tells you that is a C comp

Re: [Distutils] Handling the binary dependency management problem

2013-12-02 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/01/2013 06:38 PM, Paul Moore wrote: > I understand that things are different in the Unix world, but to be > blunt why should Windows users care? You're kidding, right? 90% or more of the reason for wheels in the first place is because Windows u

Re: [Distutils] Handling the binary dependency management problem

2013-12-02 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/01/2013 05:17 PM, Nick Coghlan wrote: > I see conda as existing at a similar level to apt and yum from a > packaging point of view, with zc.buildout as a DIY equivalent at that > level. FTR: zc.buildout does nothing to insulate you from the nee

Re: [Distutils] Handling the binary dependency management problem

2013-12-02 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/01/2013 05:07 PM, Vinay Sajip wrote: > On Sun, 1/12/13, Paul Moore wrote: > >> If the issue is simply around defining compatibility tags that >> better describe the various environments around, then let's just >> get on with that - we're going

Re: [Distutils] Handling the binary dependency management problem

2013-12-02 Thread Nick Coghlan
On 2 Dec 2013 21:57, "Paul Moore" wrote: > > On 2 December 2013 10:45, Oscar Benjamin wrote: > > Nick's proposal is basically incompatible with allowing Cristoph > > Gohlke to use pip and wheels. Christoph provides a bewildering array > > of installers for prebuilt packages that are interchangeab

Re: [Distutils] Handling the binary dependency management problem

2013-12-02 Thread Paul Moore
On 2 December 2013 10:45, Oscar Benjamin wrote: > Nick's proposal is basically incompatible with allowing Cristoph > Gohlke to use pip and wheels. Christoph provides a bewildering array > of installers for prebuilt packages that are interchangeable with > other builds at the level of Python code b

  1   2   >