Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On 10 May 2016 at 07:08, Chris Barker wrote: > But I started this whole line of conversation because it seemed that there > was desire for: > > Ability to isolate the build environment. > Ability to better handle/manage non-python dependencies I don't care about the first

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On Sun, May 8, 2016 at 5:31 AM, Nick Coghlan wrote: > > any python developer is going to > > run into these issues eventually... > > Aye, I know - conda was one of the systems I evaluated as a possible > general purpose tool for user level package management in Fedora and >

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On 8 May 2016 at 04:15, Chris Barker wrote: > On Sat, May 7, 2016 at 6:51 AM, Nick Coghlan wrote: >> >> On 7 May 2016 01:55, "Chris Barker" wrote: >> > So my point is about scope-creep -- if you want the PyPa tools to solve >> >

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On Sat, May 7, 2016 at 6:51 AM, Nick Coghlan wrote: > On 7 May 2016 01:55, "Chris Barker" wrote: > > So my point is about scope-creep -- if you want the PyPa tools to solve > all these problems, then you are re-inventing conda -- better to simply >

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On 7 May 2016 01:55, "Chris Barker" wrote: > So my point is about scope-creep -- if you want the PyPa tools to solve all these problems, then you are re-inventing conda -- better to simply adopt conda (or fork it and fix issues that I'm sure are there) conda doesn't

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On Fri, May 6, 2016 at 9:39 AM, Donald Stufft wrote: > On May 6, 2016, at 11:54 AM, Chris Barker wrote: > > So my point is about scope-creep -- if you want the PyPa tools to solve > all these problems, then you are re-inventing conda -- better to simply

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On 6 May 2016 at 13:15, Chris Barker wrote: > On Fri, May 6, 2016 at 5:41 AM, Nick Coghlan wrote: > >> [...] > > > >> So rather than saying "the bootstrapping dependency declaration file >> is Python-but-not-really", it's easier to say "it's an

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

> On May 6, 2016, at 11:54 AM, Chris Barker wrote: > > So my point is about scope-creep -- if you want the PyPa tools to solve all > these problems, then you are re-inventing conda -- better to simply adopt > conda (or fork it and fix issues that I'm sure are there….)

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On Fri, May 6, 2016 at 5:41 AM, Nick Coghlan wrote: > The "Python-with-imports" case is the status quo with setup.py, and we > already know that's a pain because you need to set up an environment > that already has the right dependencies installed to enable your > module

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On Fri, May 6, 2016 at 5:55 AM, Nick Coghlan wrote: > > And maybe it's good to keep "new style" configuration clearly separate. > > Part of my motivation for suggesting re-using setup.cfg is the > proliferation of packaging related config sprawl in project root > directories

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On Thu, May 5, 2016 at 6:37 PM, Robert Collins wrote: > > Thats good. It occurs to me that scientific builds may be univerally > broken because folk want to avoid easy-install and the high cost of > double builds of things. E.g. adding bootstrap_requires will let folk

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On Thu, May 5, 2016 at 4:32 PM, Nathaniel Smith wrote: > > You do know that we're on our way to re-writing conda, now, don't you? > :-) > > > > I think we need to be careful of scope-creep... > > conda did not invent the idea of creating separate python environments > for

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On 6 May 2016 at 06:41, Chris Barker wrote: > On Wed, May 4, 2016 at 8:09 PM, Nick Coghlan wrote: >> 3. The ongoing popularity of setup.cfg shows that while ini-style may >> not be perfect for this use case, it clearly makes it over the >> threshold of

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On 6 May 2016 at 06:30, Chris Barker wrote: > On Wed, May 4, 2016 at 7:45 PM, Nick Coghlan wrote: >> Usually that enforcement is >> handled by making the configuration declarative - it's in some passive >> format like an ini file or JSON, and if it gets

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

>From my point of view mandatory build isolation will make building thinks slow and bad, besides being totally unrelated to the idea of a working bootstrap requirements feature. On Thu, May 5, 2016 at 9:37 PM Robert Collins wrote: > On 5 May 2016 at 21:47, Nathaniel

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On 5 May 2016 at 21:47, Nathaniel Smith wrote: > On Wed, May 4, 2016 at 11:57 PM, Robert Collins ... >>> I don't think I've ever seen a package that had accurate >>> setup_requires (outside the trivial case of packages where >>> setup_requires=[] is accurate). Scientific packages

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

Clearly it should spin up a Gentoo VM from scratch each time. No half measures. On Thu, May 5, 2016, 19:32 Nathaniel Smith wrote: > On Thu, May 5, 2016 at 3:18 PM, Chris Barker > wrote: > > On Thu, May 5, 2016 at 2:10 AM, Nathaniel Smith

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On Thu, May 5, 2016 at 3:18 PM, Chris Barker wrote: > On Thu, May 5, 2016 at 2:10 AM, Nathaniel Smith wrote: >> >> ...The main thing I want to point out though, is that all of these >> problems you're raising are complications caused entirely by wanting >>

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

Last post! sorry to have not kept up last night > to call the new feature setup_requires but some prefer to eliminate > > uncertainty by calling it bootstrap_requires. > > The main advantage of a new feature name is that when someone searches > the internet for "python bootstrap_requires",

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On Thu, May 5, 2016 at 2:47 AM, Nathaniel Smith wrote: > > When you introduce isolation, the build will only have the standard > > library + whatever is declared as a dep: and pyqt5 has no source on > > PyPI. > so build isolation isolates you from arbitrary system libs? now you

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On Thu, May 5, 2016 at 2:10 AM, Nathaniel Smith wrote: > ...The main thing I want to point out though, is that all of these > problems you're raising are complications caused entirely by wanting > to avoid build isolation in the name of simplicity. If each package > gets its own

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On Wed, May 4, 2016 at 11:57 PM, Robert Collins wrote: > > No. Old pip new package will break, new pip old package is entirely safe > AFAICT. That's the goal, yes? So I think we need to get less obsessed with backward compatibility: pip will retain (for along time)

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

We've spent a huge amount of effort on reaching the point where pretty > much everything *can* be made pip installable. Heck, *PyQt5*, which is > my personal benchmark for a probably-totally-unpackageable package, > announced last week that they now have binary wheels on pypi for all > of

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On Wed, May 4, 2016 at 8:09 PM, Nick Coghlan wrote: > I know I'm one of the folks that has historically been dubious of the > "just use setup.cfg" idea, due to the assorted problems with the > ini-style format not extending particularly well to tree-structured > data (beyond

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On Wed, May 4, 2016 at 7:45 PM, Nick Coghlan wrote: > This configuration vs customisation distinction is probably worth > spelling out for folks without a formal software engineering or > computer science background, so: > fair enough -- good to be clear on the terms. >

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On Wed, May 4, 2016 at 3:28 PM, Paul Moore wrote: > On 4 May 2016 at 23:11, Chris Barker wrote: > > so it could be purely declarative, but users could also put code in > there to > > customize the configuration on the fly, too. > > That basically

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On 5 May 2016 at 13:36, Daniel Holth wrote: > Here's the kind of thing that you should expect. Someone will write > > setup.cfg: > > [bootstrap_requires] > pbr > > pip installs pbr in a directory that is added to PYTHONPATH for that build. Ah, so we don't install into the

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On 5 May 2016 at 22:36, Daniel Holth wrote: > Pedantic note > > setup_requires is a setuptools parameter used to install packages after > setup() is called. Even though very many people expect or want those > packages to be installed before setup.py executes. I think it is

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

Here's the kind of thing that you should expect. Someone will write setup.cfg: [bootstrap_requires] pbr pip installs pbr in a directory that is added to PYTHONPATH for that build. The package builds. And there was much rejoicing. The big gain from this simple feature is that people will be

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On 5 May 2016 at 10:10, Nathaniel Smith wrote: > ...The main thing I want to point out though, is that all of these > problems you're raising are complications caused entirely by wanting > to avoid build isolation in the name of simplicity. If each package > gets its own isolated

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On 5 May 2016 at 19:47, Nathaniel Smith wrote: > The reason I'm being so intense about this is that AFAICT these are all true: > > Premise 1: Without build isolation enabled by default, then in > practice everyone will putter along putting up with broken builds all > the time.

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On 5 May 2016 at 14:22, Nathaniel Smith wrote: > On Wed, May 4, 2016 at 6:28 AM, Nick Coghlan wrote: >> On 4 May 2016 at 23:00, Daniel Holth wrote: >>> +1 It would be great to start with a real setup_requires and probably would >>> not

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On Wed, May 4, 2016 at 11:57 PM, Robert Collins wrote: > On 5 May 2016 at 18:32, Nathaniel Smith wrote: >> On Wed, May 4, 2016 at 10:42 PM, Robert Collins >>... >>> Yes, things will break: anyone using this will need a new pip, by >>> definition. Not

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On Thu, May 5, 2016 at 2:02 AM, Paul Moore wrote: > On 5 May 2016 at 07:57, Robert Collins wrote: >> We've a history in this group of biting off too much and things not >> getting executed. We're *still* in the final phases of deploying >> PEP-508,

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On 5 May 2016 at 07:57, Robert Collins wrote: > We've a history in this group of biting off too much and things not > getting executed. We're *still* in the final phases of deploying > PEP-508, and it was conceptually trivial. I'm not arguing that we > shouldn't make

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On 5 May 2016 at 18:32, Nathaniel Smith wrote: > On Wed, May 4, 2016 at 10:42 PM, Robert Collins >... >> Yes, things will break: anyone using this will need a new pip, by >> definition. Not everyone will be willing to wait 10 years before using >> it :). > > Just to clarify (since

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On Wed, May 4, 2016 at 10:42 PM, Robert Collins wrote: > On 5 May 2016 at 16:22, Nathaniel Smith wrote: > ... >> I'm sympathetic to the general approach, but on net I think I prefer a >> slightly different proposal. >> >> Downsides to just standardizing

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On 5 May 2016 at 16:22, Nathaniel Smith wrote: ... > I'm sympathetic to the general approach, but on net I think I prefer a > slightly different proposal. > > Downsides to just standardizing [setup_requires]: If I write a PEP, it won't be standardising setup_requires, it will be

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On Wed, May 4, 2016 at 6:28 AM, Nick Coghlan wrote: > On 4 May 2016 at 23:00, Daniel Holth wrote: >> +1 It would be great to start with a real setup_requires and probably would >> not interfere with later build system abstractions at all. > > If we're going

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

Ok so, if i draft a pep for said proposal, will it die under the weight of a thousand bike sheds? On 5 May 2016 3:09 PM, "Nick Coghlan" wrote: > On 5 May 2016 at 06:28, Robert Collins wrote: > > the only reason I got involved in build system

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On 5 May 2016 at 06:28, Robert Collins wrote: > the only reason I got involved in build system discussions was > pushback 18months or so back when I implemented a proof of concept for > pip that just used setup.cfg. I'd be very happy to ignore all the > build system

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On 5 May 2016 at 08:28, Paul Moore wrote: > On 4 May 2016 at 23:11, Chris Barker wrote: >> so it could be purely declarative, but users could also put code in there to >> customize the configuration on the fly, too. > > That basically repeats the

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On 4 May 2016 at 23:11, Chris Barker wrote: > so it could be purely declarative, but users could also put code in there to > customize the configuration on the fly, too. That basically repeats the mistake that was made with setup.py. We explicitly don't want an executable

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On Wed, May 4, 2016 at 3:33 AM, Donald Stufft wrote: > I'd actually prefer not using JSON for something that is human > editable/writable because I think it's a pretty poor format for that case. > It > is overly restrictive in what it allows (for instance, no trailing comma >

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On 4 May 2016 at 22:33, Donald Stufft wrote: > ..> I also believe that we can't provide a replacement for setup.py without either > purposely declaring we no longer support something that people used from it or > providing a way to support that in the new, setup.py-less format.

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

Just call it Steve On Wed, May 4, 2016, 16:25 Robert Collins wrote: > On 4 May 2016 at 19:39, Nick Coghlan wrote: > > On 4 May 2016 at 16:03, Robert Collins > wrote: > >> The edits I'd expect to make if the conclusions

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On 4 May 2016 at 19:39, Nick Coghlan wrote: > On 4 May 2016 at 16:03, Robert Collins wrote: >> The edits I'd expect to make if the conclusions I suggested in >> https://mail.python.org/pipermail/distutils-sig/2016-March/028437.html >> are adopted

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

Agree. On Wed, May 4, 2016, 09:28 Nick Coghlan wrote: > On 4 May 2016 at 23:00, Daniel Holth wrote: > > +1 It would be great to start with a real setup_requires and probably > would > > not interfere with later build system abstractions at all. > > If

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On 4 May 2016 at 23:00, Daniel Holth wrote: > +1 It would be great to start with a real setup_requires and probably would > not interfere with later build system abstractions at all. If we're going to go down that path, perhaps it might make sense to just define a standard

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

+1 It would be great to start with a real setup_requires and probably would not interfere with later build system abstractions at all. On Wed, May 4, 2016 at 6:33 AM Donald Stufft wrote: > > > On May 4, 2016, at 3:39 AM, Nick Coghlan wrote: > > > > On 4

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

> On May 4, 2016, at 3:39 AM, Nick Coghlan wrote: > > On 4 May 2016 at 16:03, Robert Collins wrote: >> The edits I'd expect to make if the conclusions I suggested in >> https://mail.python.org/pipermail/distutils-sig/2016-March/028437.html >> are

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On 4 May 2016 at 16:03, Robert Collins wrote: > The edits I'd expect to make if the conclusions I suggested in > https://mail.python.org/pipermail/distutils-sig/2016-March/028437.html > are adopted are: > > - change to a Python API > - BFDL call on the file format and

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On 4 May 2016 at 05:10, Paul Moore wrote: > Nick - do you have the time to pick this up? Or does it need someone > to step up as BDFL-delegate? Robert, Nathaniel, do you have time to > spend on a final round of discussion on this, on the assumption that > the goal will be a

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On 4 May 2016 at 04:28, Nathaniel Smith wrote: > On Tue, May 3, 2016 at 10:10 AM, Paul Moore wrote: >> On 3 May 2016 at 17:47, Donald Stufft wrote: >>> It will likely get decided as part of the build system PEP, whenever that >>> gets

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On 3 May 2016 at 19:28, Nathaniel Smith wrote: > No, no, Nick's not the blocker. I'm the blocker! (Sorry) > > Donald + Robert + I had a longish conversation about this on IRC a > month ago [1]. I volunteered to summarize back to the mailing list, > and then I flaked -- so I guess

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

Thanks for the update! Glad this is still moving forward. I'll continue to prod the list if things stall again as I want to respond to "Python packaging is broken" with "actually your knowledge is just outdated, go read packaging.python.org". :) On Tue, 3 May 2016 at 11:28 Nathaniel Smith

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On Tue, May 3, 2016 at 10:10 AM, Paul Moore wrote: > On 3 May 2016 at 17:47, Donald Stufft wrote: >> It will likely get decided as part of the build system PEP, whenever that >> gets picked up again. > > Yes, but on 15th March >

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

We did separate build from install. Now we just want to be able to build without [having to emulate] distutils; just having some dependencies installed before setup.py runs would also be a great boon. I'm reading part of this conversation as "a simple bdist_wheel bug is a reason to do a lot of

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On Tue, May 3, 2016 at 12:10 PM, Paul Moore wrote: > On 3 May 2016 at 17:47, Donald Stufft wrote: >> It will likely get decided as part of the build system PEP, whenever that >> gets picked up again. > > Yes, but on 15th March >

Re: [Distutils] moving things forward (was: wheel including files it shouldn't)

On 3 May 2016 at 17:47, Donald Stufft wrote: > It will likely get decided as part of the build system PEP, whenever that > gets picked up again. Yes, but on 15th March (https://mail.python.org/pipermail/distutils-sig/2016-March/028457.html) Robert posted > Just to set

[Distutils] moving things forward (was: wheel including files it shouldn't)

On Wed, 27 Apr 2016 at 10:53 Donald Stufft wrote: > This isn't really a problem with what you're doing. Rather it's an issue > with the toolchain and and open question whether or not wheels should > conceptually be able to be produced from a checkout, or if they should only >