Re: [Distutils] Fwd: Re: PEP 517 again

On 25 August 2017 at 17:47, Paul Moore wrote: > If anyone feels that it's *not* reasonable for users to expect "build > a wheel" and "build a sdist and build a wheel from that" to produce > the same results for a given source tree (assuming the backend > supports both

Re: [Distutils] Fwd: Re: PEP 517 again

On Fri, Aug 25, 2017, at 11:56 AM, Thomas Kluyver wrote: > Great, I think we're on the same page. I'm going to make a PR against > the PEP while I still remember what page it is. ;-) https://github.com/python/peps/pull/364 ___ Distutils-SIG maillist -

Re: [Distutils] Fwd: Re: PEP 517 again

On Fri, Aug 25, 2017, at 11:45 AM, Paul Moore wrote: > If it can't build a sdist then there's no opportunity for frontends to > fall back to sdist->wheel anyway, so that's not an issue[1]. > > It's only if sdist->wheel and direct wheel are both possible that > frontends have a choice, and I feel

Re: [Distutils] Fwd: Re: PEP 517 again

On 25 August 2017 at 11:16, Thomas Kluyver wrote: >> I repeat my earlier comment here - do we have any actual cases where >> we expect in-place wheel builds to *not* enforce the equivalence? > > Flit cannot strictly enforce it if it can't build an sdist: with the way > I've

Re: [Distutils] Fwd: Re: PEP 517 again

Thanks Paul, On Fri, Aug 25, 2017, at 10:45 AM, Paul Moore wrote: > I'd *also* expect that the wheel built (and hence what gets installed) > would be the same as what you'd get if the user added write > permissions and then ran the command. That's part of what I mean by > direct wheel and

Re: [Distutils] Fwd: Re: PEP 517 again

On 25 August 2017 at 09:38, Thomas Kluyver wrote: > On Fri, Aug 25, 2017, at 08:47 AM, Paul Moore wrote: >> I've never cared about whether the backend modifies the source tree, >> only that the direct-wheel / sdist->wheel equivalence holds. > > If a user types 'pip install

Re: [Distutils] Fwd: Re: PEP 517 again

On Fri, Aug 25, 2017, at 08:47 AM, Paul Moore wrote: > I've never cared about whether the backend modifies the source tree, > only that the direct-wheel / sdist->wheel equivalence holds. If a user types 'pip install .' in a directory where they don't have write permissions, what should happen?

Re: [Distutils] Fwd: Re: PEP 517 again

On 25 August 2017 at 06:54, Nick Coghlan wrote: > On 25 August 2017 at 14:33, xoviat wrote: >> Just this morning, Paul said the following: >> >> That step's the problem. If the frontend does that it can potentially >> be copying a lot of unneeded stuff (VCS

Re: [Distutils] Fwd: Re: PEP 517 again

Okay, but I would like to hear Paul's response to that first. On Aug 25, 2017 12:54 AM, "Nick Coghlan" wrote: > On 25 August 2017 at 14:33, xoviat wrote: > > Just this morning, Paul said the following: > > > > That step's the problem. If the frontend does

Re: [Distutils] Fwd: Re: PEP 517 again

On 25 August 2017 at 14:33, xoviat wrote: > Just this morning, Paul said the following: > > That step's the problem. If the frontend does that it can potentially > be copying a lot of unneeded stuff (VCS history, for example). We > tried that with pip and it was a major issue.

Re: [Distutils] Fwd: Re: PEP 517 again

And one more thing: it would probably be prescient to require a get_backend_info hook that returns the name and url of the backend so that pip can direct people where to file bugs when there are problems. On Aug 24, 2017 11:52 PM, "xoviat" wrote: > Actually after reading the

Re: [Distutils] Fwd: Re: PEP 517 again

Actually after reading the email that Nathaniel referenced, I disagree with Nick's position on in-tree builds. It's absolutely necessary because the tree may be read only and it's enforceable by pip through a simple recursive list. On Aug 24, 2017 11:33 PM, "xoviat" wrote: >

Re: [Distutils] Fwd: Re: PEP 517 again

Just this morning, Paul said the following: That step's the problem. If the frontend does that it can potentially be copying a lot of unneeded stuff (VCS history, for example). We tried that with pip and it was a major issue. That problem is the *whole point* of all the discussions about the

Re: [Distutils] Fwd: Re: PEP 517 again

On Thu, Aug 24, 2017 at 9:17 PM, xoviat wrote: > > I'm *not* OK with banning in-tree builds in the spec, since that's > > both unnecessary and unenforceable > > Well then either we can trust the backend or we cannot. If we can, then this > is both necessary and enforceable. If

Re: [Distutils] Fwd: Re: PEP 517 again

I'm *not* OK with banning in-tree builds in the spec, since that's both unnecessary and unenforceable Well then either we can trust the backend or we cannot. If we can, then this is both necessary and enforceable. If not, then we're back to pip copying files. You can't make and argument that it's

Re: [Distutils] Fwd: Re: PEP 517 again

On 25 August 2017 at 10:08, Daniel Holth wrote: > It would be simpler if the only requirement was to produce a correct wheel. +1 To be clear on my current position: * due to the introduction of NotImplementedError, I'm now OK with explicitly out-of-tree builds being deferred

Re: [Distutils] Fwd: Re: PEP 517 again

The current proposal is to remove build_dir and leave it up to the backend. On Aug 24, 2017 7:07 PM, "Greg Ewing" wrote: > Paul Moore wrote: > >> 4. The point of all this is that the definition of build_directory >> says "If build_directory is None, the backend may

Re: [Distutils] Fwd: Re: PEP 517 again

It would be simpler if the only requirement was to produce a correct wheel. On Thu, Aug 24, 2017, 20:07 Greg Ewing wrote: > Paul Moore wrote: > > 4. The point of all this is that the definition of build_directory > > says "If build_directory is None, the backend may

Re: [Distutils] Fwd: Re: PEP 517 again

Paul Moore wrote: 4. The point of all this is that the definition of build_directory says "If build_directory is None, the backend may do an 'in place' build which modifies the source directory and may produce different results from those that would be obtained by exporting an sdist first". Is

Re: [Distutils] Fwd: Re: PEP 517 again

Actually I submitted a compliant PR 3 months ago when the PEP was first marked as accepted. Then it was marked as draft and I thought that I would just wait for distutils-sig to handle it. And then I came over here to find what was going on. On Aug 24, 2017 5:33 PM, "xoviat"

Re: [Distutils] Fwd: Re: PEP 517 again

I'm handling the setuptools backend. I will make sure that it complies with the clean directory. On Aug 24, 2017 5:30 PM, "Paul Moore" wrote: > On 24 August 2017 at 20:00, Thomas Kluyver wrote: > > So for the time being, I'd prefer to say that the

Re: [Distutils] Fwd: Re: PEP 517 again

On 24 August 2017 at 20:00, Thomas Kluyver wrote: > So for the time being, I'd prefer to say that the build_wheel() hook should > not create or modify files in the source directory. Backends may keep their > own caches elsewhere to speed up repeated builds, but any more

Re: [Distutils] Fwd: Re: PEP 517 again

So in summary: - NotImplementedError - Remove build_dir and specify that no modifications can be made to the source directory On the third issue: I understand where Nathaniel is coming from in regards to sys.path. But there should at least be an option for this because of for example, numpy:

Re: [Distutils] Fwd: Re: PEP 517 again

On Thu, Aug 24, 2017, at 07:30 PM, Leonardo Rochael Almeida wrote: > It looks like a lot of trouble for a feature that is designed to > solve a problem for a very thin intersection of the Venn diagram of > people who:> > a. wants to quickly iterate while hacking a package > > b. doesn't want

Re: [Distutils] Fwd: Re: PEP 517 again

The distutils build/ folder is what moves if you pass build_dir to the proposed function. On Thu, Aug 24, 2017, 14:39 xoviat wrote: > > It looks like a lot of trouble for a feature that is designed to solve a > problem for a very thin intersection of the Venn diagram of people

Re: [Distutils] Fwd: Re: PEP 517 again

> It looks like a lot of trouble for a feature that is designed to solve a problem for a very thin intersection of the Venn diagram of people who: For the record, I don't agree at all that build_wheel_incremental should be specified here. The suggestion is simply a compromise that I can tolerate

Re: [Distutils] Fwd: Re: PEP 517 again

On 24 August 2017 at 15:13, Thomas Kluyver wrote: > On Thu, Aug 24, 2017, at 06:20 PM, Daniel Holth wrote: > > On Thu, Aug 24, 2017 at 12:34 PM Thomas Kluyver > wrote: > > Is there a reason to ask for an 'unclean' build, though? There may be a >

Re: [Distutils] Fwd: Re: PEP 517 again

To be clear, is everyone okay with the following? build_wheel(wheel_directory, config_settings=None, metadata_directory=None): [REQUIRED] build_wheel_incremental(wheel_directory, config_settings=None, metadata_directory=None): [OPTIONAL] I still think that the frontend should not provide a

Re: [Distutils] Fwd: Re: PEP 517 again

On Thu, Aug 24, 2017, at 06:20 PM, Daniel Holth wrote: > On Thu, Aug 24, 2017 at 12:34 PM Thomas Kluyver > wrote:>> Is there a reason to ask for an 'unclean' > build, though? There may be >> a performance optimisation from reusing cached data,> > For the same reason you

Re: [Distutils] Fwd: Re: PEP 517 again

> For the same reason you would ever ask for incremental builds, to more quickly iterate while hacking, imagining that you are using the PEP 517 interface to develop, perhaps to have a uniform interface to patch something when you are not familiar with exactly the build system it uses. And so I

Re: [Distutils] Fwd: Re: PEP 517 again

On Thu, Aug 24, 2017 at 12:34 PM Thomas Kluyver wrote: > On Thu, Aug 24, 2017, at 05:26 PM, Daniel Holth wrote: > > by including the build_dir parameter, a nearly universal build system > concept, pip gets an elegant way to ask for either a clean or unclean build. > > > Is

Re: [Distutils] Fwd: Re: PEP 517 again

build_dir: bdist_wheel sometimes has a problem with unclean builds. admittedly this could be fixed. pip copies entire source tree, including unclean build/ directory, to tmpdir bdist_wheel runs in tmpdir, including copied leftover build/ data in wheel by including the build_dir parameter, a

Re: [Distutils] Fwd: Re: PEP 517 again

If the entire idea of copying out-of-tree is to work around setuptools deficiencies, then perhaps it would be a better idea to push this onto the setuptools build backend rather than bring these problems into PEP 517? 2017-08-24 10:32 GMT-05:00 xoviat : > May I ask what is

Re: [Distutils] Fwd: Re: PEP 517 again

May I ask what is wrong *in principle* with the setuptools "build" folder (other than the fact that it does not contain all tree changes)? 2017-08-24 10:27 GMT-05:00 xoviat : > That is actually the first time that build_dir makes sense to me now. > Thank you. > > 2017-08-24

Re: [Distutils] Fwd: Re: PEP 517 again

That is actually the first time that build_dir makes sense to me now. Thank you. 2017-08-24 10:24 GMT-05:00 Paul Moore : > On 24 August 2017 at 16:20, xoviat wrote: > >> I *do* care about telling backends we don't want "different > > results from those

Re: [Distutils] Fwd: Re: PEP 517 again

On 24 August 2017 at 16:23, xoviat wrote: >> I don't recall you having said "we do sdist->wheel then fall back to > requesting wheels directly" > > You're correct. I did not say that because build_sdist is required. Wait, what? Flit specifically needs to be allowed to refuse

Re: [Distutils] Fwd: Re: PEP 517 again

> I know Daniel has been involved in the discussion and I don't think he's raised any such objection about the PEP (and he developed enscons, so he has direct experience of writing backends). Daniel raised that specific objection. 2017-08-24 10:21 GMT-05:00 Paul Moore : >

Re: [Distutils] Fwd: Re: PEP 517 again

On 24 August 2017 at 16:20, xoviat wrote: >> I *do* care about telling backends we don't want "different > results from those that would be obtained by exporting an sdist > first". > > I completely agree with this statement, but I don't believe that it can or > should be

Re: [Distutils] Fwd: Re: PEP 517 again

> I don't recall you having said "we do sdist->wheel then fall back to requesting wheels directly" You're correct. I did not say that because build_sdist is required. 2017-08-24 10:21 GMT-05:00 Paul Moore : > On 24 August 2017 at 16:15, xoviat wrote: >

Re: [Distutils] Fwd: Re: PEP 517 again

On 24 August 2017 at 16:15, xoviat wrote: >> 2. I'm not completely clear on how pip's implementation will work - I > think the intention is to always build a sdist and build a wheel from > that, unless the backend reports it can't build a sdist, in which case > we ask it to

Re: [Distutils] Fwd: Re: PEP 517 again

> I *do* care about telling backends we don't want "different results from those that would be obtained by exporting an sdist first". I completely agree with this statement, but I don't believe that it can or should be accomplished with this parameter. Let me just quote the process that I

Re: [Distutils] Fwd: Re: PEP 517 again

> 2. I'm not completely clear on how pip's implementation will work - I think the intention is to always build a sdist and build a wheel from that, unless the backend reports it can't build a sdist, in which case we ask it to build a wheel directly. This was the exact process that I proposed, but

Re: [Distutils] Fwd: Re: PEP 517 again

On 25 August 2017 at 00:51, Paul Moore wrote: > 4. The point of all this is that the definition of build_directory > says "If build_directory is None, the backend may do an 'in place' > build which modifies the source directory and may produce different > results from those

Re: [Distutils] Fwd: Re: PEP 517 again

On 24 August 2017 at 15:26, Thomas Kluyver wrote: > I particularly want to hear from Nick, Daniel, Donald and Paul on their > understanding of the build_directory parameter. There were reasons why it > got added, and even if those reasons no longer seem so convincing to me,

Re: [Distutils] Fwd: Re: PEP 517 again

Thomas, I did not receive the last message. On Aug 24, 2017 9:31 AM, "Thomas Kluyver" wrote: > On Thu, Aug 24, 2017, at 03:29 PM, xoviat wrote: > > I understand that, but what I disagree with is modifying build_wheel to > make up for a lack of consensus on editable

Re: [Distutils] Fwd: Re: PEP 517 again

On Thu, Aug 24, 2017, at 03:29 PM, xoviat wrote: > I understand that, but what I disagree with is modifying build_wheel > to make up for a lack of consensus on editable installs. The reasoning for the build_directory parameter was not based around editable installs. This is the confusion I was

Re: [Distutils] Fwd: Re: PEP 517 again

I understand that, but what I disagree with is modifying build_wheel to make up for a lack of consensus on editable installs. On Aug 24, 2017 9:27 AM, "Thomas Kluyver" wrote: > On Thu, Aug 24, 2017, at 03:05 PM, xoviat wrote: > > During development, you want to use an

Re: [Distutils] Fwd: Re: PEP 517 again

On Thu, Aug 24, 2017, at 03:05 PM, xoviat wrote: > During development, you want to use an editable install, which does > modify the source tree. I think it was before you joined the discussion, but editable installs came up before and have deliberately been left out of PEP 517 because of the

Re: [Distutils] Fwd: Re: PEP 517 again

WRT the return value, I don't really care. But I am strongly in favor of removing the build_dir option as I think that the entire premise is wrong. During development, you want to use an editable install, which does modify the source tree. And if you're just installing, then you don't care about

Re: [Distutils] Fwd: Re: PEP 517 again

On 24 August 2017 at 10:41, xoviat wrote: > On Aug 24, 2017 8:28 AM, "Thomas Kluyver" wrote: > >> On Thu, Aug 24, 2017, at 02:21 PM, xoviat wrote: >> >> I mean is this golang or Python? In Python, you raise notimplementederror. >> >> >> But there's a