Re: [Distutils] Fwd: Re: PEP 517 again

Okay, but I would like to hear Paul's response to that first. On Aug 25, 2017 12:54 AM, "Nick Coghlan" wrote: > On 25 August 2017 at 14:33, xoviat wrote: > > Just this morning, Paul said the following: > > > > That step's the problem. If the frontend does

Re: [Distutils] Fwd: Re: PEP 517 again

On 25 August 2017 at 14:33, xoviat wrote: > Just this morning, Paul said the following: > > That step's the problem. If the frontend does that it can potentially > be copying a lot of unneeded stuff (VCS history, for example). We > tried that with pip and it was a major issue.

Re: [Distutils] Fwd: Re: PEP 517 again

And one more thing: it would probably be prescient to require a get_backend_info hook that returns the name and url of the backend so that pip can direct people where to file bugs when there are problems. On Aug 24, 2017 11:52 PM, "xoviat" wrote: > Actually after reading the

Re: [Distutils] Fwd: Re: PEP 517 again

Actually after reading the email that Nathaniel referenced, I disagree with Nick's position on in-tree builds. It's absolutely necessary because the tree may be read only and it's enforceable by pip through a simple recursive list. On Aug 24, 2017 11:33 PM, "xoviat" wrote: >

Re: [Distutils] Fwd: Re: PEP 517 again

Just this morning, Paul said the following: That step's the problem. If the frontend does that it can potentially be copying a lot of unneeded stuff (VCS history, for example). We tried that with pip and it was a major issue. That problem is the *whole point* of all the discussions about the

Re: [Distutils] Fwd: Re: PEP 517 again

On Thu, Aug 24, 2017 at 9:17 PM, xoviat wrote: > > I'm *not* OK with banning in-tree builds in the spec, since that's > > both unnecessary and unenforceable > > Well then either we can trust the backend or we cannot. If we can, then this > is both necessary and enforceable. If

Re: [Distutils] Fwd: Re: PEP 517 again

I'm *not* OK with banning in-tree builds in the spec, since that's both unnecessary and unenforceable Well then either we can trust the backend or we cannot. If we can, then this is both necessary and enforceable. If not, then we're back to pip copying files. You can't make and argument that it's

Re: [Distutils] Fwd: Re: PEP 517 again

On 25 August 2017 at 10:08, Daniel Holth wrote: > It would be simpler if the only requirement was to produce a correct wheel. +1 To be clear on my current position: * due to the introduction of NotImplementedError, I'm now OK with explicitly out-of-tree builds being deferred

Re: [Distutils] PEP 517 again

In case anyone is interested, here is a mostly correct implementation of PEP 517 based on the behavior discussed here: https://github.com/pypa/setuptools/pull/1039/files#diff-522bd9826e33902f7f58bd003c6a370c As I said, there are a few items to be worked on still, but I really don't think that

Re: [Distutils] PEP 517 again

On Thu, Aug 24, 2017 at 6:11 AM, Thomas Kluyver wrote: > Nathaniel seems to be busy with other things at the moment, so I hope he > won't mind me passing on this list of things he'd like to resolve with > the draft PEP. I'll quote his comments and put my responses inline.

Re: [Distutils] PEP 517 again

Nathaniel: Are you okay with what has been proposed? On Aug 24, 2017 7:29 PM, "Nathaniel Smith" wrote: > On Thu, Aug 24, 2017 at 7:52 AM, Nick Coghlan wrote: > >>> - I don't understand how out-of-tree builds and prepare_build_metadata > >>> are supposed to

Re: [Distutils] PEP 517 again

On Thu, Aug 24, 2017 at 7:52 AM, Nick Coghlan wrote: >>> - I don't understand how out-of-tree builds and prepare_build_metadata >>> are supposed to interact. > > They don't, since the backend should only implement > prepare_build_metadata if it can generate the metadata

Re: [Distutils] Fwd: Re: PEP 517 again

The current proposal is to remove build_dir and leave it up to the backend. On Aug 24, 2017 7:07 PM, "Greg Ewing" wrote: > Paul Moore wrote: > >> 4. The point of all this is that the definition of build_directory >> says "If build_directory is None, the backend may

Re: [Distutils] Fwd: Re: PEP 517 again

It would be simpler if the only requirement was to produce a correct wheel. On Thu, Aug 24, 2017, 20:07 Greg Ewing wrote: > Paul Moore wrote: > > 4. The point of all this is that the definition of build_directory > > says "If build_directory is None, the backend may

Re: [Distutils] Fwd: Re: PEP 517 again

Paul Moore wrote: 4. The point of all this is that the definition of build_directory says "If build_directory is None, the backend may do an 'in place' build which modifies the source directory and may produce different results from those that would be obtained by exporting an sdist first". Is

Re: [Distutils] Fwd: Re: PEP 517 again

Actually I submitted a compliant PR 3 months ago when the PEP was first marked as accepted. Then it was marked as draft and I thought that I would just wait for distutils-sig to handle it. And then I came over here to find what was going on. On Aug 24, 2017 5:33 PM, "xoviat"

Re: [Distutils] Fwd: Re: PEP 517 again

I'm handling the setuptools backend. I will make sure that it complies with the clean directory. On Aug 24, 2017 5:30 PM, "Paul Moore" wrote: > On 24 August 2017 at 20:00, Thomas Kluyver wrote: > > So for the time being, I'd prefer to say that the

Re: [Distutils] Fwd: Re: PEP 517 again

On 24 August 2017 at 20:00, Thomas Kluyver wrote: > So for the time being, I'd prefer to say that the build_wheel() hook should > not create or modify files in the source directory. Backends may keep their > own caches elsewhere to speed up repeated builds, but any more

[Distutils] Fwd: Fwd: Re: PEP 517 again

-- Forwarded message -- From: xoviat Date: 2017-08-24 14:26 GMT-05:00 Subject: Re: [Distutils] Fwd: Re: PEP 517 again To: Thomas Kluyver And also, are Nick and Paul okay with this? 2017-08-24 14:24 GMT-05:00 xoviat : >

Re: [Distutils] Fwd: Re: PEP 517 again

So in summary: - NotImplementedError - Remove build_dir and specify that no modifications can be made to the source directory On the third issue: I understand where Nathaniel is coming from in regards to sys.path. But there should at least be an option for this because of for example, numpy:

Re: [Distutils] Fwd: Re: PEP 517 again

On Thu, Aug 24, 2017, at 07:30 PM, Leonardo Rochael Almeida wrote: > It looks like a lot of trouble for a feature that is designed to > solve a problem for a very thin intersection of the Venn diagram of > people who:> > a. wants to quickly iterate while hacking a package > > b. doesn't want

Re: [Distutils] Fwd: Re: PEP 517 again

The distutils build/ folder is what moves if you pass build_dir to the proposed function. On Thu, Aug 24, 2017, 14:39 xoviat wrote: > > It looks like a lot of trouble for a feature that is designed to solve a > problem for a very thin intersection of the Venn diagram of people

Re: [Distutils] Fwd: Re: PEP 517 again

> It looks like a lot of trouble for a feature that is designed to solve a problem for a very thin intersection of the Venn diagram of people who: For the record, I don't agree at all that build_wheel_incremental should be specified here. The suggestion is simply a compromise that I can tolerate

Re: [Distutils] Fwd: Re: PEP 517 again

On 24 August 2017 at 15:13, Thomas Kluyver wrote: > On Thu, Aug 24, 2017, at 06:20 PM, Daniel Holth wrote: > > On Thu, Aug 24, 2017 at 12:34 PM Thomas Kluyver > wrote: > > Is there a reason to ask for an 'unclean' build, though? There may be a >

Re: [Distutils] Fwd: Re: PEP 517 again

To be clear, is everyone okay with the following? build_wheel(wheel_directory, config_settings=None, metadata_directory=None): [REQUIRED] build_wheel_incremental(wheel_directory, config_settings=None, metadata_directory=None): [OPTIONAL] I still think that the frontend should not provide a

Re: [Distutils] Fwd: Re: PEP 517 again

On Thu, Aug 24, 2017, at 06:20 PM, Daniel Holth wrote: > On Thu, Aug 24, 2017 at 12:34 PM Thomas Kluyver > wrote:>> Is there a reason to ask for an 'unclean' > build, though? There may be >> a performance optimisation from reusing cached data,> > For the same reason you

Re: [Distutils] Fwd: Re: PEP 517 again

> For the same reason you would ever ask for incremental builds, to more quickly iterate while hacking, imagining that you are using the PEP 517 interface to develop, perhaps to have a uniform interface to patch something when you are not familiar with exactly the build system it uses. And so I

Re: [Distutils] Fwd: Re: PEP 517 again

On Thu, Aug 24, 2017 at 12:34 PM Thomas Kluyver wrote: > On Thu, Aug 24, 2017, at 05:26 PM, Daniel Holth wrote: > > by including the build_dir parameter, a nearly universal build system > concept, pip gets an elegant way to ask for either a clean or unclean build. > > > Is

Re: [Distutils] Fwd: Re: PEP 517 again

build_dir: bdist_wheel sometimes has a problem with unclean builds. admittedly this could be fixed. pip copies entire source tree, including unclean build/ directory, to tmpdir bdist_wheel runs in tmpdir, including copied leftover build/ data in wheel by including the build_dir parameter, a

Re: [Distutils] Fwd: Re: PEP 517 again

If the entire idea of copying out-of-tree is to work around setuptools deficiencies, then perhaps it would be a better idea to push this onto the setuptools build backend rather than bring these problems into PEP 517? 2017-08-24 10:32 GMT-05:00 xoviat : > May I ask what is

Re: [Distutils] Fwd: Re: PEP 517 again

May I ask what is wrong *in principle* with the setuptools "build" folder (other than the fact that it does not contain all tree changes)? 2017-08-24 10:27 GMT-05:00 xoviat : > That is actually the first time that build_dir makes sense to me now. > Thank you. > > 2017-08-24

Re: [Distutils] Fwd: Re: PEP 517 again

That is actually the first time that build_dir makes sense to me now. Thank you. 2017-08-24 10:24 GMT-05:00 Paul Moore : > On 24 August 2017 at 16:20, xoviat wrote: > >> I *do* care about telling backends we don't want "different > > results from those

Re: [Distutils] Fwd: Re: PEP 517 again

On 24 August 2017 at 16:23, xoviat wrote: >> I don't recall you having said "we do sdist->wheel then fall back to > requesting wheels directly" > > You're correct. I did not say that because build_sdist is required. Wait, what? Flit specifically needs to be allowed to refuse

Re: [Distutils] Fwd: Re: PEP 517 again

> I know Daniel has been involved in the discussion and I don't think he's raised any such objection about the PEP (and he developed enscons, so he has direct experience of writing backends). Daniel raised that specific objection. 2017-08-24 10:21 GMT-05:00 Paul Moore : >

Re: [Distutils] Fwd: Re: PEP 517 again

On 24 August 2017 at 16:20, xoviat wrote: >> I *do* care about telling backends we don't want "different > results from those that would be obtained by exporting an sdist > first". > > I completely agree with this statement, but I don't believe that it can or > should be

Re: [Distutils] Fwd: Re: PEP 517 again

> I don't recall you having said "we do sdist->wheel then fall back to requesting wheels directly" You're correct. I did not say that because build_sdist is required. 2017-08-24 10:21 GMT-05:00 Paul Moore : > On 24 August 2017 at 16:15, xoviat wrote: >

Re: [Distutils] Fwd: Re: PEP 517 again

On 24 August 2017 at 16:15, xoviat wrote: >> 2. I'm not completely clear on how pip's implementation will work - I > think the intention is to always build a sdist and build a wheel from > that, unless the backend reports it can't build a sdist, in which case > we ask it to

Re: [Distutils] Fwd: Re: PEP 517 again

> I *do* care about telling backends we don't want "different results from those that would be obtained by exporting an sdist first". I completely agree with this statement, but I don't believe that it can or should be accomplished with this parameter. Let me just quote the process that I

Re: [Distutils] Fwd: Re: PEP 517 again

> 2. I'm not completely clear on how pip's implementation will work - I think the intention is to always build a sdist and build a wheel from that, unless the backend reports it can't build a sdist, in which case we ask it to build a wheel directly. This was the exact process that I proposed, but

Re: [Distutils] Fwd: Re: PEP 517 again

On 25 August 2017 at 00:51, Paul Moore wrote: > 4. The point of all this is that the definition of build_directory > says "If build_directory is None, the backend may do an 'in place' > build which modifies the source directory and may produce different > results from those

Re: [Distutils] PEP 517 again

On 24 August 2017 at 23:11, Thomas Kluyver wrote: > Nathaniel seems to be busy with other things at the moment, so I hope he > won't mind me passing on this list of things he'd like to resolve with > the draft PEP. I'll quote his comments and put my responses inline. > >> -

Re: [Distutils] Fwd: Re: PEP 517 again

On 24 August 2017 at 15:26, Thomas Kluyver wrote: > I particularly want to hear from Nick, Daniel, Donald and Paul on their > understanding of the build_directory parameter. There were reasons why it > got added, and even if those reasons no longer seem so convincing to me,

Re: [Distutils] Fwd: Re: PEP 517 again

Thomas, I did not receive the last message. On Aug 24, 2017 9:31 AM, "Thomas Kluyver" wrote: > On Thu, Aug 24, 2017, at 03:29 PM, xoviat wrote: > > I understand that, but what I disagree with is modifying build_wheel to > make up for a lack of consensus on editable

Re: [Distutils] Fwd: Re: PEP 517 again

On Thu, Aug 24, 2017, at 03:29 PM, xoviat wrote: > I understand that, but what I disagree with is modifying build_wheel > to make up for a lack of consensus on editable installs. The reasoning for the build_directory parameter was not based around editable installs. This is the confusion I was

Re: [Distutils] Fwd: Re: PEP 517 again

I understand that, but what I disagree with is modifying build_wheel to make up for a lack of consensus on editable installs. On Aug 24, 2017 9:27 AM, "Thomas Kluyver" wrote: > On Thu, Aug 24, 2017, at 03:05 PM, xoviat wrote: > > During development, you want to use an

Re: [Distutils] Fwd: Re: PEP 517 again

On Thu, Aug 24, 2017, at 03:05 PM, xoviat wrote: > During development, you want to use an editable install, which does > modify the source tree. I think it was before you joined the discussion, but editable installs came up before and have deliberately been left out of PEP 517 because of the

Re: [Distutils] Fwd: Re: PEP 517 again

WRT the return value, I don't really care. But I am strongly in favor of removing the build_dir option as I think that the entire premise is wrong. During development, you want to use an editable install, which does modify the source tree. And if you're just installing, then you don't care about

Re: [Distutils] Fwd: Re: PEP 517 again

On 24 August 2017 at 10:41, xoviat wrote: > On Aug 24, 2017 8:28 AM, "Thomas Kluyver" wrote: > >> On Thu, Aug 24, 2017, at 02:21 PM, xoviat wrote: >> >> I mean is this golang or Python? In Python, you raise notimplementederror. >> >> >> But there's a

[Distutils] Fwd: Re: PEP 517 again

-- Forwarded message -- From: "xoviat" Date: Aug 24, 2017 8:39 AM Subject: Re: [Distutils] PEP 517 again To: "Thomas Kluyver" Cc: That's actually the general argument against exceptions and why golang doesn't have them. I have not seen

Re: [Distutils] PEP 517 again

On 24 August 2017 at 10:28, Thomas Kluyver wrote: > On Thu, Aug 24, 2017, at 02:21 PM, xoviat wrote: > > I mean is this golang or Python? In Python, you raise notimplementederror. > > > But there's a NotImplemented singleton in Python as well. The argument for > using a

Re: [Distutils] PEP 517 again

On Thu, Aug 24, 2017, at 02:21 PM, xoviat wrote: > I mean is this golang or Python? In Python, you raise > notimplementederror. But there's a NotImplemented singleton in Python as well. The argument for using a return value is that the hook code has to deliberately return that, whereas a

Re: [Distutils] Mirroring PyPI JSON Locally

Thanks all for your points. Is it fine if I get a Informational PEP going to discuss 'Metadata Repository API'? I will structure it similar to PEP503, but also talk about: - The data exposed today (Specification) -- And possibly call on some PyPI people to correct me where I guess wrong - How

Re: [Distutils] PEP 517 again

I mean is this golang or Python? In Python, you raise notimplementederror. On Aug 24, 2017 8:17 AM, "Thomas Kluyver" wrote: > Nathaniel seems to be busy with other things at the moment, so I hope he > won't mind me passing on this list of things he'd like to resolve with >

[Distutils] PEP 517 again

Nathaniel seems to be busy with other things at the moment, so I hope he won't mind me passing on this list of things he'd like to resolve with the draft PEP. I'll quote his comments and put my responses inline. > - needs some sort of NotImplemented(Error) handling specified We've gone round