Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-03 Thread Nick Coghlan
On 4 June 2017 at 10:39, Ralf Gommers wrote: > > > On Sat, Jun 3, 2017 at 8:59 PM, Paul Moore wrote: >> >> On 3 June 2017 at 03:14, Nathaniel Smith wrote: >> > So far my belief is that packages with expensive build processes are >> >

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-03 Thread Ralf Gommers
On Sat, Jun 3, 2017 at 8:59 PM, Paul Moore wrote: > On 3 June 2017 at 03:14, Nathaniel Smith wrote: > > So far my belief is that packages with expensive build processes are > > going to ignore you and implement, ship, document, and recommend the > > direct

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-03 Thread Nathaniel Smith
On Sat, Jun 3, 2017 at 5:09 PM, C Anthony Risinger wrote: > Fair enough. It seems like there will almost certainly emerge some way of > chaining small "source tree mutators" (leading to an sdist) with truly > custom build backends (that may ultimately terminate on either

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-03 Thread C Anthony Risinger
On Jun 3, 2017 4:47 AM, "Paul Moore" wrote: On 3 June 2017 at 04:53, C Anthony Risinger wrote: > I want to make sure I understand what I'd need to do, as a user, in a post > PEP 517 world. Say I wanted to accomplish the following three things: > >

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-03 Thread Paul Moore
On 3 June 2017 at 11:09, Thomas Kluyver wrote: > More generally, though, I'd question why you don't want those files to > be in an sdist? Why should an sdist be any different to a snapshot of > your VCS at release time, including all of your thoughts and tools used > in

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-03 Thread Nick Coghlan
On 3 June 2017 at 20:09, Thomas Kluyver wrote: > On Sat, Jun 3, 2017, at 10:55 AM, Paul Moore wrote: >> This is getting very off-topic, but what if I wanted to patch the >> source and then build a sdist to put into my local PyPI index? I >> presume the answer is that I

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-03 Thread Nick Coghlan
On 3 June 2017 at 19:45, Thomas Kluyver wrote: > On Sat, Jun 3, 2017, at 03:14 AM, Nathaniel Smith wrote: >> If the pip devs don't trust build systems in general, but (as >> suggested by copy_files discussion) are ok with trusting them if they >> promise to be super

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-03 Thread Thomas Kluyver
On Sat, Jun 3, 2017, at 10:55 AM, Paul Moore wrote: > This is getting very off-topic, but what if I wanted to patch the > source and then build a sdist to put into my local PyPI index? I > presume the answer is that I either have to checkout the original > sources from VCS or I have to build only

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-03 Thread Paul Moore
On 3 June 2017 at 10:45, Thomas Kluyver wrote: >> One thing that's not clear to me: a crucial use case for sdists is (1) >> download, (2) unpack, (3) patch the source, possibly adding new files, >> (4) build and install. (After all, the whole reason we insist on >>

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-03 Thread Paul Moore
On 3 June 2017 at 04:53, C Anthony Risinger wrote: > I want to make sure I understand what I'd need to do, as a user, in a post > PEP 517 world. Say I wanted to accomplish the following three things: > > * Generate version info from my VCS > * Generate .h and .c from

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-03 Thread Thomas Kluyver
On Sat, Jun 3, 2017, at 03:14 AM, Nathaniel Smith wrote: > If the pip devs don't trust build systems in general, but (as > suggested by copy_files discussion) are ok with trusting them if they > promise to be super trustworthy, alternate proposal: > - add a 'in_place_build_safe = True' hook, which

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-03 Thread Paul Moore
On 3 June 2017 at 08:47, Donald Stufft wrote: > > That also means that we can adjust our answer to it in the future. If such a > tool gets built and a lot of people end up using it and asking for it in > pip, we can revisit that decision in a future version of pip. Part of the >

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-03 Thread Paul Moore
On 3 June 2017 at 09:59, Paul Moore wrote: > On 3 June 2017 at 03:14, Nathaniel Smith wrote: >> So far my belief is that packages with expensive build processes are >> going to ignore you and implement, ship, document, and recommend the >> direct

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-03 Thread Paul Moore
On 3 June 2017 at 03:14, Nathaniel Smith wrote: > So far my belief is that packages with expensive build processes are > going to ignore you and implement, ship, document, and recommend the > direct source-tree->wheel path for developer builds. You can force the >

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-03 Thread Nick Coghlan
On 3 June 2017 at 15:53, Nathaniel Smith wrote: > That's not what I'm talking about. The case I'm talking about is, > like, a baby dev taking their first steps, or someone trying to get a > build of a package working on an unusual system: > > git clone /numpy.git > cd numpy >

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-02 Thread Nathaniel Smith
On Fri, Jun 2, 2017 at 10:21 PM, Nick Coghlan wrote: > On 3 June 2017 at 13:38, Donald Stufft wrote: >> However, what I was able to find was what appears to be the original reason >> pip started copying the directory to begin with, >>

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-02 Thread Nathaniel Smith
On Fri, Jun 2, 2017 at 8:38 PM, Donald Stufft wrote: > > On Jun 2, 2017, at 10:14 PM, Nathaniel Smith wrote: > >> So far my belief is that packages with expensive build processes are >> going to ignore you and implement, ship, document, and recommend the >>

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-02 Thread Nick Coghlan
On 3 June 2017 at 13:38, Donald Stufft wrote: > However, what I was able to find was what appears to be the original reason > pip started copying the directory to begin with, > https://github.com/pypa/pip/issues/178 which was caused by the build system > reusing the build

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-02 Thread C Anthony Risinger
On Fri, Jun 2, 2017 at 3:58 PM, Thomas Kluyver wrote: > On Fri, Jun 2, 2017, at 06:14 PM, Donald Stufft wrote: > > The steps here would basically be (for building from something that isn’t > already a .tar.gz or a .whl): > > > That sounds OK to me. I think the only

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-02 Thread Donald Stufft
> On Jun 2, 2017, at 10:14 PM, Nathaniel Smith wrote: > > On Fri, Jun 2, 2017 at 9:39 AM, Paul Moore wrote: >> Note that there's an implication here - if we define the build process >> in terms of the effect of "going via a sdist", then we need to at >>

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-02 Thread Nathaniel Smith
On Fri, Jun 2, 2017 at 9:39 AM, Paul Moore wrote: > Note that there's an implication here - if we define the build process > in terms of the effect of "going via a sdist", then we need to at > least have an intuitive understanding of what that means in practice. > I don't

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-02 Thread Thomas Kluyver
On Fri, Jun 2, 2017, at 06:14 PM, Donald Stufft wrote: > The steps here would basically be (for building from something that > isn’t already a .tar.gz or a .whl): That sounds OK to me. I think the only remaining point of contention is whether the build_sdist hook should make an archive or an

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-02 Thread Donald Stufft
> On Jun 2, 2017, at 12:39 PM, Paul Moore wrote: > >> >> * The copy_the_files hook is optional, if it exists it SHOULD produce a tree >> that when the build_wheel hook is called in it, will produce a wheel that is >> equivalent to one that would have been built had the

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-02 Thread Nick Coghlan
On 3 June 2017 at 02:50, Thomas Kluyver wrote: > On Fri, Jun 2, 2017, at 05:26 PM, Nick Coghlan wrote: >> Could you provide a little more detail as to what you would do >> differently in exporting the contents of an sdist that wouldn't apply >> to export a build tree? (aside

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-02 Thread Paul Moore
On 2 June 2017 at 18:02, Donald Stufft wrote: > > I’m pretty sure the only reason we don’t do it now is because nobody has had > the time to make it happen yet. The problems before weren’t from going via > sdist, they were from trying to modify our copy tree implementation to >

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-02 Thread Donald Stufft
> On Jun 2, 2017, at 12:39 PM, Paul Moore wrote: > > On 2 June 2017 at 16:27, Donald Stufft > wrote: >> So my preference is that everything goes through the sdist step as I think >> that is most likely to provide consistent

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-02 Thread Thomas Kluyver
On Fri, Jun 2, 2017, at 05:26 PM, Nick Coghlan wrote: > Could you provide a little more detail as to what you would do > differently in exporting the contents of an sdist that wouldn't apply > to export a build tree? (aside from skipping emitting PKG-INFO) When creating an sdist, I query the VCS

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-02 Thread Paul Moore
On 2 June 2017 at 16:27, Donald Stufft wrote: > So my preference is that everything goes through the sdist step as I think > that is most likely to provide consistent builds everywhere both from a VCS > checkout and from a sdist that was released to PyPI. Agreed. That's the

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-02 Thread Nick Coghlan
[Note: I've reverted the PEP to Draft status while this discussion is ongoing: https://github.com/python/peps/blob/master/pep-0517.txt] On 3 June 2017 at 00:56, Thomas Kluyver wrote: > On Fri, Jun 2, 2017, at 03:41 PM, Nick Coghlan wrote: >> Instead, the flow that I think

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-02 Thread Jeremy Stanley
On 2017-06-02 15:12:19 +0100 (+0100), Paul Moore wrote: [...] > I'm struggling to understand why building a sdist in flit should need > a VCS. It bothers me that I'm missing something important about how > backends will work, that explains why (for example) you can't create a > sdist from an

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-02 Thread Donald Stufft
> On Jun 2, 2017, at 10:41 AM, Nick Coghlan wrote: > > On 2 June 2017 at 23:42, Thomas Kluyver wrote: >> As was suggested at some point, I have added a build_sdist hook to my >> PR, with the following details: >> >> - A brief definition of the minimal

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-02 Thread Thomas Kluyver
On Fri, Jun 2, 2017, at 03:41 PM, Nick Coghlan wrote: > Instead, the flow that I think makes sense is "VCS -> sdist tree [-> > sdist tree -> sdist tree -> ...] -> wheel", and the above model where > the export filtering is only used when PKG-INFO doesn't exist yet will > give us that. I still

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-02 Thread Nick Coghlan
On 2 June 2017 at 23:42, Thomas Kluyver wrote: > As was suggested at some point, I have added a build_sdist hook to my > PR, with the following details: > > - A brief definition of the minimal requirements of an sdist. > - I have limited the definition to gzipped tarballs.

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-02 Thread Thomas Kluyver
On Fri, Jun 2, 2017, at 03:12 PM, Paul Moore wrote: > I'm struggling to understand why building a sdist in flit should need > a VCS. It bothers me that I'm missing something important about how > backends will work, that explains why (for example) you can't create a > sdist from an export of a VCS

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-02 Thread Paul Moore
On 2 June 2017 at 14:42, Thomas Kluyver wrote: > - Backends (like flit) where building an sdist has extra requirements > should define prepare_build_files. I'm struggling to understand why building a sdist in flit should need a VCS. It bothers me that I'm missing something

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-02 Thread Thomas Kluyver
As was suggested at some point, I have added a build_sdist hook to my PR, with the following details: - A brief definition of the minimal requirements of an sdist. - I have limited the definition to gzipped tarballs. Zip files also work as sdists, but we're moving towards standardising on

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-02 Thread Nick Coghlan
On 2 June 2017 at 18:37, Paul Moore wrote: > On 2 June 2017 at 08:49, Nick Coghlan wrote: >> The last point means that requiring an sdist export command shouldn't >> impose an unreasonable burden on backend developers, as the only >> differences between

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-02 Thread Paul Moore
On 2 June 2017 at 08:49, Nick Coghlan wrote: > The last point means that requiring an sdist export command shouldn't > impose an unreasonable burden on backend developers, as the only > differences between "generate an sdist tree" and "export a build tree" > would be: > > -

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-02 Thread Nick Coghlan
On 2 June 2017 at 06:45, Brett Cannon wrote: > And so I think in a very wordy way, I just said we need to stop saying "pip > needs a standardized way to produce an sdist" and instead start saying > "twine needs a way to produce an sdist". And that leads to the question > about

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-01 Thread Donald Stufft
> On Jun 1, 2017, at 9:40 PM, Jeremy Stanley wrote: > > On 2017-06-01 21:09:57 -0400 (-0400), Donald Stufft wrote: > [...] >> I think a separate tool for each of these roles is somewhat user >> unfriendly TBH. > [...] > > I'll do my best not to be offended that you don't

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-01 Thread Glyph
> On Jun 1, 2017, at 6:09 PM, Donald Stufft wrote: > > I sometimes wonder if we should be folding twine into pip itself Yes please. WTB `pip upload`. -g___ Distutils-SIG maillist - Distutils-SIG@python.org

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-01 Thread Jeremy Stanley
On 2017-06-01 21:09:57 -0400 (-0400), Donald Stufft wrote: [...] > I think a separate tool for each of these roles is somewhat user > unfriendly TBH. [...] I'll do my best not to be offended that you don't consider me a user (or representative of some broader class of users). ;) At any rate, I

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-01 Thread Donald Stufft
> On Jun 1, 2017, at 7:53 PM, Jeremy Stanley wrote: > > On 2017-06-01 20:45:53 + (+), Brett Cannon wrote: > [...] >> I think *twine* is the tool that needs a way to specify how to >> produce an sdist. If we want to view twine as the tool to upload >> artifacts to PyPI

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-01 Thread Jeremy Stanley
On 2017-06-01 20:45:53 + (+), Brett Cannon wrote: [...] > I think *twine* is the tool that needs a way to specify how to > produce an sdist. If we want to view twine as the tool to upload > artifacts to PyPI then we need twine to know how to produce sdists > and wheels in a PEP 517 world,

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-01 Thread Donald Stufft
> On Jun 1, 2017, at 6:28 PM, Paul Moore wrote: > > On 1 June 2017 at 23:14, Thomas Kluyver wrote: >> On Thu, Jun 1, 2017, at 10:49 PM, Paul Moore wrote: >>> pip also needs a way to deal with "pip install . In >>> this case, pip (under its current

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-01 Thread Paul Moore
On 1 June 2017 at 23:14, Thomas Kluyver wrote: > On Thu, Jun 1, 2017, at 10:49 PM, Paul Moore wrote: >> pip also needs a way to deal with "pip install . In >> this case, pip (under its current model) copies that directory to a >> working area. In that area, it runs the build

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-01 Thread Thomas Kluyver
On Thu, Jun 1, 2017, at 10:49 PM, Paul Moore wrote: > pip also needs a way to deal with "pip install . In > this case, pip (under its current model) copies that directory to a > working area. In that area, it runs the build command to create a > wheel, and proceeds from there. In principle,

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-01 Thread Paul Moore
On 1 June 2017 at 21:45, Brett Cannon wrote: > And so I think in a very wordy way, I just said we need to stop saying "pip > needs a standardized way to produce an sdist" and instead start saying > "twine needs a way to produce an sdist". And that leads to the question > about

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-01 Thread Brett Cannon
On Wed, 31 May 2017 at 14:14 Thomas Kluyver wrote: > On Wed, May 31, 2017, at 09:16 PM, Donald Stufft wrote: > > How you build the release-quality sdist isn’t really of concern of PEP 517 > any more than building a release quality wheel is, it’s up to the build > tool to

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-01 Thread C Anthony Risinger
On Thu, Jun 1, 2017 at 1:22 PM, Donald Stufft wrote: > > On Jun 1, 2017, at 2:12 PM, C Anthony Risinger > wrote: > > Because the build itself can output additional source files, that may be > desirable to include in an sdist later, I honestly don't

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-01 Thread Donald Stufft
> On Jun 1, 2017, at 2:12 PM, C Anthony Risinger > wrote: > > Because the build itself can output additional source files, that may be > desirable to include in an sdist later, I honestly don't think you can pass > through a "proper" sdist before a wheel. I think

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-01 Thread Jeremy Stanley
On 2017-05-31 20:08:51 -0400 (-0400), Donald Stufft wrote: [...] > Both {name} and {version} MUST have any - characters escaped to a > _ to match the escaping done by Wheel. Thus a sdist for a project > named foo-bar with version 1.0-2 which is using a .tar.gz > container for the sdist would

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-01 Thread C Anthony Risinger
On Thu, Jun 1, 2017 at 5:34 AM, Donald Stufft wrote: > > On Jun 1, 2017, at 3:44 AM, Paul Moore wrote: > > On 1 June 2017 at 01:08, Donald Stufft wrote: > > A sdist is a .tar.gz or a .zip file with a directory structure like (along >

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-01 Thread Marius Gedminas
On Wed, May 31, 2017 at 08:08:51PM -0400, Donald Stufft wrote: > I think this should cover the case of actually making the project pip > installable (assuming of course the setup.py or build backend doesn’t do > something silly like always sys.exit(1) instead of produce the expected > outcome) My

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-01 Thread Donald Stufft
> On Jun 1, 2017, at 3:44 AM, Paul Moore wrote: > > On 1 June 2017 at 01:08, Donald Stufft wrote: >> A sdist is a .tar.gz or a .zip file with a directory structure like (along >> with whatever additional files the project needs in the sdist): > [...] > >

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-01 Thread Paul Moore
On 1 June 2017 at 01:08, Donald Stufft wrote: > A sdist is a .tar.gz or a .zip file with a directory structure like (along > with whatever additional files the project needs in the sdist): [...] I'm confused. Isn't this basically what PEP 517 says already? You've added some

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Donald Stufft
> On May 31, 2017, at 6:48 PM, Paul Moore wrote: > > On 31 May 2017 at 22:13, Thomas Kluyver wrote: >> >> But if we have a hook for building something called an sdist, we need to >> define what an sdist is. > > OK, so can we do that? > > At the

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Fred Drake
On Wed, May 31, 2017 at 6:48 PM, Paul Moore wrote: > What do consumers of the sdist format want to do? I don't actually know, ... > They also want to publish the sdist to PyPI - so they need to > name it according to the current convention. I think we can rule this out for

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Paul Moore
On 31 May 2017 at 22:13, Thomas Kluyver wrote: > > But if we have a hook for building something called an sdist, we need to > define what an sdist is. OK, so can we do that? At the moment, we have a de facto definition of a sdist - it's something with a setup.py, some

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Thomas Kluyver
On Wed, May 31, 2017, at 09:16 PM, Donald Stufft wrote: > How you build the release-quality sdist isn’t really of concern of PEP > 517 any more than building a release quality wheel is, it’s up to the > build tool to implement that as it makes sense for them. But if we have a hook for building

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Donald Stufft
> On May 31, 2017, at 3:45 PM, Thomas Kluyver wrote: > > On Wed, May 31, 2017, at 08:20 PM, Donald Stufft wrote: >> I am struggling to figure out where there is opposition to simply exposing >> something in a standard way, that you were already planning on implementing

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Donald Stufft
> On May 31, 2017, at 3:38 PM, Paul Moore wrote: > > On 31 May 2017 at 20:20, Donald Stufft wrote: >> The most likely outcome if PEP 517 is implemented as defined and people who >> aren’t steeped in packaging lore hear about is, is they get excited about

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Thomas Kluyver
On Wed, May 31, 2017, at 08:20 PM, Donald Stufft wrote: > I am struggling to figure out where there is opposition to simply > exposing something in a standard way, that you were already planning > on implementing anyways. I have issues with it because: 1. Building a *release-quality* sdist is a

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Paul Moore
On 31 May 2017 at 20:20, Donald Stufft wrote: > The most likely outcome if PEP 517 is implemented as defined and people who > aren’t steeped in packaging lore hear about is, is they get excited about > being able to kill setup.py, they implement it, they find out some tool they

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Donald Stufft
> On May 31, 2017, at 3:12 PM, Thomas Kluyver wrote: > > On Wed, May 31, 2017, at 07:40 PM, Donald Stufft wrote: >> Taking pip completely off the table a second, let’s take a look at tox. >> Tox’s default mode of operation is to produce a sdist. Now let’s say I’m >>

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Thomas Kluyver
On Wed, May 31, 2017, at 07:40 PM, Donald Stufft wrote: > Taking pip completely off the table a second, let’s take a look at > tox. Tox’s default mode of operation is to produce a sdist. Now let’s > say I’m writing a project that I want to use PEP 517 and get rid of > setup.py, except now tox is

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Daniel Holth
In my experience tools think an archive is an sdist if it is named properly, contains PKG-INFO with a minimum number of fields Metadata-Version: 1.1Name: bobVersion: 1.0 and setup.py. setuptools sdists also contain .egg-info but it is unnecessary. On Wed, May 31, 2017 at 2:41 PM Donald Stufft

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Donald Stufft
> On May 31, 2017, at 2:01 PM, Paul Moore wrote: > > On 31 May 2017 at 18:03, Donald Stufft wrote: >> No you’re correct, it currently just invokes ``setup.py sdist bdist_wheel``. >> The hook is needed so that Travis can have a singular tool to invoke

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Thomas Kluyver
I've made an alternative PR against PEP 517 which defines a prepare_build_files hook, defined in terms of copying the files needed to build a package rather than making an sdist. https://github.com/python/peps/pull/277 ___ Distutils-SIG maillist -

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Paul Moore
On 31 May 2017 at 18:03, Donald Stufft wrote: > No you’re correct, it currently just invokes ``setup.py sdist bdist_wheel``. > The hook is needed so that Travis can have a singular tool to invoke (likely > twine?) instead of needing to determine if it needs to invoke flit or >

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Thomas Kluyver
On Wed, May 31, 2017, at 06:03 PM, Donald Stufft wrote: > generating a sdist is an important thing to have be possible, and it > needs to be done in a way that it can be invoked generically. I agree that it needs to be possible to make an sdist, but can we leave the generic interface for it to be

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Donald Stufft
> On May 31, 2017, at 12:33 PM, Paul Moore wrote: > > On 31 May 2017 at 16:31, Donald Stufft wrote: >> The hook is also so a tool like tox or TravisCI or twine can produce a sdist >> that can be uploaded to PyPI or similar. > > Understood. The part that

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Paul Moore
On 31 May 2017 at 16:31, Donald Stufft wrote: > The hook is also so a tool like tox or TravisCI or twine can produce a sdist > that can be uploaded to PyPI or similar. Understood. The part that the backend can do (these are the files needed for the build process) is done via

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Daniel Holth
Sounds like we have bad support for out of  builds On Wed, May 31, 2017, 11:53 Thomas Kluyver wrote: > On Wed, May 31, 2017, at 04:31 PM, Donald Stufft wrote: > > The hook is also so a tool like tox or TravisCI or twine can produce a > > sdist that can be uploaded to PyPI

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Thomas Kluyver
On Wed, May 31, 2017, at 04:31 PM, Donald Stufft wrote: > The hook is also so a tool like tox or TravisCI or twine can produce a > sdist that can be uploaded to PyPI or similar. This seems like a distinct operation from 'prepare the files needed to build a wheel', although they are related. For

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Donald Stufft
The hook is also so a tool like tox or TravisCI or twine can produce a sdist that can be uploaded to PyPI or similar. Sent from my iPhone > On May 31, 2017, at 11:16 AM, Paul Moore wrote: > > Hmm. The proposed API (whether it's "create a sdist" or "tell me what > files

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Paul Moore
On 31 May 2017 at 15:05, Thomas Kluyver wrote: > On Wed, May 31, 2017, at 03:00 PM, Thomas Kluyver wrote: >> I would be fine with specifying a hook to copy only the files needed to >> build the wheel, but if we do that, let's not call it 'sdist' or >> anything that suggests

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Thomas Kluyver
On Wed, May 31, 2017, at 03:00 PM, Thomas Kluyver wrote: > I would be fine with specifying a hook to copy only the files needed to > build the wheel, but if we do that, let's not call it 'sdist' or > anything that suggests that. Also, if this is all we're after, I'd like to push again for making

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Thomas Kluyver
On Wed, May 31, 2017, at 02:57 PM, Paul Moore wrote: > My feeling is that in the current context we're talking about "minimal > set of files needed to build the wheel". That's not precisely a sdist > (because it ignores additional files that the user might want to > include like README, COPYRIGHT,

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Paul Moore
On 31 May 2017 at 14:39, Thomas Kluyver wrote: > On Wed, May 31, 2017, at 02:24 PM, Nick Coghlan wrote: >> Cool, thanks for doing that - I'll let you thrash out the details with >> Thomas as the PEP author, and then take a look once you're in >> agreement on the amendments

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Leonardo Rochael Almeida
Hi, On 29 May 2017 at 04:05, Nathaniel Smith wrote: > On Sun, May 28, 2017 at 10:37 PM, Donald Stufft wrote: > > > > [...] > > > > 1. Using the {package-name}-{package-version}.dist-info in the > > get_wheel_metadata() metadata is a mistake I think. In pip

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Thomas Kluyver
On Wed, May 31, 2017, at 02:24 PM, Nick Coghlan wrote: > Cool, thanks for doing that - I'll let you thrash out the details with > Thomas as the PEP author, and then take a look once you're in > agreement on the amendments you want to make. I've had a quick look over the PR, and my main thought is

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Nick Coghlan
On 31 May 2017 at 21:56, Donald Stufft wrote: > I don’t think an *optional* interface is the correct way to implement this. > I’m not entirely sure why folks seem to be ignoring the fact that this is > not just for pip, but well, this isn’t just for pip. There are a wide range >

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Donald Stufft
> On May 31, 2017, at 4:17 AM, Nick Coghlan wrote: > > On 30 May 2017 at 21:37, Donald Stufft wrote: >> I don’t think there is any value in decoupling the generation of what goes >> into an sdist from the tool that builds them. If we did that, I suspect

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Nick Coghlan
On 31 May 2017 at 04:17, Brett Cannon wrote: > Just to make sure I'm following this correctly, Donald is asking for: > > A way for pip to ask back-ends what files should be in an sdist from a > source checkout or to make an actual sdist > > Because sdists are a thing and so we

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Nick Coghlan
On 30 May 2017 at 21:37, Donald Stufft wrote: > I don’t think there is any value in decoupling the generation of what goes > into an sdist from the tool that builds them. If we did that, I suspect that > 100% of the time the exact same tool is going to be used to handle both >

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-30 Thread Brett Cannon
On Tue, 30 May 2017 at 11:40 Donald Stufft wrote: > > On May 30, 2017, at 2:17 PM, Brett Cannon wrote: > > Just to make sure I'm following this correctly, Donald is asking for: > >- A way for pip to ask back-ends what files should be in an sdist from >

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-30 Thread Donald Stufft
> On May 30, 2017, at 2:17 PM, Brett Cannon wrote: > > Just to make sure I'm following this correctly, Donald is asking for: > A way for pip to ask back-ends what files should be in an sdist from a source > checkout or to make an actual sdist > Because sdists are a thing and

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-30 Thread Paul Moore
On 30 May 2017 at 19:17, Brett Cannon wrote: > Just to make sure I'm following this correctly, Donald is asking for: > > A way for pip to ask back-ends what files should be in an sdist from a > source checkout or to make an actual sdist > > Because sdists are a thing and so we

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-30 Thread Paul Moore
On 30 May 2017 at 17:17, Donald Stufft wrote: > This is a long standing issue with pip that people hit with semi regularity— > refusing to fix it is user hostile. Personally I don’t really have much > interest in seeing something land in pip that prevents fixing issues that >

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-30 Thread Brett Cannon
Just to make sure I'm following this correctly, Donald is asking for: - A way for pip to ask back-ends what files should be in an sdist from a source checkout or to make an actual sdist - Because sdists are a thing and so we should support them properly - To make it so building

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-30 Thread Donald Stufft
> On May 30, 2017, at 12:29 PM, Thomas Kluyver wrote: > > What about saying that the copying step, if necessary, is part of the build > backend's responsibilities? I.e. pip doesn't copy the whole directory to a > temporary build location, but the build backend may decide

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-30 Thread Thomas Kluyver
What about saying that the copying step, if necessary, is part of the build backend's responsibilities? I.e. pip doesn't copy the whole directory to a temporary build location, but the build backend may decide to do that at its discretion when it's asked to build a wheel. pip would continue to

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-30 Thread Donald Stufft
> On May 30, 2017, at 11:36 AM, Thomas Kluyver wrote: > > I'm struggling to understand the objections here. As I understand the PEP, > the input to building a wheel is a source tree. That may come from an > unpacked sdist or a VCS checkout; hopefully those contain

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-30 Thread Thomas Kluyver
I'm struggling to understand the objections here. As I understand the PEP, the input to building a wheel is a source tree. That may come from an unpacked sdist or a VCS checkout; hopefully those contain basically the same files, give or take some unimportant generated files in an sdist. This seems

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-30 Thread Donald Stufft
> On May 30, 2017, at 7:37 AM, Donald Stufft wrote: > >> >> >> Post a to-be-written sdist-source-tree-export PEP: >> >> - pip (et al) can use setup.py to build from an unfiltered source tree >> - pip (et al) can use setup.py to build from an sdist >> - pip (et al) can use

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-30 Thread Donald Stufft
> On May 30, 2017, at 6:34 AM, Nick Coghlan wrote: > > Note that I'm also fine with pip as a project saying that it will only > ship support for the build-backend interface once the source filtering > interface is also defined and implemented. As an addendum, this seems

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-30 Thread Donald Stufft
> On May 30, 2017, at 6:34 AM, Nick Coghlan wrote: > > On 30 May 2017 at 17:07, Donald Stufft wrote: >> It can require that it is either unfiltered or an unpacked sdist since that >> is how a lot of build time projects treat it now. They handle a sdist

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-30 Thread Nick Coghlan
On 30 May 2017 at 17:07, Donald Stufft wrote: > It can require that it is either unfiltered or an unpacked sdist since that > is how a lot of build time projects treat it now. They handle a sdist > differently from a vcs source, for example pbr. Swapping out a call to >

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-30 Thread Donald Stufft
It can require that it is either unfiltered or an unpacked sdist since that is how a lot of build time projects treat it now. They handle a sdist differently from a vcs source, for example pbr. Swapping out a call to setup.py for an internal shim that calls a Python API doesn't change anything

  1   2   >