Re: [Distutils] PEP for specifying build dependencies

[Donald Stufft]

> On May 11, 2016, at 9:45 PM, Nathaniel Smith  wrote:
> 
> On May 11, 2016 6:33 PM, "Donald Stufft"  > wrote:
> >
> [...]
> >
> > I don't like any of these options nearly as much as [package] TBH. I don’t
> > think that base, super, common, standard, or shared are any less ambiguous 
> > than
> > package (in fact I think they are _more_ ambigious).
> >
> >
> > I don't really think of it as package vs tool, I think of it as an implicit
> >  vs an explicit . I think it makes the 
> > file
> > uglier to have the  explicit, particularly since I think the
> > example should really be something like:
> >
> > [standard.package.build-system]
> > requires = ["setuptools", "wheel"]
> >
> > [tool.flake8]
> > ...
> >
> > Because the value of the [package] namespace isn't that it separates us from
> > the [tool] namespace (we could get that easily without it), but that it
> > separates us from *other*, non packaging related but "standard" stuff that
> > might be added in the future.
> 
> Can you give an example of something that would go in your hypothetical 
> implicit a pyproject.tml [standard] section, but that would not be related to 
> configuring that project's package/packages and thus go into [package]? 
> Partly asking because I'm not sure what the difference is between a "project" 
> and a "package", partly because if we can articulate a clear guideline then 
> that'd be useful for the future.
> 
> -n
> 


This is somewhat of a contrived example because I’m not sure how useful it 
would be to have a standard representation of it, but one possible example is 
PEP8 (the actual PEP not the tool on PyPI) linters and what rules they follow 
that would allow people to use arbitrary linters against a code base (which may 
or may not be a “package” in the PyPI/pip/PyPA sense) but is just a chunk of 
code sitting in a directory.

A less contrived answer is that I simply don’t know, but it feels like the 
“cost” of introducing the [package] top level is pretty low (a total of 8 
additional characters per table) and in my head it has some meaning (this is 
the stuff for a Python distributable package, that you could, but maybe don’t, 
ship on PyPI and install with pip). I view the “project” as a superset of that, 
where part of configuring a “project” may include configuring the package side 
of things (assuming it is even a package and it isn’t just some arbitrary code 
in a dir) but might also include other things.

On the other hand, I feel like `[standard]` or whatever isn’t really meaningful 
as anything other than “the stuff that isn’t in [tool]”, which makes me feel 
like adding it in is mostly a purity thing and the cost (a total of 9 extra 
characters per table) doesn’t seem worth it since any human will be able to 
trivially identify the set of things which are not in the tool namespace, and 
computers can also do that pretty easily (although slightly clumsily).

Now, you could argue that the [package] keyword is superfluous and in reality 
it’s highly unlikely that we ever get anything major that would ever sit as a 
sibling to it (besides tool) and thus it doesn’t make sense to pay the cost of 
those extra 8 characters when it is probably going to be the only non-tool 
value ever. Personally I think hedging our bets and leaving the door open for 
that possibility is a nice thing to do when the cost is so low. However, I 
don’t think it’d be unreasonable or silly to make the other trade off and just 
say that having it isn’t valuable and just stick [build-system] at the top 
level along with [tool.*] and say that if we ever come up with something that 
is not related to a package (in the PyPA sense) that it really won’t be that 
big of a deal to just have it live beside stuff like [build-system].

So I think we should either have:

[package.build-system]
requires = [“setuptools”, “wheel”]

[tool.flake8]
…

OR

[build-system]
requires = [“setuptools”, “wheel”]

[tool.flake8]
…

but I don’t think trying to make the parsed tree fit some “correct” hierarchy 
of data types when you consider the [tool] section (which really only exists to 
prevent collisions, otherwise we’d just let people stick [flake8] etc at the 
top level) is worth it.

-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP for specifying build dependencies

[Nathaniel Smith]

On May 11, 2016 6:33 PM, "Donald Stufft"  wrote:
>
[...]
>
> I don't like any of these options nearly as much as [package] TBH. I don’t
> think that base, super, common, standard, or shared are any less
ambiguous than
> package (in fact I think they are _more_ ambigious).
>
>
> I don't really think of it as package vs tool, I think of it as an
implicit
>  vs an explicit . I think it makes the
file
> uglier to have the  explicit, particularly since I think
the
> example should really be something like:
>
> [standard.package.build-system]
> requires = ["setuptools", "wheel"]
>
> [tool.flake8]
> ...
>
> Because the value of the [package] namespace isn't that it separates us
from
> the [tool] namespace (we could get that easily without it), but that it
> separates us from *other*, non packaging related but "standard" stuff that
> might be added in the future.

Can you give an example of something that would go in your hypothetical
implicit a pyproject.tml [standard] section, but that would not be related
to configuring that project's package/packages and thus go into [package]?
Partly asking because I'm not sure what the difference is between a
"project" and a "package", partly because if we can articulate a clear
guideline then that'd be useful for the future.

-n
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP for specifying build dependencies

[Donald Stufft]

> On May 11, 2016, at 9:08 PM, Brett Cannon  wrote:
> 
> 
> 
> On Wed, 11 May 2016 at 16:01 Nathaniel Smith  > wrote:
> On Wed, May 11, 2016 at 11:32 AM, Brett Cannon  > wrote:
> [...]
> > the file is for the project, not just the package(s) the project
> > contains ("package" is an overloaded term and I don't want to contribute to
> > that with the filename; I can live with the build details being in relation
> > to a package in the project and thus named [package], but other things that
> > may end up in this file might not relate to any package in the project).
> 
> We went back and forth on the overloaded "package" name a bit while
> drafting too, and eventually just gave up and went ahead because it's
> not that important.
> 
> To me this feels similar to situations I've encountered in the past,
> where I've spent a bunch of time debating between two things, and it
> turned out that the reason we couldn't agree was because both
> proposals were wrong and a third solution was much better :-).
> 
> I still don't think the [package] name part is worth arguing about
> much, but throwing this out there in case it turns out to be that
> "third way" that suddenly makes everyone go "a-ha!":
> 
> If you think about it, really the stuff in [package.build-system] is
> there to tell pip how to run the build system. It's associated with
> building the project/package, sure, but that's not what makes it
> special -- everything that goes in this file will be associated with
> building or developing the project/package: [tool.flit],
> [tool.coverage], [tool.pytest], [tool.tox], whatever. The build-system
> stuff could easily and comfortably have gone into
> [tool.pip.build-system] instead... *except* that we don't want it to
> be specific to pip, we want it to be a piece of shared/common
> configuration that's defined by a shared process (PEPs) and used by
> *multiple* tools. That's why it doesn't belong in [tool.pip].
> 
> Or another way to put it, contrasting [tool.*] versus [package.*] is
> kinda weird, because those categories aren't actually contradictory --
> it's like having categories [red] versus [square].
> 
> So, proposal: maybe we should rename the [package] namespace to
> something that reflects what distinguishes it from [tool], like:
> 
>   [standard.build-system]
> 
> or
> 
>   [common.build-system]
> 
> or
> 
>   [shared.build-system]
> 
> 
> or
> 
> [base.build-system]
> 
> or
> 
> [super.build-system]
> 
> I'm +1 on base, super, or common, +0 on standard, -0 on shared.
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org 
> 
> https://mail.python.org/mailman/listinfo/distutils-sig 
> 

I don't like any of these options nearly as much as [package] TBH. I don’t
think that base, super, common, standard, or shared are any less ambiguous than
package (in fact I think they are _more_ ambigious).


I don't really think of it as package vs tool, I think of it as an implicit
 vs an explicit . I think it makes the file
uglier to have the  explicit, particularly since I think the
example should really be something like:

[standard.package.build-system]
requires = ["setuptools", "wheel"]

[tool.flake8]
...

Because the value of the [package] namespace isn't that it separates us from
the [tool] namespace (we could get that easily without it), but that it
separates us from *other*, non packaging related but "standard" stuff that
might be added in the future.

The value of the [tool] namespace isn't that it doesn't make sense for a [flit]
and a [package] to be at the same level, but rather that we have no idea what
keys people might use there (and indeed, `package` is taken on PyPI) but that
it allows us to separate the wild west of "anything goes" from the strictly
defined the rest of the file.

IOW, the reason to omit the [standard] namespace and the reason to include the
[tool] namespace is practicality being purity and designing this file first for
humans to edit it, and only second for machines to access it and some sort of
semantic purity a distant third.

-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP for specifying build dependencies

[Brett Cannon]

On Wed, 11 May 2016 at 16:01 Nathaniel Smith  wrote:

> On Wed, May 11, 2016 at 11:32 AM, Brett Cannon  wrote:
> [...]
> > the file is for the project, not just the package(s) the project
> > contains ("package" is an overloaded term and I don't want to contribute
> to
> > that with the filename; I can live with the build details being in
> relation
> > to a package in the project and thus named [package], but other things
> that
> > may end up in this file might not relate to any package in the project).
>
> We went back and forth on the overloaded "package" name a bit while
> drafting too, and eventually just gave up and went ahead because it's
> not that important.
>
> To me this feels similar to situations I've encountered in the past,
> where I've spent a bunch of time debating between two things, and it
> turned out that the reason we couldn't agree was because both
> proposals were wrong and a third solution was much better :-).
>
> I still don't think the [package] name part is worth arguing about
> much, but throwing this out there in case it turns out to be that
> "third way" that suddenly makes everyone go "a-ha!":
>
> If you think about it, really the stuff in [package.build-system] is
> there to tell pip how to run the build system. It's associated with
> building the project/package, sure, but that's not what makes it
> special -- everything that goes in this file will be associated with
> building or developing the project/package: [tool.flit],
> [tool.coverage], [tool.pytest], [tool.tox], whatever. The build-system
> stuff could easily and comfortably have gone into
> [tool.pip.build-system] instead... *except* that we don't want it to
> be specific to pip, we want it to be a piece of shared/common
> configuration that's defined by a shared process (PEPs) and used by
> *multiple* tools. That's why it doesn't belong in [tool.pip].
>
> Or another way to put it, contrasting [tool.*] versus [package.*] is
> kinda weird, because those categories aren't actually contradictory --
> it's like having categories [red] versus [square].
>
> So, proposal: maybe we should rename the [package] namespace to
> something that reflects what distinguishes it from [tool], like:
>
>   [standard.build-system]
>
> or
>
>   [common.build-system]
>
> or
>
>   [shared.build-system]
>
>
or

[base.build-system]

or

[super.build-system]

I'm +1 on base, super, or common, +0 on standard, -0 on shared.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP for specifying build dependencies

[Nathaniel Smith]

On Wed, May 11, 2016 at 11:32 AM, Brett Cannon  wrote:
[...]
> the file is for the project, not just the package(s) the project
> contains ("package" is an overloaded term and I don't want to contribute to
> that with the filename; I can live with the build details being in relation
> to a package in the project and thus named [package], but other things that
> may end up in this file might not relate to any package in the project).

We went back and forth on the overloaded "package" name a bit while
drafting too, and eventually just gave up and went ahead because it's
not that important.

To me this feels similar to situations I've encountered in the past,
where I've spent a bunch of time debating between two things, and it
turned out that the reason we couldn't agree was because both
proposals were wrong and a third solution was much better :-).

I still don't think the [package] name part is worth arguing about
much, but throwing this out there in case it turns out to be that
"third way" that suddenly makes everyone go "a-ha!":

If you think about it, really the stuff in [package.build-system] is
there to tell pip how to run the build system. It's associated with
building the project/package, sure, but that's not what makes it
special -- everything that goes in this file will be associated with
building or developing the project/package: [tool.flit],
[tool.coverage], [tool.pytest], [tool.tox], whatever. The build-system
stuff could easily and comfortably have gone into
[tool.pip.build-system] instead... *except* that we don't want it to
be specific to pip, we want it to be a piece of shared/common
configuration that's defined by a shared process (PEPs) and used by
*multiple* tools. That's why it doesn't belong in [tool.pip].

Or another way to put it, contrasting [tool.*] versus [package.*] is
kinda weird, because those categories aren't actually contradictory --
it's like having categories [red] versus [square].

So, proposal: maybe we should rename the [package] namespace to
something that reflects what distinguishes it from [tool], like:

  [standard.build-system]

or

  [common.build-system]

or

  [shared.build-system]

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP for specifying build dependencies

[Randy Syring]

 On 05/11/2016 02:32 PM, Brett Cannon wrote:
I also didn't rename the file as Randy argued for because the file is 
for the project, not just the package(s) the project contains 
("package" is an overloaded term and I don't want to contribute to 
that with the filename; I can live with the build details being in 
relation to a package in the project and thus named [package], but 
other things that may end up in this file might not relate to any 
package in the project).


This makes sense, thanks for your consideration.

*Randy Syring*
Husband | Father | Redeemed Sinner

/"For what does it profit a man to gain the whole world
and forfeit his soul?" (Mark 8:36 ESV)/
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP for specifying build dependencies

[Robert Collins]

On 12 May 2016 at 08:46, Donald Stufft  wrote:
>
>> On May 11, 2016, at 4:27 PM, Robert Collins  
>> wrote:
>>
>>> probably too late or out of scope for this pep. However the distro packaging
>>> for python packages recommends to run the testsuite during the package
>>> build. Would it be possible to extend this pep to test depends, or maybe
>>> track this as a separate pep?
>>
>> It's covered as well- this is general purpose 'what is needed to run
>> setup.py' - once you can run setup.py, you can machine interrogate any
>> further dependencies.
>
>
> It’s not *exactly* covered— I mean people could stick test dependencies in
> this new field but I don’t think that setuptools actually exposes the
> test_requires in any meaningful way (nor do I think people actually use
> setuptools test support that consistently). So setuptools could get better
> support for testing dependencies independently of this PEP *or* another PEP
> could add a similar section that layered ontop of this. It’s definitely out
> of scope for this particular PEP though.

Right - in more detail though:
tox using projects:
 - set it to setuptools, wheel and any other 'needed to run setup.py
sdist' things. Then tox will work - and its list is generally plain
text that packagers can introspect, install separately and run the tox
commands directly.

setup.py test using projects:
 -  a small setuptools entrypoint can be written to allow
introspecting test_requires, so we just need to set the requires as
for the tox case

etc.

-Rob


-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP for specifying build dependencies

[Donald Stufft]

> On May 11, 2016, at 4:27 PM, Robert Collins  wrote:
> 
>> probably too late or out of scope for this pep. However the distro packaging
>> for python packages recommends to run the testsuite during the package
>> build. Would it be possible to extend this pep to test depends, or maybe
>> track this as a separate pep?
> 
> It's covered as well- this is general purpose 'what is needed to run
> setup.py' - once you can run setup.py, you can machine interrogate any
> further dependencies.


It’s not *exactly* covered— I mean people could stick test dependencies in
this new field but I don’t think that setuptools actually exposes the
test_requires in any meaningful way (nor do I think people actually use
setuptools test support that consistently). So setuptools could get better
support for testing dependencies independently of this PEP *or* another PEP
could add a similar section that layered ontop of this. It’s definitely out
of scope for this particular PEP though.

-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP for specifying build dependencies

[Robert Collins]

On 12 May 2016 at 08:22, Matthias Klose  wrote:
> On 11.05.2016 02:39, Brett Cannon wrote:
>>
>> Donald, Nathaniel, and I have finished our proposed PEP for specifying a
>> projects' build dependencies. The PEP is being kept at
>> https://github.com/brettcannon/build-deps-pep, so if you find spelling
>> mistakes and grammatical errors please feel free to send a PR to fix them.
>
>
> probably too late or out of scope for this pep. However the distro packaging
> for python packages recommends to run the testsuite during the package
> build. Would it be possible to extend this pep to test depends, or maybe
> track this as a separate pep?

It's covered as well- this is general purpose 'what is needed to run
setup.py' - once you can run setup.py, you can machine interrogate any
further dependencies.

-Rob

-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP for specifying build dependencies

[Matthias Klose]

On 11.05.2016 02:39, Brett Cannon wrote:

Donald, Nathaniel, and I have finished our proposed PEP for specifying a
projects' build dependencies. The PEP is being kept at
https://github.com/brettcannon/build-deps-pep, so if you find spelling
mistakes and grammatical errors please feel free to send a PR to fix them.


probably too late or out of scope for this pep. However the distro packaging for 
python packages recommends to run the testsuite during the package build. Would 
it be possible to extend this pep to test depends, or maybe track this as a 
separate pep?


Matthias

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP for specifying build dependencies

[Robert Collins]

On 12 May 2016 at 06:32, Brett Cannon  wrote:
> Since you can easily read the PEP at
> https://github.com/brettcannon/build-deps-pep in a nice, rendered format I
> won't repost the whole thing, but here are the changes I have made based on
> the comments so far.
>
> 1. I rephrased throughout the PEP to not explicitly say this is for building
> wheels (for Robert). I was just trying to tie the PEP into what happens
> today, not what we plan to have happen in the future.
>
> 2. Fixed the quotation marks on the TOML examples (for Nathaniel). It's just
> habit from Python why I did it the way I did.
>
> 3. Reworked the Abstract to the following (for Robert):

Thanks Brett.

-Rob


-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP for specifying build dependencies

[Antoine Pitrou]

On Wed, 11 May 2016 18:32:30 +
Brett Cannon  wrote:
> >
> > The only open issue in the PEP at the moment is the bikeshedding topic of
> > what to name the sub-section containing the requirements: `[package.build]`
> > or `[package.build-system]` (we couldn't reach consensus among the three of
> > us on this). Otherwise the three of us are rather happy with the way the
> > PEP has turned out and look forward to this being the first step towards
> > allowing projects to customize their build process better!
> >
> 
> So far the votes for this open issue are:
> 
> build-system:
> 
>1. Nathaniel
>2. Ian
>3. Ethan
>4. Nick
>5. Paul
> 
> build:
> 
>1. Donald
>2. Daniel
> 
> build-configuration (write-in candidate):
> 
>1. Antoine

Ha :) I like "build" actually.

Regards

Antoine.


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP for specifying build dependencies

[Brett Cannon]

Since you can easily read the PEP at
https://github.com/brettcannon/build-deps-pep in a nice, rendered format I
won't repost the whole thing, but here are the changes I have made based on
the comments so far.

1. I rephrased throughout the PEP to not explicitly say this is for
building wheels (for Robert). I was just trying to tie the PEP into what
happens today, not what we plan to have happen in the future.

2. Fixed the quotation marks on the TOML examples (for Nathaniel). It's
just habit from Python why I did it the way I did.

3. Reworked the Abstract to the following (for Robert):
"""
This PEP specifies how Python software packages should specify what
dependencies they have in order to execute their chosen build system.
As part of this specification, a new configuration file is introduced
for software packages to use to specify their build dependencies (with
the expectation that the same configuration file will be used for
future configuration details).
"""

4. Added a bit to the end of the Rationale section about where this fits in
future plans (for Robert):
"""
To provide more context and motivation for this PEP, think of the
(rough) steps required to produce a built artifact for a project:

1. The source checkout of the project.
2. Installation of the build system.
3. Execute the build system.

This PEP covers step #2. It is fully expected that a future PEP will
cover step #3, including how to have the build system dynamically
specify more dependencies that the build system requires to perform
its job. The purpose of this PEP though, is to specify the minimal set
of requirements for the build system to simply begin execution.
"""

5. Added a JSON Schema for the resulting data (for Nick because he must be
writing a lot of specs at work recently if he needs typing information for
2 fields :). This is based on Nathaniel's initial work, but I made the
"requires" key a requirement when [package.build-system] is defined. I did
not change the name of the key to "schema-version" for the reasons
Nathaniel outlined in his response to Nick on the topic.

  {
  "$schema": "http://json-schema.org/schema#;,

  "type": "object",
  "additionalProperties": false,

  "properties": {
  "package": {
  "type": "object",
  "additionalProperties": false,

  "properties": {
  "semantics-version": {
  "type": "integer",
  "default": 1
  },
  "build-system": {
  "type": "object",
  "additionalProperties": false,

  "properties": {
  "requires": {
  "type": "array",
  "items": {
  "type": "string"
  }
  }
  },
  "required": ["requires"]
  }
  }
  },

  "tool": {
  "type": "object"
  }
  }
  }


I didn't add an example using 'distutils.numpy' as Nick asked because it's
just the same example as in the PEP but with one field changed; IOW it
doesn't really add anything. I also didn't rename the file as Randy argued
for because the file is for the project, not just the package(s) the
project contains ("package" is an overloaded term and I don't want to
contribute to that with the filename; I can live with the build details
being in relation to a package in the project and thus named [package], but
other things that may end up in this file might not relate to any package
in the project).


On Tue, 10 May 2016 at 17:39 Brett Cannon  wrote:

> Donald, Nathaniel, and I have finished our proposed PEP for specifying a
> projects' build dependencies. The PEP is being kept at
> https://github.com/brettcannon/build-deps-pep, so if you find spelling
> mistakes and grammatical errors please feel free to send a PR to fix them.
>
> The only open issue in the PEP at the moment is the bikeshedding topic of
> what to name the sub-section containing the requirements: `[package.build]`
> or `[package.build-system]` (we couldn't reach consensus among the three of
> us on this). Otherwise the three of us are rather happy with the way the
> PEP has turned out and look forward to this being the first step towards
> allowing projects to customize their build process better!
>

So far the votes for this open issue are:

build-system:

   1. Nathaniel
   2. Ian
   3. Ethan
   4. Nick
   5. Paul

build:

   1. Donald
   2. Daniel

build-configuration (write-in candidate):

   1. Antoine
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP for specifying build dependencies

[Daniel Holth]

Thanks for the suggested bikeshed topic. Clever!

I like build also. "package build requires". I see that build is what you
used in the example [rejected] file formats. To me further segmentation
between build systems and the actual build that the build system effects is
something you might do to avoid the cost of installing a dependency that
might not actually be needed during that phase. IMO installation of an
occasional extra package is a fair price to pay if it reduces the number of
concepts in the system.

On Wed, May 11, 2016 at 10:26 AM Antoine Pitrou  wrote:

> On Thu, 12 May 2016 00:20:32 +1000
> Nick Coghlan  wrote:
> >
> > When I say "build system configuration" in the context of
> > distutils/setuptools, I mean things like:
> >
> > * MANIFEST.in
> > * non-dependency related setup() arguments (packages, package_dir,
> > py_modules, ext_modules, namespace_packages, entry_points,
> > include_package_data, zip_safe, etc)
> > * the Extension class and its parameters:
> >
> https://docs.python.org/2/distutils/setupscript.html#describing-extension-modules
> >
> > Those are the settings that actually tell the build system what to
> > build and (in some cases) how to build it.
>
> That's confusing :-) You should really call it "build configuration".
>
> Regards
>
> Antoine.
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP for specifying build dependencies

[Antoine Pitrou]

On Thu, 12 May 2016 00:20:32 +1000
Nick Coghlan  wrote:
> 
> When I say "build system configuration" in the context of
> distutils/setuptools, I mean things like:
> 
> * MANIFEST.in
> * non-dependency related setup() arguments (packages, package_dir,
> py_modules, ext_modules, namespace_packages, entry_points,
> include_package_data, zip_safe, etc)
> * the Extension class and its parameters:
> https://docs.python.org/2/distutils/setupscript.html#describing-extension-modules
> 
> Those are the settings that actually tell the build system what to
> build and (in some cases) how to build it.

That's confusing :-) You should really call it "build configuration".

Regards

Antoine.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP for specifying build dependencies

[Nick Coghlan]

On 11 May 2016 at 19:27, Antoine Pitrou  wrote:
> On Wed, 11 May 2016 17:11:54 +1000
> Nick Coghlan  wrote:
>>
>> Take the default case: for a distutils/setuptools based project, the
>> actual build settings are the arguments to setup() in setup.py, *not*
>> these new settings in pyproject.toml. By contrast, the settings in
>> [package.build-system] are the ones that tell pip and other installers
>> what is needed to make "setup.py bdist_wheel" work (and, in the
>> future, will tell them when to invoke something other than "setup.py
>> bdist_wheel" to do the binary build)
>
> Side question: if the build system needs configuring, is a
> user-provided configuration file really the best place to do so?
> People will end up having to copy and paste a bunch of configuration
> directives that are not directly relevant to their own project (also
> those directives will need maintaining as a build tool may evolve
> its dependencies over time).

When I say "build system configuration" in the context of
distutils/setuptools, I mean things like:

* MANIFEST.in
* non-dependency related setup() arguments (packages, package_dir,
py_modules, ext_modules, namespace_packages, entry_points,
include_package_data, zip_safe, etc)
* the Extension class and its parameters:
https://docs.python.org/2/distutils/setupscript.html#describing-extension-modules

Those are the settings that actually tell the build system what to
build and (in some cases) how to build it.

> Alternative: have a single "build system" configuration directive:
>
>   [package.build-system]
>
>   tool = "foopackage:fooexe"
>
> ... which instructs the runner to install dependency "foopackage", and
> then invoking "fooexe" with a single well-known option (e.g.
> "--pybuild-bootstrap-config") produces a JSON file on stdout describing
> how to invoke precisely the tool for each build scenario (sdist, wheel,
> etc.).

Specifying alternate build systems likely won't be far behind this PEP
(since that's what PEPs 516 and 517 were about), but we decided it
made sense to split the two discussions (i.e. figuring out the static
configuration basics, then figuring out how to eliminate the need for
setup.py shims)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] comparison of configuration languages

[Donald Stufft]

> On May 11, 2016, at 9:46 AM, Nicholas Chammas  
> wrote:
> 
> On Tue, May 10, 2016 at 11:15 AM Donald Stufft don...@stufft.io 
>  wrote:
> 
> 
> 
> 
> > On May 10, 2016, at 11:00 AM, Antoine Pitrou  > > wrote:
> >
> > (as an aside, if there's the question of forking an existing parser
> > implementation for better vendorability, forking a YAML parser may be
> > more useful to third-party folks than forking a TOML parser :-))
> 
> 
> I’m seeing what I can come up with (https://github.com/dstufft/yaml 
> ) though
> I don’t know that I feel like actually maintaining whatever it is I end up
> figuring out there.
> 
> 
> Did you intend to fork from https://github.com/yaml/pyyaml 
> ? I believe the latest PyYAML is actually 
> hosted on BitBucket:
> 
> https://bitbucket.org/xi/pyyaml/ 
> 

Yea, there was only a very tiny, mostly meaningless delta between Github and 
bitbucket and I’m lazy.


> Also, a better starting point than PyYAML may be this existing fork of the 
> library, ruamel.yaml:
> 
> https://bitbucket.org/ruamel/yaml 
> The fork makes several  
> changes 
> 
>  which may be relevant to you.
> 
> 

To be clear, my thing was a proof of concept to see how hard it would be to 
help inform the decision about what file format to use. The PEP has decided 
that TOML best represents the tradeoffs we want to make so my lib will likely 
just die on my GitHub :)

-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] comparison of configuration languages

[Nicholas Chammas]

On Tue, May 10, 2016 at 11:15 AM Donald Stufft don...@stufft.io
 wrote:


> > On May 10, 2016, at 11:00 AM, Antoine Pitrou 
> wrote:
> >
> > (as an aside, if there's the question of forking an existing parser
> > implementation for better vendorability, forking a YAML parser may be
> > more useful to third-party folks than forking a TOML parser :-))
>
>
> I’m seeing what I can come up with (https://github.com/dstufft/yaml)
> though
> I don’t know that I feel like actually maintaining whatever it is I end up
> figuring out there.
>
Did you intend to fork from https://github.com/yaml/pyyaml? I believe the
latest PyYAML is actually hosted on BitBucket:

https://bitbucket.org/xi/pyyaml/

Also, a better starting point than PyYAML may be this existing fork of the
library, ruamel.yaml:

https://bitbucket.org/ruamel/yaml

The fork makes several 
changes

which may be relevant to you.

Nick
​
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP for specifying build dependencies

[Paul Moore]

On 11 May 2016 at 01:39, Brett Cannon  wrote:
> Donald, Nathaniel, and I have finished our proposed PEP for specifying a
> projects' build dependencies. The PEP is being kept at
> https://github.com/brettcannon/build-deps-pep, so if you find spelling
> mistakes and grammatical errors please feel free to send a PR to fix them.
>
> The only open issue in the PEP at the moment is the bikeshedding topic of
> what to name the sub-section containing the requirements: `[package.build]`
> or `[package.build-system]` (we couldn't reach consensus among the three of
> us on this). Otherwise the three of us are rather happy with the way the PEP
> has turned out and look forward to this being the first step towards
> allowing projects to customize their build process better!

+1 on the PEP as a whole - good work, all of you!

On the requirements sub-section, I have a mild preference for
[package.build-system] (but a stronger preference for not
bikeshedding, so I'm OK with either :-))

One thought, I understand that many projects assume they can import
particular Python modules in setup.py (numpy is a common one AIUI, for
getting the C header location). Would it be worth specifically calling
that out as a legitimate usage (it's not just to define the tools you
need to do the build, but also to specify the build environment in
general) and giving an example?

Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP for specifying build dependencies

[Nathaniel Smith]

On Wed, May 11, 2016 at 12:11 AM, Nick Coghlan  wrote:
> On 11 May 2016 at 10:39, Brett Cannon  wrote:
[...]
>> For the vast majority of Python projects that rely upon setuptools,
>> the ``pyproject.toml`` file will be::
>>
>>   [package.build-system]
>>   requires = ['setuptools', 'wheel']  # PEP 508 specifications.
>
> It would be worthwhile showing an example of using the new mechanism
> to bootstrap a project that relies on numpy.distutils.

It's just that with "numpy" added, but sure.

@Brett: I also just noticed reading the example above that you're
using single-quotes for strings in the TOML instead of double-quote
strings, which is a bit odd -- single quote strings in TOML are the
same as raw strings in Python, which does work for this case but
probably isn't the example we want to set.
>> Rejected Ideas
>> ==
>>
>> Other semantic version key names
>> 
>>
>> Names other than ``semantics-version`` were considered to represent
>> the version of semantics that the configuration file was written for.
>> Both ``configuration-version`` and ``metadata-version`` were both
>> considered, but were rejected due to how people may confuse the
>> key as representing a version of the files contents instead of the
>> version of semantics that the file is interpreted under.
>
> Would you be open to using schema-version rather than
> semantic-version, and then formally defining the format via jsonschema
> and/or JSL [1]?

I kinda like the semantics-version name (schema = structure, semantics
= structure + interpretation), and I'm not sure what the name of that
key has to do with defining a json schema, but anyway, here's a
first-pass json schema :-)

https://gist.github.com/njsmith/89021cd9ef1a6724579229de164d02d2

(NOTE that that schema's written to check that a file matches the
currently defined spec, and should NOT be used to validate real
pyproject.toml files, because the additionalProperties: false keys
will cause it to error out on future backwards-compatible changes.)

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP for specifying build dependencies

[Nick Coghlan]

On 11 May 2016 at 10:39, Brett Cannon  wrote:
> Donald, Nathaniel, and I have finished our proposed PEP for specifying a
> projects' build dependencies. The PEP is being kept at
> https://github.com/brettcannon/build-deps-pep, so if you find spelling
> mistakes and grammatical errors please feel free to send a PR to fix them.
>
> The only open issue in the PEP at the moment is the bikeshedding topic of
> what to name the sub-section containing the requirements: `[package.build]`
> or `[package.build-system]` (we couldn't reach consensus among the three of
> us on this). Otherwise the three of us are rather happy with the way the PEP
> has turned out and look forward to this being the first step towards
> allowing projects to customize their build process better!

I prefer [package.build-system]. My rationale for that is that the
build system may have *its own* configuration mechanism, separate from
this file, and I don't want people to get confused between a project's
"build settings" and its "build system identification".

Take the default case: for a distutils/setuptools based project, the
actual build settings are the arguments to setup() in setup.py, *not*
these new settings in pyproject.toml. By contrast, the settings in
[package.build-system] are the ones that tell pip and other installers
what is needed to make "setup.py bdist_wheel" work (and, in the
future, will tell them when to invoke something other than "setup.py
bdist_wheel" to do the binary build)


> For the vast majority of Python projects that rely upon setuptools,
> the ``pyproject.toml`` file will be::
>
>   [package.build-system]
>   requires = ['setuptools', 'wheel']  # PEP 508 specifications.

It would be worthwhile showing an example of using the new mechanism
to bootstrap a project that relies on numpy.distutils.

> Open Issues
> ===
>
> Name of the build-related sub-table
> ---
>
> The authors of this PEP couldn't decide between the names
> ``[package.build]`` and ``[package.build-system]``, and so it is an
> open issue over which one to go with.

"package.build-system", for the reason given above (i.e. "build"
conflicts with the project's actual build settings).

> Rejected Ideas
> ==
>
> Other semantic version key names
> 
>
> Names other than ``semantics-version`` were considered to represent
> the version of semantics that the configuration file was written for.
> Both ``configuration-version`` and ``metadata-version`` were both
> considered, but were rejected due to how people may confuse the
> key as representing a version of the files contents instead of the
> version of semantics that the file is interpreted under.

Would you be open to using schema-version rather than
semantic-version, and then formally defining the format via jsonschema
and/or JSL [1]?

Cheers,
Nick.

[1] The latter is like an ORM for jsonschema: https://pypi.python.org/pypi/jsl

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig