Re: [Distutils] comparison of configuration languages

[Ionel Cristian Mărieș]

On Fri, May 13, 2016 at 9:22 PM, Brett Cannon  wrote:

> No need to think; the decision is made and it's TOML. I know Chris doesn't
> mean to stir up trouble, but at this point if someone wants to propose
> something other than TOML they are going to have to write their own PEP.


​Not asking for any change but has anyone looked at libconfig
? ​It looks
quite interesting: simple grammar and nesting support. What do you think of
it?



Thanks,
-- Ionel Cristian Mărieș, http://blog.ionelmc.ro
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] comparison of configuration languages

[Lele Gaifax]

Chris Barker  writes:

> Oh, and why not "JSON with comments and trailing commas" - it would be well
> defined and easy to implement.

And mostly done, even: https://bitbucket.org/intellimath/pyaxon

ciao, lele.
-- 
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
l...@metapensiero.it  | -- Fortunato Depero, 1929.

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] build system requirements PEP, 3rd draft

[Brett Cannon]

Biggest changes since the initial draft:

   1. No more semantics-version
   2. No more [package] table
   3. Settled on [build-system] as the table name
   4. The "requires" key is required if [build-system] is defined
   5. Changed the title and clarified that this is all about the minimum
   requirements for the build system to execute (some added support for things
   like dynamic dependencies for producing build artifacts is for another PEP)
   6. Added a JSON Schema for the resulting data from the table because
   Nick likes his specs :)

--
PEP: NNN
Title: Specifying Minimum Build System Requirements for Python Projects
Version: $Revision$
Last-Modified: $Date$
Author: Brett Cannon ,
Nathaniel Smith ,
Donald Stufft 
BDFL-Delegate: Nick Coghlan
Discussions-To: distutils-sig 
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 10-May-2016
Post-History: 10-May-2016,
  11-May-2016,
  13-May-2016


Abstract


This PEP specifies how Python software packages should specify what
dependencies they have in order to execute their chosen build system.
As part of this specification, a new configuration file is introduced
for software packages to use to specify their build dependencies (with
the expectation that the same configuration file will be used for
future configuration details).


Rationale
=

When Python first developed its tooling for building distributions of
software for projects, distutils [#distutils]_ was the chosen
solution. As time went on, setuptools [#setuptools]_ gained popularity
to add some features on top of distutils. Both used the concept of a
``setup.py`` file that project maintainers executed to build
distributions of their software (as well as users to install said
distribution).

Using an executable file to specify build requirements under distutils
isn't an issue as distutils is part of Python's standard library.
Having the build tool as part of Python means that a ``setup.py`` has
no external dependency that a project maintainer needs to worry about
to build a distribution of their project. There was no need to specify
any dependency information as the only dependency is Python.

But when a project chooses to use setuptools, the use of an executable
file like ``setup.py`` becomes an issue. You can't execute a
``setup.py`` file without knowing its dependencies, but currently
there is no standard way to know what those dependencies are in an
automated fashion without executing the ``setup.py`` file where that
information is stored. It's a catch-22 of a file not being runnable
without knowing its own contents which can't be known programmatically
unless you run the file.

Setuptools tried to solve this with a ``setup_requires`` argument to
its ``setup()`` function [#setup_args]_. This solution has a number
of issues, such as:

* No tooling (besides setuptools itself) can access this information
  without executing the ``setup.py``, but ``setup.py`` can't be
  executed without having these items installed.
* While setuptools itself will install anything listed in this, they
  won't be installed until *during* the execution of the ``setup()``
  function, which means that the only way to actually use anything
  added here is through increasingly complex machinations that delay
  the import and usage of these modules until later on in the
  execution of the ``setup()`` function.
* This cannot include ``setuptools`` itself nor can it include a
  replacement to ``setuptools``, which means that projects such as
  ``numpy.distutils`` are largely incapable of utilizing it and
  projects cannot take advantage of newer setuptools features until
  their users naturally upgrade the version of setuptools to a newer
  one.
* The items listed in ``setup_requires`` get implicily installed
  whenever you execute the ``setup.py`` but one of the common ways
  that the ``setup.py`` is executed is via another tool, such as
  ``pip``, who is already managing dependencies. This means that
  a command like ``pip install spam`` might end up having both
  pip and setuptools downloading and installing packages and end
  users needing to configure *both* tools (and for ``setuptools``
  without being in control of the invocation) to change settings
  like which repository it installs from. It also means that users
  need to be aware of the discovery rules for both tools, as one
  may support different package formats or determine the latest
  version differently.

This has cumulated in a situation where use of ``setup_requires``
is rare, where projects tend to either simply copy and paste snippets
between ``setup.py`` files or they eschew it all together in favor
of simply documenting elsewhere what they expect the user to have
manually installed prior to attempting to build or install their
project.

All of this has led pip [#pip]_ to simply assume that setuptools is
necessary when 

Re: [Distutils] PEP for specifying build dependencies

[Nick Coghlan]

On 14 May 2016 at 06:31, Chris Barker  wrote:
> On Fri, May 13, 2016 at 1:09 PM, Nathaniel Smith  wrote:
>>
>> But, the plan *is* to make wheels the standard way to build packages --
>> that will be in the next pep :-). I'm not sure I'd call it "lock down",
>> because there's nothing that will stop you running setup.py bdist_rpm or
>> whatever. But our goal is to reach the point where package authors get a
>> choice of what build system to use, and there's no guarantee that every
>> build system will implement bdist_rpm.
>
> hmm -- this really feels like mingling packaging and building.
>
> Does making sure everything builds a wheel help systems like rpm and the
> like? Honestly I have no idea.

Yes, it does. The reason is that separating the build system from the
deployment system is normal practice in those tools, but not the case
with approaches like "./setup.py install". Direct invocation of the
latter also loses semantic information about the package contents that
may be relevant for Filesystem Hierarchy Standard compliance purposes.

When the upstream installation process is instead broken up into
"build a binary artifact" and "install a binary artifact", that brings
a few benefits:

- the fact that the build system and the deployment target may be
different machines can be handled upstream
- we get a new metadata source (the binary artifact format) that can
be used in tools like pyp2rpm
- the semantic information about file types is captured in the binary artifact

Over time, what this means is that distros can move away from the
current practice of defining a packaging configuration once, and then
updating it manually when rebasing to a new upstream release, in
favour of automatically generating the downstream packaging config,
and submitting patches to the upstream project to improve the built
binary artifact as necessary (patching it locally in the meantime).
That's not something that's going to happen quickly, but you can see
the foundations of it being laid as people start trying to do things
like automatically rebuild all of PyPI as RPMs.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig