> On Jan 22, 2016, at 7:18 PM, Scott Kitterman <deb...@kitterman.com> wrote:
> The Zen of Python says, among other things, "There should be one-- and 
> preferably only one --obvious way to do it".  Build systems seem to me like a 
> great place to apply that.

We have a sliding scale of complexity in what a project needs from it’s build 
system that generally breaks down into a few major classes of projects:

1) Projects that are pure python, single source, whose only real “compilation” 
is shuffling files to the correct location.
2) Projects that are pure python, but require some sort of generation step as 
part of the build process (2to3, etc).
3) Projects that have some basic C code that needs to be compiled, but which 
doesn’t link against anything special besides Python itself.
4) Projects that have some basic C code that needs to be compiled, but which 
links against other libraries like OpenSSL, libpq, etc.
5) Projects that have some basic C code, that needs to be compiled, that links 
against other libraries, and needs to be able to conditionally link against 
different libraries based on the capabilities and what is available in the 
6) Extremely complex projects that need to link against many different 
libraries, possibly hard to build, possibly not C (e.g., Numpy with it’s blas 
libraries, Fortran, etc).

The problem with a one size fits all solution is that it’s very difficult to 
actually cover all of these cases in a way that is not horrible to actually use 
for each particular case. For an example, there is currently a build system 
called flit which doesn’t support anything but building straight to wheels 
(because we don’t have any sort of sdist 2.0 or anything yet). It doesn’t 
attempt to solve anything but the first class of users up there, and because of 
that it is able to create a very simple and easy to use packaging experience 
for authors, you just add a __version__ = “the version” to the top level of 
your package, and then drop in a flit.ini that looks something like:

    author=Sir Robin

    # If you want command line scripts, this is how to declare them.
    # If not, you can leave this section out completely.
    # foobar:main means the script will do: from foobar import main; main()

And that’s it. You’re done. From there flit can do the rest of the work for you 
because it didn’t need to concern itself with trying to work on anything 

In addition to the above types of problems, you also have other things like 
what the “source of truth” is for your metadata. A common thing that people 
want is to be able to not have to duplicate their version in multiple locations 
(sometimes that even extends to the tags in the version control), however it’s 
not currently possible to do that in a particularly easy way. You have systems 
like pbr, setuptools_scm, versioner, etc that all do it but which all rely on 
some level of terribleness to deal with the sort of weird inverted control flow 
we have.

Once you get to all of the possible options for things people reasonably want 
to do, you quickly end up in a place where the only reasonable solution is the 
full complexity of a programming language like what we have in setup.py. 
However, that has it’s own problems which we’ve discovered over two decades of 
doing that :) It tends to end up with people doing badly tested adhoc code that 
they cargo cult from project to project and when there’s a problem there ends 
up being very little understanding why some code did that since it had been 
copy/pasted around so much. Pip has to go to a lot of effort to try and work 
around common mistakes people make in their setup.py files (like, depending on 
it being invoked with the project root as CURDIR) which most people will just 
never notice their slightly broken setup.py.

So basically, by allowing multiple build systems we will enable a world where 
your bog standard pure python project can get an extremely simple tool/ux and 
projects with crazy hard requirements like Numpy can get something more 
complex, without the two groups of users fighting each other over simplicity + 
limited vs complexity + powerful.

In the end, I think it’s likely we’ll end up with 2-3 really popular build 
tools, one that is for the complex projects, one that is for the simpler 
projects with some basic C needs, and *maybe* one that is for pure python (but 
that may be able to be handled by the basic C needs one) though there will be a 
long tail I’m sure.

Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

Attachment: signature.asc
Description: Message signed with OpenPGP using GPGMail

Reply via email to