At 06:58 PM 10/15/2006 +0200, Martin v. Löwis wrote: >Phillip J. Eby schrieb: > > Last, but not least, the distutils suffer from an excess of policy > > flexibility in areas where flexibility is undesirable from a community > > perspective. There should be "one obvious way" to do a lot of things > > that the distutils provide a ridiculous number of ways to do (e.g. > > insane project directory layouts including package-by-package directory > > mapping). This is not something that affects the distutils themselves > > much, but it essentially prevents the distutils from improving in > > certain ways because there are so few assumptions one can make about its > > use. > >While each of these points might be true, I also believe they can >be fixed. For example, it would be possible to deprecate-then-remove >a non-standard package mapping. I'm not sure how many packages actually >use such a mapping,
They are not common in published packages (although PIL is a notable exception). But they often are in use in unpublished packages, judging by distutils-sig threads inquiring about the use of more esoteric directory layouts with setuptools. Setuptools tries hard to support arbitrary package mappings, but there are some things that just won't work, like making the in-development source code available on sys.path. > >> Actually, it is possible to add a new compiler to some Python > >> installation, so that users can pass it to the --compiler option, > >> and distutils packages don't have to be modified for that. > >> You just have to extend the dictionary > >> distutils.ccompiler.compiler_class. > > > > Yes, and how do you accomplish *that*? You have to either modify some > > Python code (e.g. sitecustomize) or use a big -c to python when running > > the setup script. > >There is a third option: write a .pth file, and put that on the path. Which means you are now importing the distutils whenever Python is started, just as in the sitecustomize case. > > With entry point extension, it is only necessary to have an extension > > installed on sys.path; it is not necessary to run some code to do the > > registration. > >One thing that bothers me about setuptools is the strange, non-intuitive >terminology. What is "entry point extension"? If you'd call it "ini >files listing command names", I could understand much better. When I name a new thing, I give it a new name. If I called it something you merely *thought* you understood, I would have a much harder time correcting any misunderstanding. It is intended that you ask "what is that?", because then there is some chance you will listen to my answer, or that you will investigate to find the answer yourself. :) Terminology is also a group/social engineering tool, as well as an educational engineering tool. In the same way that such documents as the Zen of Python create a community vocabulary of concepts such as EIBTI, so too do setuptools terms create a community vocabulary by which its users can recognize each other's level of understanding. You can observe this in blog discussions, hallway conversations at PyCon, etc., where the idea that "entry points are cool" is like a little secret handshake among the informed, and the opportunity to *explain* them to outsiders provides an opportunity for social reward. A lot of people leave these kinds of things to chance; I consider them as much a part of the design of a total system, as anything in the actual code. (See also WSGI "middleware" for a similar social phenomenon, and Kathy Sierra's "Creating Passionate Users" blog for general discussion of the concepts involved in creating a passionate user community.) Certainly it is true that there are people who do not care about these things or see them as counterproductive. However, if I am doing "community engineering", I do not allow this to get in the way, as those people are simply not a "compatible platform" for the socialware I'm creating. There are enough people who *will* run socialware that there is always a viable and even vital community for it. > > Adding setuptools-style plugin ability to the distutils is very simple, > > and doesn't require a wholesale reorganization. After all, setuptools > > manages to do it! > >Ok, if that's the most desired property, I think it should be added >(provided somebody is willing to contribute, of course); one doesn't >have to wait for 3.0 to add it. Of course, and I've already said this could be done in the 2.x series. It doesn't even need the whole of setuptools to be incorporated into the stdlib; the pkg_resources module suffices. (Of course, there would need to be additions to support building and installing the entry point data.) > > However, there are areas of build functionality that > > the distutils does not cover or provides incomplete coverage for, such > > as configuration and build of more complex C libraries and programs, > > shared libraries, documentation production, and so on, that the existing > > architecture of the distutils doesn't really encompass. > >I agree it doesn't cover it. I don't see why it isn't "easy" to add >it (where "easy" really means "as easy doing it stand-alone"). If you look at how much distutils infrastructure ends up being reinvented in specific distutils extension, because the original wasn't customizable in a useful way, you would see that there is in many respects very little difference between adding something to the distutils, and "doing it standalone". Distutils' lower layers encode a bit too many assumptions from its upper layers, and it's rarely customizable in the places you need it to be. But even if you have the ability to change the distutils itself, you may be stuck because changing these assumptions means breaking the existing distutils, but not having any decent way to *discover* that you have done so via tests. Having done massive extensions to the distutils, I think I'm qualified to say that, if I had the choice, I would've built something that was merely API-compatible with the distutils, borrowing code and ideas from the distutils, rather than layering on top of it. Of course, I didn't have that choice, because one of my design goals was to "embrace and extend" by supporting automated installation of a large number of existing packages. However, with setuptools' current marketshare, an API-compatible version of setuptools that wasn't based on the current distutils architecture could likely be quite successful. > > Meanwhile, the higher-level layer is based on the idea of dependency > > between sequentially invoked commands, rather than dependency between > > things being built. This makes it difficult to deal with concepts like > > having a bunch of reST release note docs combined with a LaTeX > > documentation build. > >This I don't understand. Couldn't there be a subcommand of "build" that >creates the documentation? Of course, that command would have to do >the time-stamp checking, and use whatever build machinery is appropriate >(make, ant, scons, ...). What I'm saying is that the reason there is no build_docs command is because documentation build processes aren't sufficiently uniform to fit within the distutils' simple command-oriented framework. Most use of distutils extensions "in the wild" are trying to solve issues related to this. For example, Twisted tries to have C extensions that are optionally built, based on whether a compiler is available and what libraries are available. This is another scenario that doesn't really work in the distutils' concept framework. > > So, distutils' extensibility *could* be improved dramatically in the 2.x > > line by adding setuptools-style plugin entry points. However, adding > > any *new* build functionality to the distutils is probably going to be > > better off being based on something that offers an extensible > > object-dependency layer, ala zc.buildout. > >I don't mind adding a time-stamp-or-content-based dependency >infrastructure to the standard library, either, and it might be useful >to reimplement the existing build commands on top of that >(for example, it bothers me that the standard setup.py doesn't take > header file dependencies into account, for the modules of the standard > library). >That's still no reason to drop distutils. I'm not sure what you mean. Do you mean that there should be a set of modules *called* distutils? I agree with that, and that it should offer something closely resembling the existing API, so that a large subset of existing packages could continue to work. But do you also mean that those "distutils" modules must be created only by modifying the existing ones? In that case I disagree. I personally don't believe that the distutils code can be rehabilitated without clean-rooming it for testability. The lifecycle for effective testing against the *enormous* variations in how it is used, means that to make any substantial change you must rule out backward compatibility except for a strictly defined feature set. And if you do that, there is nothing stopping you from simply implementing a testable and less-fragile architecture from the beginning, and then putting a limited backward-compatibility layer on top of that. The result would be far less fragile than building new floors on a shaky foundation, as setuptools has tried to do. > > (Which reminds me. Does anybody have any idea why the distutils compile > > modules in build/? There seems to be no point to this, since any > > installation of the modules is going to need to recompile them. Even > > bdist operations end up recompiling them in a different location!) > >It was added to build_py.py with this checkin: > >r17820 | gward | 2000-10-02 04:19:04 +0200 (Mon, 02 Oct 2000) | 8 lines > >Added the ability to do byte-compilation at build time, currently off >by default (since compiling at install time works just fine). Details: > - added 'compile' and 'optimize' options > - added 'byte_compile()' method > - changed 'get_outputs()' so it includes bytecode files >A lot of the code added is very similar to code in install_lib.py; >would be nice to factor it out further. > >Notice that it is meant to be off by default, and, indeed, for me, >it is off. Not sure why you are getting .pyc files in your >build directory. That's not what I said. I asked why it even has this capability at all. The checkin comment seems to agree that it is useless, "since compiling at install time works just fine". :) _______________________________________________ Python-3000 mailing list [email protected] http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
