Tarek Ziadé a écrit : > Here's a quick summary of the main things that are going to happen in > Distutils, and Distribute, and a few words on virtualenv and pip. > (there is much much more work going on, but I don't want to drown > people with details) > > = Distutils = > > Distutils is a package manager and competes with OS package managers. > This is a good thing because, unless you are developing a library or > an application that will only run one specific system that has its own > packaging system like Debian, you will be able to reach much more > people. Of course the goal is to avoid making the work of a Debian > packager (or any other OS that has a package manager) too hard. In > other words, re-packaging a Distutils-based project should be easy and > Distutils should not get in their way (or as less as possible). > > But right now Distutils is incomplete in many ways and we are trying to > fix'em. > > == What's installed ? what's the installation format ? how to uninstall ? == > > First, it's an incomplete package manager : you can install a > distribution using it, but there's no way to list installed > distributions. Worst, you can't uninstall a distribution. > > PEP 376 resolves this, and once it's finished, the goal is to include > the APIs described there into Distutils itself and into the pkgutil > module in stdlib. Notice that there's an implementation at > http://bitbucket.org/tarek/pep376 that is kept up to date with PEP 376 > so people can see what we are talking about. > > Another problem that popped during the last years is the fact that, in > the same site-packages, depending on the tool that was used to install > a Distribution, and depending if this distribution uses Distutils or > Setuptools, you can have different installation formats. > > End-users end up with zipped eggs (one file), unzipped eggs (one > self-contained format in a directory) and regular Distutils (packages > and modules in site-packages). And the Metadata are also located in > many different places depending on the installation format used. > > That can't be. there's no point to keep various installation format in > the *same* site-packages directory. > > PEP 376 also resolves this by describing a *unique* format that works > for all. Once this is finished, Distutils will implement it by > changing the install command accordingly. > > - Work left to do in PEP 376 : restrict its scope to a disk-based, > file-based site-packages. > - Goal: 2.7 / 3.2 > > == Dependencies == > > The other feature that makes a packaging system nice is dependencies. > e.g. a way to list in a distribution, the distributions it requires to > run. As a matter of fact, PEP 314 has introduced in the Metadata new > fields for this purpose ("Requires", "Provides and "Obsoletes"). So, > you can write things like "Requires: lxml >= 2.2.1", meaning that your > distribution requires lxml 2.2.1 or a newer version to run. But this > was just description fields and Distutils was not providing any > feature based on these new fields. > > In fact, no third-party tool either provided a feature based on those > fields. Setuptools provided "easy_install" a script that looks for the > dependencies and install them, by querying the Python Package Index > (PyPI). But this feature was implemented with its own metadata: you > can add an "install_requires" option in the setup() call in setup.py, > and it will end up in a "requires.txt" file at installation time that > is located alongside the Metadata for you distribution. > > So the goal is to review PEP 314 and update the Metadata w.r.t. the > setuptools feedback and community usage. Once it's done, Distutils > will implement this new metadata version and promote its usage. > Promoting its usage means that Distutils will provide some APIs to > work with these APIs, like a version comparison algorithm. > > And while we're at it, we need to work out some inconsistency with the > "Author" and "Maintainer" fields. (The latter doesn't exists in the > Metadata but exists on setup.py side). > > - Work left to do in PEP 314 : finish PEP 386, finish the discussion > on the "maintainer" field. > - Goal: 2.7 / 3.2 > > == Version comparison == > > Once you provide dependency fields in the metadata, you need to > provide a version scheme: a way to compare two versions. Distutils has > two version comparison algorithms that are not used in its code and in > only one place in the stdlib where it could be removed with no pain. > One version scheme is "strict" and one is "loose". And Setuptools has > another one, which is more heuristic (it will deal with any version > string and compare it, wether it's wrong or not). > > PEP 386 goal is to describe a version scheme that can be used by all > and if we can meet a consensus there, we can move on and add > it as a reference in the update done in PEP 314, besides the > dependencies fields. Then, in Distutils we can deprecate the existing > version > comparison algorithms and provide a new one based on PEP 386 and > promote its usage. > > One very important point: we will not force the community to use the > scheme described in PEP 386, but *there is* already a > de-facto convention on version schemes at PyPI if you use Pip or > easy_install, so let's have a documented standard for this, > and a reference implementation in Distutils. > > There's an implementation at > http://bitbucket.org/tarek/distutilsversion that is kept up-to-date > with PEP 386. > > - Work left to do in PEP 386 : another round with the community > - Goal: 2.7 / 3.2 > > == The fate of setup.py, and static metadata == > > Setup.py is a CLI to create distribution, install them etc. You can > also use it to retrieve the metadata of a distribution. For > example you can call "python setup.py --name" and the name will be > displayed. That's fine. That's great for developers. > > But there's a major flaw: it's Python code. It's a problem because, > depending on the complexity of this file, an OS packager that just > wants to get the metadata for the platform he's working on, will run > arbitrary code that mught do unwanted things (or even that light not > work) > > So we are going to separate the metadata description from setup.py, in > a static configuration file, that can be open and read by anyone > without > running any code. The only problem with this is the fact that some > metadata fields might depend on the execution environment. For > instance, once "Requires" is re-defined and re-introduced via PEP 314, > we will have cases where "pywin32" will be a dependency to have only > on win32 systems. > > So we've worked on that lately in Distutils-SIG and came up with a > micro-language, based on a ConfigParser file, that allows > writing metadata fields that depends on sys.platform etc. I won't > detail the syntax here but the idea is that the interpretation > of this file can be done with a vanilla Python without running arbitrary code. > > In other words : we will be able to get the metadata for a > distribution without having to install it or to run any setup.py > command. > One use case is the ability to list all dependencies a distribution > requires for a given platform, just by querying PyPI. > > So I am adding this in Distutils for 2.7. > > Of course setup.py stays, and this is backward compatible. > > - Work left to do : publish the final syntax, and do the implementation > - Goal: 2.7 / 3.2 > > == The fate of bdist_* commands == > > During last Pycon summit we said that we would remove commands like > bdist_rpm because Python is unable, due to its release cycle, > to do a good work there. Here's an example: I have from time to time > cryptic issues in the issue tracker from people from Fedora (or any > rpm-based system), and I have all the pain in the world for these very > specific problems to do the proper fix unless some RPM expert helps > around. And by the time it's detected then fixed, it can be year(s) > before it's available on their side. That's why, depending on the > communities, commands like bdist_rpm are just totally ignored, and OS > packager have their own tools. > > So the best way to handle this is to ask these communities to build > their own tool and to encourage them to use Distutils as a basis for > that. > > This does not concern bdist_* commands for win32 because those are > very stable and don't change too much: Windows doesn't have a package > manager that would require these commands to evolve with it. > > Anyways, when we said that we would remove bdist_rpm, this was very > controversial because some people use it and love it. > > So what is going to happen is a status-quo: no bdist_* command will be > removed but no new bdist_* command wil be added. That's why I've > encouraged Andrew and Garry, that are working on a bdist_deb command, > to keep it in the "stdeb" project, and eventually we will > refer to it in the Distutils documentation if this bdist_deb comply > with Distutils standard. It doesn't right now because it uses a > custom version of the Distribution class (through Setuptools) that > doesn't behave like Distutils' one anymore. > > For Distutils, I'll add some documentation explaining this, and a > section that will list community-driven commands. > > - Work left to do : write the documentation > - Goal: 2.7 / 3.2 > > = Distribute = > > I won't explain here again why we have forked, I think it's obvious to > anyone here now. I'll rather explain what > we are planning in Distribute and how it will interact with Distutils. > > Distribute has two branches: > > - 0.6.x : provides a Setuptools-0.6c9 compatible version > - 0.7.x : will provide a refactoring > > == 0.6.x == > > Not "much" is going to happen here, we want this branch to be helpful > to the community *today* by addressing the 40-or-so bugs > that were found in Setuptools and never fixed. This is eventually > happen soon because its development is > fast : there are up to 5 commiters that are working on it very often > (and the number grows weekly.) > > The biggest issue with this branch is that it is providing the same > packages and modules setuptools does, and this > requires some bootstrapping work where we make sure once Distribute is > installed, all Distribution that requires Setuptools > will continue to work. This is done by faking the metadata of > Setuptools 0.6c9. That's the only way we found to do this. > > There's one major thing though: thanks to the work of Lennart, Alex, > Martin, this branch supports Python 3, > which is great to have to speed up Py3 adoption. > > The goal of the 0.6.x is to remove as much bugs as we can, and try if > possible to remove the patches done > on Distutils. We will support 0.6.x maintenance for years and we will > promote its usage everywhere instead of > Setuptools. > > Some new commands are added there, when they are helpful and don't > interact with the rest. I am thinking > about "upload_docs" that let you upload documentation to PyPI. The > goal is to move it to Distutils > at some point, if the documentation feature of PyPI stays and starts to be > used. > > == 0.7.x == > > We've started to refactor Distribute with this roadmap in mind (and > no, as someone said, it's not vaporware, > we've done a lot already) > > - 0.7.x can be installed and used with 0.6.x > > - easy_install is going to be deprecated ! use Pip ! > > - the version system will be deprecated, in favor of the one in Distutils > > - no more Distutils monkey-patch that happens once you use the code > (things like 'from distutils import cmd; cmd.Command = CustomCommand') > > - no more custom site.py (that is: if something misses in Python's > site.py we'll add it there instead of patching it) > > - no more namespaced packages system, if PEP 381 (namespaces package > support) makes it to 2.7 > > - The code is splitted in many packages and might be distributed under > several distributions. > > - distribute.resources: that's the old pkg_resources, but > reorganized in clean, pep-8 modules. This package will > only contain the query APIs and will focus on being PEP 376 > compatible. We will promote its usage and see if Pip wants > to use it as a basis. And maybe PyPM once it's open source ? > (<hint> <hint>). > It will probably shrink a lot though, once the stdlib provides PEP 376 > support. > > - distribute.entrypoints: that's the old pkg_resources entry points > system, but on its own. it uses distribute.resources > > - distribute.index: that's package_index and a few other things. > everything required to interact with PyPI. We will promote > its usage and see if Pip wants to use it as a basis. > > - distribute.core (might be renamed to main): that's everything > else, and uses the other packages. > > > Goal: A first release before (or when) Python 2.7 / 3.2 is out. > > > = Virtualenv and the multiple version support in Distribute = > > (I am not saying "We" here because this part was not discussed yet > with everyone) > > Virtualenv allows you to create an isolated environment to install > some distribution without polluting the > main site-packages, a bit like a user site-packages. > > My opinion is that this tool exists only because Python doesn't > support the installation of multiple versions for the same > distributions.
Not only, for me it exists also to provide a way to isolate/chroot an application from the host system. > But if PEP 376 and PEP 386 support are added in Python, we're not far > from being able to provide multiple version support with > the help of importlib. > > Setuptools provided a multiple version support but I don't like its > implementation and the way its works. > I would like to create a new site-packages format that can contains > several versions of the same distribution, and : > > - a special import system using importlib that would automatically > pick the latest version, thanks to PEP 376. > - an API to force at runtime a specific version (that would be located > at the beginning of all imports, like __future__) > - a layout that is compatible with the way OS packagers works with > python packages > > Goal: a prototype asap (one was started under the "VSP" name (virtual > site-packages) but not finished yet) > > Regards > Tarek > -- -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF
signature.asc
Description: OpenPGP digital signature
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
