Tarek Ziadé wrote: > Hi, > > I am back on that problem with the code that builds the file list. The > current Distutils trunk isn't working anymore with setuptools because > of a recursive loop: > > distutils.sdist.run() -> setuptools.build_py.data_files -> > setuptools.egg_info.run() -> distutils.sdist.add_defaults() -> > setuptools.build_py.data_files -> etc > > The mess is introduced by the fact that build_py is patched by > setuptools in order to inject the files that are provided by the > (D)VCS. > But it also uses some APIs provided by sdist to complete the file list. > > In order to solve this, we need to define clearly what is the role of > each command, and make sure it's a distinct role. > > which is not the case right now : > > 1/ distutils.sdist = this command is used to build a source distribution > 2/ setuptools.egg_info = this command is used to build an .egg-info > directory but *also* injects the files founded with (D)VCS in the > MANIFEST > 3/ distutils.build_py = this command is used to build pure python > module but *also* to find all the .py files to include in a > distribution (used by sdist). In fact, it plays a central role in > sdist to get the files to include. > > Here's a first thaught to discuss: > > what about introducing a new simple command called "manifest" that > could be used by sdist or any other command, and that would be > responsible of collecting the files that are part of the distribution > (and nothing else). This command would generate the MANIFEST file and > also provide the APIs to get the files included into MANIFEST. > > This command would introduce and use a simple plugin system similar to > the "setuptools.file_finders" entry points setuptools has for (D)VCS > files collecting. But with the files list being built as an argument > to the plugin. (partly described here > http://wiki.python.org/moin/Distutils/ManifestPluginSystem) so > Distutils, setuptools or any third party tool can register some code > that add or remove file in this file list. > > The manifest command would provide default plugins, and setuptools > could refactor part of its code to use the manifest command rather > than calling sdist APIs. The goal would be to have the same result at > the end, but make it simpler to extend, and avoid command > interdependencies like what we have today (and that makes it hard to > maitain and make evolve). > > For instance the MANIFEST.in templating system would be one of the > default plugin provided by Distutils. > > The initial work would consist of refactoring the current code that > gets the files, using different strategies, into plugins for the new > manifest command, then make sdist uses this command as a subcommand. > I think this is good stuff. You might find that the pieces that collect files for the MANIFEST are useful in more than creating the MANIFEST, though. Unless I've missed a message saying that we're going to collapse sdist and package-installed-files together.
> The second phase would consist of working at setuptools level to use > the same technique. Setuptools would be able to make existing > "setuptools.file_finders" entry points work with Distutils manifest > command registery by providing a bridge. > > Now for the plugin system, I see two options so far: > > 1/ create a simple ad-hoc plugin system in Distutils, and declare the > plugins in a new section in distutils.cfg / pydistutils.cfg for > loading them (setuptools existing entry points would be collected > through a unique plugin declared that would act like a bridge) > > 2/ use entry points (so add them into Distutils) and define a entry > point name based on the command name, maybe > "distutils:metadata.file_finders" so the plugin system could be used > elsewhere in distutils. > Having a library that makes creating plugins easy is a good general purpose thing. Whatever plugin system is adopted/created, it would be good for it to not be internal to distutils. I've always hated that distutils option handling is different from the option handling that a coder can use from the standard library. -Toshio
signature.asc
Description: OpenPGP digital signature
_______________________________________________ Distutils-SIG maillist - [email protected] http://mail.python.org/mailman/listinfo/distutils-sig
