Hi, TL;DR: should we depend on setuptools by default? Alternatively, should we add distutils_enable_setuptools API to provide at least partial validity checks.
The problem =========== The vast majority of Python packages nowadays uses setuptools as their build system. According to a cheap grep, 1633 out of 2415 packages using distutils-r1 depends on it -- and I suspect the vast majority of those that do not are simply missing the dependency. Are there valid cases for not using setuptools? Notably, packages needed to bootstrap setuptools aren't using it. Plus some simple packages that really don't need its features. There are also packages that use setuptools but have distutils fallback. We don't like them because switching between build systems involves file <-> directory replacement that isn't handled cleanly by our package managers. Therefore, we want to always force one build system in them. The setuptools dependency is so common it's easy to miss. Notably, it's a prerequisite of specifying package dependencies, so it's normally not listed in dependencies. Finally, while most of the time setuptools is just BDEPEND, there are cases when it's RDEPEND as well. The most common is the use of entry_points -- and again, upstreams don't mention it explicitly. All that considered, I think we should work on providing a better API for depending on setuptools, to reduce the number of missing dependencies and make it possible to automatically test for them (reminder: PMS doesn't permit inspecting *DEPEND). Variant 1: automatic dependency on setuptools ============================================= Basically, we add a new trinary pre-inherit variable: DISTUTILS_USE_SETUPTOOLS=no -> no deps DISTUTILS_USE_SETUPTOOLS=bdepend -> add to BDEPEND (the default) DISTUTILS_USE_SETUPTOOLS=rdepend -> add to BDEPEND+RDEPEND This is roughly 'erring on the safe side'. The default will work for the majority of packages. We will have to disable it for setuptools bootstrap deps, and devs will be able to adjust it to correct values as they update ebuilds. For the time being, existing *DEPEND on setuptools will avoid breaking stuff. This will also enable me to add extra QA checks to esetup.py. It should be able to reasonably detect incorrect value and report it. This will imply some 'false positives' for packages that use the old method of specifying setuptools in RDEPEND but that's a minor hassle. Pros: - works out of the box for the majority of packages - enables full-range QA checking Cons: - pre-inherit variable - some (harmless) false positives on existing packages Variant 2: distutils_enable_setuptools ====================================== The alternative method is to add another function to the distutils_enable* series: distutils_enable_setuptools [-r] The basic form adds it to BDEPEND, -r B+RDEPEND. Of course, no dep is present by default. The main difference from setting deps explicitly is that it permits us to do a minimal QA check between pure BDEPEND and B+RDEPEND for now. When all ebuilds are migrated from explicit dependencies to this method, we can also start detecting missing deps completely. However, that presumes we require using this function rather than explicit deps. Pros: - no pre-inherit variables Cons: - only partial QA check possible for the time being - requires migrating all ebuilds long-term Variant 3: leave as-is, add minimal install-qa-check.d ====================================================== We can just continue adding deps manually, and add a minimal install-qa- check.d that greps installed scripts for entry_points usage. This will let us detect missing RDEPEND on setuptools but not BDEPEND. Pros: - no changes to ebuilds Cons: - only partial QA check possible WDYT? ===== Both options have their pros and cons. I think V1 is the best since it avoids a common mistake, and gives full range QA check. It also doesn't interfere with existing deps. V2 neatly fits into the recent series but still requires users to remember to call it, and we can't report missing calls until we clean up all ebuilds. V3 provides only minimal QA improvement without changing the eclass. WDYT? Do you have any other ideas? -- Best regards, Michał Górny
signature.asc
Description: This is a digitally signed message part
