On Sat, 2022-04-09 at 19:08 +0300, Arthur Zamarin wrote: > On 09/04/2022 18.20, Michał Górny wrote: > > Hi, everyone. > > > > TL;DR I think I came up with a feasible plan towards cleanly removing > > dev-python/namespace-* packages that aren't technically needed anymore > > with modern versions of Python. > > > > > > What are namespace packages? Let's take "zope" as an example. > > Normally, various subpackages like "zope.interface" and > > "zope.configuration" would have all to be present in a single directory, > > that is inside the "zope" package. However, if "zope" is made a > > namespace package, its subpackages can be loaded from different > > locations. Notably, in order to test zope.configuration prior to > > installing it, we load it from build tree but its dependency > > zope.interface from system site-packages. > > > > Historically, for this to work, the "zope/__init__.py" had to specify > > some magic incantations. On Gentoo, we've added dev-python/namespace-* > > that installed these magical "__init__.py" files and made all > > subpackages (e.g. dev-python/zope-*) depend on it. > > > > However, Python 3.3 introduced PEP 420 implicit namespaces that made > > magical incantations unnecessary -- if "zope" didn't include > > "__init__.py", it was implicitly treated as a namespace. Unfortunately, > > there were some interoperability issues between the two kinds of magical > > incantations and implicit namespaces. So while we were able to retire > > pkgutil-style namespaces, setuptools-style namespaces remained. > > > > I think we can change that now. > > > > > > I've done some experiments, particularly with dev-python/zope-interface > > and dev-python/zope-configuration. If nobody finds a problem with this > > solution, I think we can aim to bump all packages relying on namespaces > > and retire them in 1-2 months. > > > > The rough idea is to remove RDEP on dev-python/namespace-* from > > the package, and updating python_test to use a temporary pkgutil-style > > incantation (yes, for setuptools-style packages), e.g.: > > > > ``` > > python_compile() { > > distutils-r1_python_compile > > find "${BUILD_DIR}" -name '*.pth' -delete || die > > } > > > > python_test() { > > cd "${BUILD_DIR}/install$(python_get_sitedir)" || die > > # this is needed to keep the tests working while > > # dev-python/namespace-zope is still installed > > cat > zope/__init__.py <<-EOF || die > > __path__ = __import__('pkgutil').extend_path(__path__, > > __name__) > > EOF > > eunittest > > rm zope/__init__.py || die > > } > > ``` > > I quite like that we make the "hack" more local, just in test phase. But > I think this code is quite error prone, and an easy mistake can be made > if someone copies it incorrectly. > > Is it possible to add an eclass function to auto create and cleanup this > file, and all I need to do is to pass it the correct namespace ("zope" > in above example)? > > Also, by using a common function name, it would be easy to fix it if we > did a mistake and find all consumers when we feel safe to cleanup this code. > > One point of failure for this function is how to do the auto cleanup? I > was thinking using environment variables to hold our added magic names, > which is created before `python_test` call and cleans after it is done. > I really prefer if we could call it during prepare phase (so we can > still use auto generated d_e_t calls). >
That's pretty much the big problem -- I don't really know. I feel like adding two functions is pretty much error prone itself, especially given that this 'cd' logic is not necessarily correct for other packages. In the end, we're talking about a few ebuilds for period of a few months, so I'm not sure if there's really a point in putting more effort into it. -- Best regards, Michał Górny