Re: sysconfig default scheme change in Python 3.10
Hi Simon (2022.03.28_21:44:38_+) > On Mon, 28 Mar 2022 at 21:17:40 +, Stefano Rivera wrote: > > We've fixed this issue in pybind11 and automake-1.16, so far. > > Does this mean that packages that build using Automake, but use the > pregenerated configure/Makefile.in provided by the upstream maintainer > (often on an older or non-Debian distro) instead of regenerating the > build system with dh_autoreconf or equivalent, are going to be broken now? The change is to python.m4, so it gets included in aclocal.m4 and configure. If you rebuild those, you'll get the correct behaviour. I don't think the generated makefiles are affected, just the configured paths. We have upstreamed the patch, https://debbugs.gnu.org/54412 so eventually upstreams will ship something that works correctly, again. SR -- Stefano Rivera http://tumbleweed.org.za/ +1 415 683 3272
Re: sysconfig default scheme change in Python 3.10
On Mon, 28 Mar 2022 at 21:17:40 +, Stefano Rivera wrote: > We've fixed this issue in pybind11 and automake-1.16, so far. Does this mean that packages that build using Automake, but use the pregenerated configure/Makefile.in provided by the upstream maintainer (often on an older or non-Debian distro) instead of regenerating the build system with dh_autoreconf or equivalent, are going to be broken now? If true, this will hopefully not affect too many packages, because using dh_autoreconf is the debhelper default and has been considered a Debian best-practice for a while; but it seems worth being aware of, because I'm reasonably sure there are still some packages that deliberately do not regenerate the build system, and some of those probably build CPython modules. smcv
sysconfig default scheme change in Python 3.10
I realise we made a consequential changes in Python 3.10, that was never announced to the team. At the time, I didn't realise how consequential it would be, but it has forced a few packages to change in response. The change was in python3.10 3.10.2-3 (25 Feb 2022). The default sysconfig change has changed from posix_prefix to posix_local. This matches the long-existing behaviour in Debian's distutils.sysconfig. distutils is being removed from Python's stdlib in 3.12 [PEP-632], so setuptools etc. are being migrated to be configured by sysconfig instead. The practical reason for this is to make "python3 setup.py install" install to /usr/local/lib/... rather than into the generally package-managed /usr/lib/python3/. $ python3.10 -m sysconfig ... Current installation scheme: "posix_local" Paths: data = "/usr/local" include = "/usr/include/python3.10" platinclude = "/usr/include/python3.10" platlib = "/usr/local/lib/python3.10/dist-packages" platstdlib = "/usr/lib/python3.10" purelib = "/usr/local/lib/python3.10/dist-packages" scripts = "/usr/local/bin" stdlib = "/usr/lib/python3.10" ... This has caused some changes for some Debian packages. Generally it can be fixed at a build-tool level, and shouldn't break other Python packages, but for non-standard tools that haven't been fixed yet, you may notice that modules install into /usr/local instead of /usr. dh_python3 will move modules from /usr/local back to /usr, but it's called after dh_install, so dh_install configuration can get broken by this change. The solution in build tools is to explicitly select the posix_prefix scheme, and specify the appropriate prefix (/usr or /usr/local). For build tools that default to /usr/local, this should be all you need to do. We've fixed this issue in pybind11 and automake-1.16, so far. If you find any more tools that need work, feel free to ask me to help. There have been some tweaks to this, since 3.10.2-3, to reduce breakage. One of the significant wants was to not prefix the include paths with /local/, as they are defined as referencing the location of cPython's headers, not the install location for headers [bpo-5]. But some build tools may have their own interpretations of this... Related to this change is something else worth noting: Because distutils is being removed, modern setuptools now carries its own distutils fork, internally. This doesn't have Debian's patches implementing these scheme differences. Instead, it's configured by the presence of a _distutils_system_mod module, which monkey-patches distutils. [PEP-632]: https://peps.python.org/pep-0632/ [bpo-5]: https://bugs.python.org/issue5 SR -- Stefano Rivera http://tumbleweed.org.za/ +1 415 683 3272
Re: DebConf22 -- Python Team BoF + Sprint?
On 2022-03-26 18 h 02, Stefano Rivera wrote: > Hi Louis-Philippe (2022.03.20_21:51:45_+) >> I think it would also be neat to have a team sprint during DebCamp. Here >> are a few ideas of things we could work on: >> >> * pybuild improvements (getting the autopkgtest MR in would be great) >> * general team QA (maybe based on [2]?) >> * lintian tags used for the team > > +1 to a sprint, as usual. > > I think upstream cpython would also appreciate it if we did a pass > through all of our cpython patches and ensured they were forwarded. Ping > bugs, etc. Same goes for any core libraries / big packages. > I've attempted to document them all, this year, which is a start. But > only a start. In many cases forwarding the patch would require clearly > defining the bug, writing reproducer scripts, etc. > > I'd happily mentor people in working on this. I've just submitted the sprint too, the description links to: https://wiki.debian.org/DebConf/22/Sprints Which links to: https://pad.riseup.net/p/dc22pythonsprint-keep Please add relevant goals there :) -- ⢀⣴⠾⠻⢶⣦⠀ ⣾⠁⢠⠒⠀⣿⡁ Louis-Philippe Véronneau ⢿⡄⠘⠷⠚⠋ po...@debian.org / veronneau.org ⠈⠳⣄