Bug#1067398: python3-numpy: Missing /usr/include/python3.11/numpy link breaks builds
> Backporting sounds like a reasonable approach. If that does not work > as expected, I'll restore the symlink. Excellent. Let me know when this is done, and I'll then update mrbuild to use it, and the builds will then work again. I see you just tagged 1:1.26.4+ds-6 in git with the .pc stuff. I'll make my mrbuild changes now against that, and will push them when you tell me that you're done and that you have pushed it. Thanks!
Bug#1067398: python3-numpy: Missing /usr/include/python3.11/numpy link breaks builds
Hi Dima, * Dima Kogan [2024-03-21 07:23]: My feeling is that being confused by that symlink is a bug, but I didn't read #998084 in great detail, so maybe I'm missing some nuance. So let's make this work. My understanding is that /usr/include/pythonX.Y as Python header directory will be made available from inside a venv if you use the system interpreter (which venv does by default). Now, the system NumPy headers will leak into the venv, regardless of the NumPy package that is actually installed there, causing conflicts. Proposal: if pkg-config files will be added in the near future, can we wait until those are available before removing the symlink? Or, can you backport them into the current package? Backporting sounds like a reasonable approach. If that does not work as expected, I'll restore the symlink. But if something was confused by the symlink, would it also be confused by the .pc file? Very good question. I suspect the answer is yes, but there is a pull request that would make it work as expected [1]. [1] https://github.com/pypa/virtualenv/issues/2637 Currently the Makefile launches Python and imports sysconfig, which is relatively fast. As we can see, importing numpy also, is MUCH slower. All I want is the include path; I shouldn't need to initialize all of numpy to do that. I understand and I would not want to do that either. Cheers Timo -- ⢀⣴⠾⠻⢶⣦⠀ ╭╮ ⣾⠁⢠⠒⠀⣿⡁ │ Timo Röhling │ ⢿⡄⠘⠷⠚⠋⠀ │ 9B03 EBB9 8300 DF97 C2B1 23BF CC8C 6BDD 1403 F4CA │ ⠈⠳⣄ ╰╯ signature.asc Description: PGP signature
Bug#1067398: python3-numpy: Missing /usr/include/python3.11/numpy link breaks builds
Hi Timo. Thanks for replying.
My feeling is that being confused by that symlink is a bug, but I didn't
read #998084 in great detail, so maybe I'm missing some nuance. So let's
make this work.
** Short version **
Proposal: if pkg-config files will be added in the near future, can we
wait until those are available before removing the symlink? Or, can you
backport them into the current package? Essentially, moving the
information that was previously in the symlink into the .pc file? If I
can assume the symlink exists, I will use it in mrbuild and/or the
builds of projects that use it.
But if something was confused by the symlink, would it also be confused
by the .pc file?
** Long version **
The breakage is in packages where I'm upstream. These use my build
system: mrbuild. This asks Python for its header directory:
sysconfig.get_config_var('INCLUDEPY'), and asks the compiler to look
there. Today this ends up passing
-I/usr/include/python3.11
which would previously succesfully resolve (via the now-missing symlink)
#include
mrbuild very explicitly does not use setup.py or anything of that
nature. It's used for mixed-language projects, and I don't want multiple
build systems for each language that wanted to rewrite the world. Some
discussion and rationale in a blog post:
https://notes.secretsauce.net/notes/2017/11/14_python-extension-modules-without-setuptools-or-distutils.html
I just tried to patch mrbuild to use np.get_includes(). This works, but
it's slow. After warming up the caches, I see this:
$ time python3 -c 'import sysconfig'
python3 -c 'import sysconfig' 0.02s user 0.01s system 97% cpu 0.025 total
$ time python3 -c 'import numpy'
python3 -c 'import numpy' 0.51s user 0.67s system 634% cpu 0.185 total
Currently the Makefile launches Python and imports sysconfig, which is
relatively fast. As we can see, importing numpy also, is MUCH slower.
All I want is the include path; I shouldn't need to initialize all of
numpy to do that.
Thanks much. Hopefully we can find a nice way to satisfy everybody
Bug#1067398: python3-numpy: Missing /usr/include/python3.11/numpy link breaks builds
Hi Dima, * Dima Kogan [2024-03-20 18:23]: Can we get that symlink back, please? I removed that symlink in the latest release, because it is not created by upstream but a Debian specific patch, and according to #998084 [1], it can badly interfere with venv setups. I would very much prefer if other Debian packages used the canonical way of discovering NumPy include directories, i.e., numpy.get_include(). Starting with NumPy 2.0, there will be a pkgconf file available, too. That being said, I can temporarily reinstate the symlinks if you cannot fix the issue on your side in a timely manner, but I'd like them gone for the trixie release. How do you wish to proceed? Cheers Timo [1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=998084 -- ⢀⣴⠾⠻⢶⣦⠀ ╭╮ ⣾⠁⢠⠒⠀⣿⡁ │ Timo Röhling │ ⢿⡄⠘⠷⠚⠋⠀ │ 9B03 EBB9 8300 DF97 C2B1 23BF CC8C 6BDD 1403 F4CA │ ⠈⠳⣄ ╰╯ signature.asc Description: PGP signature
Bug#1067398: python3-numpy: Missing /usr/include/python3.11/numpy link breaks builds
Package: python3-numpy Version: 1:1.26.4+ds-5 Severity: important X-Debbugs-Cc: none, Dima Kogan Hi. Many of my packages just started to FTBFS. For instance this: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1067270 This is due to the /usr/include/python3.11/numpy (and/or other versions) symlink not being shipped in python3-numpy anymore. I haven't bisected, but it has worked as recently as python3-numpy = 1:1.24.2-2. It no longer works as of 1:1.26.4+ds-5. Can we get that symlink back, please? Thanks! -- System Information: Debian Release: trixie/sid APT prefers unstable APT policy: (800, 'unstable'), (500, 'unstable-debug'), (500, 'stable') Architecture: amd64 (x86_64) Kernel: Linux 6.6.13-amd64 (SMP w/4 CPU threads; PREEMPT) Kernel taint flags: TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE Locale: LANG=en_US.utf-8, LC_CTYPE=en_US.utf-8 (charmap=UTF-8) (ignored: LC_ALL set to en_US.utf-8), LANGUAGE=en_US.utf-8 Shell: /bin/sh linked to /usr/bin/dash Init: systemd (via /run/systemd/system) Versions of packages python3-numpy depends on: ii libatlas3-base [liblapack.so.3]3.10.3-13 ii libblas3 [libblas.so.3]3.11.0-2 ii libc6 2.37-7 ii libgcc-s1 13.2.0-2 ii liblapack3 [liblapack.so.3]3.11.0-2 ii libopenblas0-openmp [liblapack.so.3] 0.3.25+ds-1 ii libopenblas0-pthread [liblapack.so.3] 0.3.25+ds-1 ii python33.11.4-5+b1 ii python3-pkg-resources 68.1.2-1 python3-numpy recommends no packages. Versions of packages python3-numpy suggests: ii gcc 4:13.2.0-1 ii gfortran4:13.2.0-1 ii python3-dev 3.11.4-5+b1 ii python3-pytest 7.4.0-2 -- no debconf information
