Bug#1020561: python3-scipy: Scipy upgrade requires c++ compiler
Package: python3-scipy Followup-For: Bug #1020561 Confirmed that tests still pass (amd64) if python3-pythran is forcibly not installed. Making an audit of where pythran is actually used (in v.10), at runtime that is: scipy/interpolate _rbfinterp_pythran.py see also setup.py, meson.build scipy/optimize _group_columns.py used by ._numdiff.group_columns scipy/linalg _matfuncs_sqrtm_triu.py (not clear that this is used. meson.build refers to the cython variant _matfuncs_sqrtm_triu.pyx) scipy/stats _stats_pythran.py _hypotests.py _stats_mstats_common.py scipy/signal _spectral.py The pythran definitions are supplied as # pythran export ... So they are enclosed in comments. If pythran is not present then the definition is handled as a normal comment, i.e. ignored. At build time python extensions are built using these definitions via meson.build e.g. interpolate/_rbfinterp_pythran.cpython-39-x86_64-linux-gnu.so But once these a built pythran is not needed to rebuild them. This does confuse me, I thought the advantage of pythran was a jit optimisation at runtime. In this case pythran just provides an automated means of running cython, rather than an optimisation to the runtime cpu. Not clear then what the advantage of optimize/_group_columns.py is over optimize/_group_columns.pyx Perhaps the pythran variant is better tuned. So, what is pythran is doing is essentially replacing the .py file with a .so library. It's an ahead-of-time compiler, not a just-in-time compiler. Conclusion:, we want to use pythran at build time. But there's no further reason to depend on it at runtime (not even as Recommends)
Bug#1020561: python3-scipy: Scipy upgrade requires c++ compiler
On 2024-01-27 09:30, Graham Inggs wrote: Hi It seems (at least in scipy/1.10.1-6), that python3-pythran was a build-dependency for all architectures [1], yet, on armhf, python3-scipy did not have a runtime dependency on python3-pythran [2]. The build log of scipy/1.10.1-6 on armhf [3], confirms: Building scipy with SCIPY_USE_PYTHRAN=1 I do not recall seeing any bug reports or autopkgtest failures due to this. Is it possible that scipy can be built with Pythran support, and python3-pythran can be an optional dependency at runtime? If this is true, then we can downgrade python3-pythran from a Depends to a Recommends. A good question. We can look into this and check.
Bug#1020561: python3-scipy: Scipy upgrade requires c++ compiler
Hi It seems (at least in scipy/1.10.1-6), that python3-pythran was a build-dependency for all architectures [1], yet, on armhf, python3-scipy did not have a runtime dependency on python3-pythran [2]. The build log of scipy/1.10.1-6 on armhf [3], confirms: Building scipy with SCIPY_USE_PYTHRAN=1 I do not recall seeing any bug reports or autopkgtest failures due to this. Is it possible that scipy can be built with Pythran support, and python3-pythran can be an optional dependency at runtime? If this is true, then we can downgrade python3-pythran from a Depends to a Recommends. Regards Graham [1] https://salsa.debian.org/python-team/packages/scipy/-/blob/debian/1.10.1-6/debian/control?ref_type=tags#L28 [2] https://salsa.debian.org/python-team/packages/scipy/-/blob/debian/1.10.1-6/debian/control?ref_type=tags#L54 [3] https://buildd.debian.org/status/fetch.php?pkg=scipy=armhf=1.10.1-6=1701791801=0
Bug#1020561: python3-scipy: Scipy upgrade requires c++ compiler
Source: scipy Followup-For: Bug #1020561 We could consider creating a pythran-free build of the package, providing an alternative via Conflicts/Replaces. As you noted, it would require some effort to get the management of the alternative build in place to run alongside the default build. But it can in principle be done. It would possibly want to be done with a python3-scipy-common package dependency containing just the plain .py files separate from the compiled .so extensions.
Bug#1020561: python3-scipy: Scipy upgrade requires c++ compiler
On Tue, 08 Nov 2022 14:39:37 +0100 Drew Parsons wrote: > Source: scipy > Followup-For: Bug #1020561 > > I appreciate the dilemma, but I think we won't be able to avoid it. > To avoid the C++ compiler then, the choice would have to be to not use > pythran with scipy. I can see why pythran would want to use the compiler and scipy want to get the maximum advantage. Still, if it's not necessary, this is a big drawback and especially so if an application was depending on these through some chain. One solution that comes to mind would be having two versions of some part of this chain where one depends on and uses a compiler and the other doesn't. Both would provide the dependency but have different build options. Some conflict or replacement scheme would stop installing both. This would be extra trouble of course.
Bug#1020561: python3-scipy: Scipy upgrade requires c++ compiler
Source: scipy Followup-For: Bug #1020561 I appreciate the dilemma, but I think we won't be able to avoid it. As you noticed, it's not scipy that needs c++, it's pythran. pythran is recommended by scipy developers upstream. The purpose of pythran is to accelerate scipy execution by runtime generation of optimized machine code for particular functions and algorithms. The way pythran works is that it generates at runtime a C++ code block to replace the scipy operatoin in question. The piece of code is then compiled to furnish the machine code that provides the acceleration, taking advantage of CPU optimisations for the local processor that scipy is running on. So availability of a C++ compiler is intrinsic to the way pythran operates. To avoid the C++ compiler then, the choice would have to be to not use pythran with scipy. But the speed-up advantage provided by pythran is supposed to be substantial. In the trade-off between disk storage and execution performance, disk storage comes second to performance. Unfortunately as far as I know we don't have the facility to choose whether to use pythran or not at runtime. The scipy binary package has to be configured at build-time to either use pythran, or not use it at all. I don't think we want to stop using pythran with scipy. But the cost of using it is the requirement to make the C++ compiler available.
Bug#1020561: python3-scipy: Scipy upgrade requires c++ compiler
Package: python3-scipy Version: 1.7.3-2 Severity: normal X-Debbugs-Cc: x...@iki.fi Dear Maintainer, I was running an regular upgrade and noticed a colossal rise in required space. It seems that python3-scipy requires c++-compiler via python3-pythran. The difference seems to be in order of 600 MB. Any change this could be optional? It makes the package much harder to install and use on any sort of production machine which doesn't normally need mahoosive compiler packages and their assorted headaches and security risks. -- System Information: Debian Release: bookworm/sid APT prefers testing APT policy: (500, 'testing'), (490, 'unstable') Architecture: amd64 (x86_64) Kernel: Linux 5.18.0-4-amd64 (SMP w/12 CPU threads; PREEMPT) Locale: LANG=en_DK.UTF-8, LC_CTYPE=en_DK.UTF-8 (charmap=UTF-8), LANGUAGE=en_GB:en Shell: /bin/sh linked to /usr/bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enabled Versions of packages python3-scipy depends on: ii libblas3 [libblas.so.3] 3.10.1-2 ii libc6 2.34-8 ii libgcc-s1 12.2.0-2 ii libgfortran512.2.0-2 ii liblapack3 [liblapack.so.3] 3.10.1-2 ii liblbfgsb0 3.0+dfsg.3-11 ii libstdc++6 12.2.0-2 ii python3 3.10.6-1 ii python3-decorator 5.1.1-2 ii python3-numpy [python3-numpy-abi9] 1:1.21.5-1+b1 Versions of packages python3-scipy recommends: pn g++ | c++-compiler ii python3-pil 9.2.0-1+b1 Versions of packages python3-scipy suggests: pn python-scipy-doc -- no debconf information
