On Sat, 2022-03-26 at 11:42 +0100, Stephan Lachnit wrote: > On Sat, Mar 26, 2022 at 2:36 AM M. Zhou <lu...@debian.org> wrote: > > > > Indeed supporting number crunching programs on ancient > > hardware is not meaningful, but the demand on Debian's > > support for number crunching is not that strong according > > to my years of observation. > > > > For popular applications that can take advantage of above-baseline > > instruction sets, they will eventually write the dynamic code > > dispatcher and add the fallback. > > > > For applications seriously need performance, they will > > leave CPU and go to GPU or other hardware. If the user correctly > > write the code and fully leverage GPU, the non-optimal CPU > > code won't necessarily be a bottleneck. > > > > For applications seriously need CPU performance, they are > > possibly going to tell the users how to tweak compiling > > parameters and how to compile locally. > > I have to disagree on this one. Yes, runtime detection and GPU > acceleration is great and all, but not every scientific library does > it and I think it's unrealistic for us to patch them all up.
Please note I wrote "they (i.e. the upstream)" will implement the runtime detection or GPU acceleration, instead of us (Debian). > Also I don't like the point "since there is low demand for number > crunching on Debian, so let's just continue to ignore this problem". If it was 6 years ago, I would disagree with what I've said in the original post. Whether you like it or not, what I said is my changed mind after closely working on numerical related libraries for 6 years in Debian. And to be clear, I hold a negative opinion on what we Debian could actually change besides the upstream. If the upstream does not write runtime detection or GPU acceleration, they are either not facing a wide range of audience, or the problem does not matter, or simply the software isn't appropriate for Debian packaging. I mentioned infinite times that the eigen3 library which implements the core numerical computation part of TensorFlow does not support runtime detection -- because CPU acceleration does not matter for most of the users. Sane users who really need CPU performance are able to recompile tensorflow themselves. > At least I know a decent amount of people that use Debian (or > downstream distros) for scientific number crunching. Compiling > optimized for large workloads will always be a thing no matter the > baseline, but when getting started distro packages are just one less > thing to care about. I humbly believe over 1/3 of packages I (co-)maintained for Debian are for number crunching. And I INSIST in my NEGATIVE opinion after trying to do some experiments over the years. The number of people who really care about the ISA baseline for Debian distributed package is very likely less than you expected. I appreciate people who speak for ISA baseline, and appreciate any actual effort in this regard. But the lack of care eventually changed my mind and make me hold a negative opinion. If you think I was simply unsuccessful in promoting any solution for the topics in this discussion, please go ahead and I will support you in a visible way. > On Sat, Mar 26, 2022 at 7:25 AM Andrey Rahmatullin <w...@debian.org> wrote: > > > > A partial arch (whatever that is, yeah) with the x86-64-v3 baseline, and > > optionally raise the main amd64 baseline to x86-64-v2? > > +1 So again, that's possibly something like a partial debian archive with a dpkg fork I mentioned. That's probably the same idea as the ancient SIMDebian proposal. See the example patch for dpkg: https://github.com/SIMDebian/dpkg/commit/13b062567ac58dd1fe5395fb003d6230fd99e7c1 So that a partial archive with selected source packages can be rebuilt automatically in bumped ISA baseline. To be clear, the fact that tensorflow does not support runtime detection while the baseline code sucks in performance is the direct reason why I proposed SIMDebian. The project is abandoned, and patch is only for reference.