Hi,
I'm involved in packaging NumPy for http://pkgsrc.org/. We install a
set of possible BLAS/CBLAS/LAPACK/LAPACKE packages side-by-side in the
same prefix. This includes multiple variants of OpenBLAS with regard to
multithreading (and indexing). For this purpose, we point software to
use the build-time chosen BLAS implementation via BLAS_LIBS and similar
variables, or, as seems to be appropriate for the new meson build of
NumPy, via .pc files.
A package depends on the generic BLAS library family and central user
configuration chooses which one the packages should use during build.
What would be the correct way to force the NumPy build to just use our
BLAS choice, avoiding any automatisms that might surprise us?
How agnostic is NumPy regarding the offered BLAS? Does it have to know
that it is using OpenMP-parallelized OpenBLAS vs. the serial one (I'd
imagine just setting OMP_NUM_THREADS handles paralellism) or MKL,
Netlib, … ? It doesn't scale to have to tell it "openblas" or "netlib",
as there is no universal vocabulary to name the variants, and NumPy
doesn't even know openblas_openmp from serial openblas or
openblas_pthread (right?).
Basically, I want to do
meson setup -Dcblas_pc=$CBLAS_PC
with CBLAS_PC being the module name of one of
$prefix/lib/pkgconfig/cblas.pc
$prefix/lib/pkgconfig/openblas_pthread.pc
$prefix/lib/pkgconfig/openblas_openmp.pc
$prefix/lib/pkgconfig/openblas64_openmp.pc
so that pkg-config does its thing without the NumPy build guessing
around. Is that feasible already? Is it easily supportable with some
changes to the build? I dream of a world where package build scripts
don't have to add dozens of idiosyncratic lines to detect these libs.
I'd like things to work like for CMake's FindBLAS with
-DBLA_PREFER_PKGCONFIG and -DBLA_PKGCONFIG_BLAS=$BLAS_PC (see
https://cmake.org/cmake/help/latest/module/FindBLAS.html
since version 3.25).
Can we have that?
And: NumPy needs CBLAS … does it also need LAPACKE instead of LAPACK?
These are differing libraries, possibly coming in differing binaries,
even if your OpenBLAS builds also combine them. So I guess it should be
-Dcblas_pc and -Dlapacke_pc, both being possibly identical. A build of
the reference Netlib implementation provides four distinct libraries
and .pc files:
$prefix/lib/pkgconfig/cblas.pc
$prefix/lib/pkgconfig/blas.pc
$prefix/lib/pkgconfig/lapacke.pc
$prefix/lib/pkgconfig/lapack.pc
We do support installing openblas64 and friends alongside the others
and I imagine just setting an ILP64 option and repective symbol suffix
(none as of yet, as it's not a settled thing upstream) for the NumPy
build if a 64 variant is chosen by the user. I wonder a bit if there
are possible pitfalls combining other libraries with Python and
indirectly some incompatible BLAS variant via NumPy … but one point of
our user choice is that they could ensure that all packages really use
the same BLAS.
Alrighty then,
Thomas
--
Dr. Thomas Orgis
HPC @ Universität Hamburg
_______________________________________________
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com
