(re-sending to list) On Fri, Dec 29, 2023 at 11:34 AM Ralf Gommers <ralf.gomm...@gmail.com> wrote:
> > > On Fri, Dec 29, 2023 at 12:00 AM Dr. Thomas Orgis < > thomas.or...@uni-hamburg.de> wrote: > >> Am Thu, 28 Dec 2023 20:51:27 +0100 >> schrieb Ralf Gommers <ralf.gomm...@gmail.com>: >> >> > > libblas.so >> > > liblapack.so (NEEDing libblas.so) >> > > libcblas.so (NEEDing libblas.so) >> > > libpapacke.so (NEEDing liblapack.so, hence libblas.so) >> > > >> > > and their respective .pc files. This is the natural order that occus >> to >> > > me when building from netlib upstream. >> > >> > >> > This should work fine. It's auto-detected in NumPy already, and will be >> in >> > SciPy in the future. For now, using `-Dblas=blas -Dlapack=lapack` in the >> > SciPy build should work. >> >> I noticed that with -Dblas=blas, which is in pkgsrc now. The detection >> code sets cblas and finds libcblas by dark magic / defaults that happen >> to match. But what if my setup uses -Dblas=netlib_blas? Then the >> internal guesswork would fail. >> > > If the library name is libcblas.so it will still be found. If it's also a > nonstandard name, then yes it's going to fail. I'd say though that (a) this > isn't a real-world situation as far as we know, (b) just don't do this as a > packager, and (c) if you really must, you can still make it work by > providing a custom `cblas.pc` (see > http://scipy.github.io/devdocs/building/blas_lapack.html#using-pkg-config-to-detect-libraries-in-a-nonstandard-location > ). > > Please consider a mode where the user specifies separate names for all >> 4 components. For package builds, we do not want any guess work, >> including assuming that libblas.so is accompanied by libcblas.so with >> that exact name. >> >> So I'd like >> >> -Dblas=$BLAS_PACKAGE -Dcblas=$CBLAS_PACKAGE \ >> -Dlapack=$LAPACK_PACKAGE -Dlapacke=$LAPACKE_PACKAGE >> >> where the values may all be the same or not. If I fail to provide one >> of those, feel free to guess for the rest (for example, assuming/trying >> that all of those are openblas if I say -Dblas=openblas). > > > We don't use LAPACKE, so that one can be ignored. For CBLAS, I'd honestly > rather get a bug report than add new CLI flags for a situation that seems > to be purely hypothetical. Things work on all known distributions I > believe, and this design isn't new but was the same that numpy.distutils > uses. We can consider a new `-Dcblas` flag at any point, there is nothing > in the design preventing us from adding it later. But I'd rather only do so > if there's a real need. > > >> I also realized that including LAPACK in OpenBLAS is needed, but any >> new BLAS code could start out just replacing the netlib piece by piece. >> The partitioning is there and it is probably good for managing the >> complexity, limiting scope of the individual libraries. >> >> > > Telling the meson build that BLAS is libcblas works as long as >> actually >> > > CBLAS symbols are used. >> > >> > >> > Please never do this. The library is BLAS, so you should use >> `-Dblas=blas` >> > for NumPy. It will find `cblas` just fine that way. >> >> Oh. As I wrote before, we now have >> >> -Csetup-args=-Dblas=${CBLAS_PC} >> -Csetup-args=-Dlapack=${LAPACK_PC} >> >> for math/py-numpy. That's CBLAS_PC, not BLAS_PC. And this works. >> > > I assume that it also passes if you'd pass in BLAS_PC? > > >> >> > This is probably a bug in SciPy. >> >> Well, apparently its just a miscommunication between us two. Scipy is >> fine with >> > > Phew:) I also just confirmed by writing a new SciPy CI job for the split > Netlib BLAS situation, based on how OpenSUSE packages it. And that passes. > > >> >> -Csetup-args=-Dblas=${BLAS_PC} >> -Csetup-args=-Dlapack=${LAPACK_PC} >> >> locating licblas by inferring it from libblas, and finding cblas in >> openblas_foobar, apparently. It prints those lines: >> >> Run-time dependency blas found: YES 3.11.0 >> Run-time dependency cblas found: YES 3.11.0 >> Run-time dependency lapack found: YES 3.11.0 >> blas : blas >> lapack : lapack >> >> While the numpy build does this: >> >> Run-time dependency cblas found: YES 3.11.0 >> Message: BLAS symbol suffix: >> Run-time dependency lapack found: YES 3.11.0 >> blas : cblas >> lapack : lapack >> >> This looks similar to the case of openblas_openmp for -Dblas and -Dlapack: >> >> Run-time dependency openblas_openmp found: YES 0.3.24 >> Message: BLAS symbol suffix: >> Run-time dependency openblas_openmp found: YES 0.3.24 >> blas : openblas_openmp >> lapack : openblas_openmp >> >> So scipy locates cblas based on the name blas, but doesn't really use >> cblas. > > > It does in a few places, like SuperLU. > > >> Numpy is happy with libcblas bringing libblas in and calls it >> blas, but really uses the cblas interface. This looks a bit confusing. >> > > I may be able to add something to the docs, but there should be no > confusion. We need "BLAS with CBLAS symbols". CBLAS should simply not be > considered as a completely separate dependency (it's one library with two > interfaces). I can't think of a reason to do so, nor of a build system that > does it like that. There is no FindCBLAS in CMake for example, it handles > it transparently within FindBLAS. > > >> I guess it makes more sense to continue that discussion on the meson >> PRs for this functionality … as it transcends NumPy, anyway. I hope we >> can settle on something that works for autodetection and prescription >> of all parts. >> > > I'll note that the -D command-line switches are specific to NumPy and > SciPy. Meson only provides a uniform API and the detection methods used > under the hood. Nothing will change in Meson if we decide to add `-Dcblas` > for NumPy and SciPy. > > >> And I need to ponder if I leave it at -Dblas=$CBLAS_PC for pkgsrc now. >> It's somewhat wrong, but also more correct, as NumPy _really_ means to >> use CBLAS API, not BLAS. >> > > My opinion is that this is not right. It may work with pkg-config in your > case, but I don't think it'll be robust for the fallback detection that is > used when pkg-config is not installed. > > Cheers, > Ralf > > > >> Alrighty then, >> >> Thomas >> >> -- >> Dr. Thomas Orgis >> HPC @ Universität Hamburg >> >
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
