On Thu, Jan 23, 2020 at 5:11 PM Dima Pasechnik <[email protected]> wrote: > > On Wed, Jan 22, 2020 at 5:20 PM Isuru Fernando <[email protected]> wrote: > > > > [root@50586643ff22 /]# pkg-config --modversion blas > > 0.3.7 > > [root@50586643ff22 /]# pkg-config --modversion cblas > > 3.9.0 > > [root@50586643ff22 /]# pkg-config --modversion lapack > > 3.9.0 > > [root@50586643ff22 /]# pkg-config --modversion openblas > > 0.3.7 > > To make it robust, I'd only allow cblas.pc to be used if it really is > based on openblas > and the same for lapack.pc > > One way to do this is to test openblas for a symbol that is on Arch > in cblas, not in openblas, > and the same for lapack. > > Could someone figure out these symbols, or give me access to an Arch box?
I've opened https://trac.sagemath.org/ticket/29071 to deal with this issue. > > > > > On Wed, Jan 22, 2020 at 11:18 AM Dima Pasechnik <[email protected]> wrote: > >> > >> On Wed, Jan 22, 2020 at 5:03 PM Isuru Fernando <[email protected]> wrote: > >> > > >> > > On Arch, does cblas.pc exist? > >> > > >> > Yes. > >> > >> Thanks. I wonder what on Arch is the output of > >> > >> pkg-config --modversion X > >> > >> for X in [cblas,blas,lapack,openblas] > >> > >> > >> > > >> > > this is system-dependent. > >> > > >> > Yes, that's why we need to have fallbacks. > >> > > >> > Isuru > >> > > >> > On Wed, Jan 22, 2020 at 11:00 AM Dima Pasechnik <[email protected]> > >> > wrote: > >> >> > >> >> On Wed, Jan 22, 2020 at 4:17 PM Isuru Fernando <[email protected]> wrote: > >> >> > > >> >> > > maybe they split openblas.so into separate sections for an unclear > >> >> > > to me reason, but most applications I know uses cblas, blas and > >> >> > > lapack, so they need to link them all anyway. > >> >> > > >> >> > True, but in sage you are copying `openblas.pc` to `cblas.pc` and > >> >> > `lapack.pc` which is wrong. `openblas.pc` should only be expected to > >> >> > provide blas. > >> >> > >> >> this is system-dependent. Debian and Fedora have no libcblas and no > >> >> cblas.pc, cblas is provided by openblas. > >> >> Some *BSD systems have libcblas conflcting with libopenblas. > >> >> > >> >> On Arch, does cblas.pc exist? > >> >> > >> >> > >> >> > > >> >> > A solution would be to check that `blas.pc`, `cblas.pc`, `lapack.pc` > >> >> > are already on the system and if not copy `openblas.pc` to the three > >> >> > names. > >> >> > > >> >> > Isuru > >> >> > > >> >> > On Wed, Jan 22, 2020 at 10:14 AM Isuru Fernando <[email protected]> > >> >> > wrote: > >> >> >> > >> >> >> > errors we see here, regarding absence of certain openmp functions > >> >> >> > in cblas, seem to indicate that cblas on Arch does not come from > >> >> >> > openblas. perhaps what we see are reference cblas and blas from > >> >> >> > openblas installed at the same time, in error. > >> >> >> > >> >> >> Can you please explain more? As I said earlier, it doesn't matter > >> >> >> where CBLAS is coming from. I checked in Arch linux and cblas is > >> >> >> linked to openblas. > >> >> >> > >> >> >> [root@50586643ff22 /]# ldd /usr/lib/libcblas.so.3 > >> >> >> linux-vdso.so.1 (0x00007ffd2f7bb000) > >> >> >> libblas.so.3 => /usr/lib/libblas.so.3 (0x00007f848f079000) > >> >> >> libc.so.6 => /usr/lib/libc.so.6 (0x00007f848eeb2000) > >> >> >> libm.so.6 => /usr/lib/libm.so.6 (0x00007f848ed6c000) > >> >> >> libpthread.so.0 => /usr/lib/libpthread.so.0 > >> >> >> (0x00007f848ed4a000) > >> >> >> libgomp.so.1 => /usr/lib/libgomp.so.1 (0x00007f848ed11000) > >> >> >> /usr/lib64/ld-linux-x86-64.so.2 (0x00007f84902dc000) > >> >> >> libdl.so.2 => /usr/lib/libdl.so.2 (0x00007f848ed0c000) > >> >> >> > >> >> >> [root@50586643ff22 /]# ls -al /usr/lib/libblas.so.3 > >> >> >> lrwxrwxrwx 1 root root 22 Aug 20 10:35 /usr/lib/libblas.so.3 -> > >> >> >> libopenblasp-r0.3.7.so > >> >> >> > >> >> >> > >> >> >> Isuru > >> >> >> > >> >> >> On Wed, Jan 22, 2020 at 10:09 AM Dima Pasechnik <[email protected]> > >> >> >> wrote: > >> >> >>> > >> >> >>> > >> >> >>> > >> >> >>> On Wed, 22 Jan 2020, 15:59 Isuru Fernando, <[email protected]> wrote: > >> >> >>>> > >> >> >>>> > But Arch does not do this, it instead provides dog-slow CBLAS and > >> >> >>>> LAPACK, built > >> >> >>>> from another (reference, i.e. no assembler, no optimisation) > >> >> >>>> implementation. > >> >> >>>> > >> >> >>>> This is simply not true. CBLAS's performance does not depend on > >> >> >>>> which implementation it comes from. CBLAS's performance depends on > >> >> >>>> the underlying BLAS implementation and on ArchLinux, when OpenBLAS > >> >> >>>> is installed, CBLAS uses OpenBLAS as the underlying BLAS > >> >> >>>> implementation. In fact, OpenBLAS uses the CBLAS code from > >> >> >>>> Netlib's reference implementation. > >> >> >>> > >> >> >>> > >> >> >>> errors we see here, regarding absence of certain openmp functions > >> >> >>> in cblas, seem to indicate that cblas on Arch does not come from > >> >> >>> openblas. perhaps what we see are reference cblas and blas from > >> >> >>> openblas installed at the same time, in error. > >> >> >>> > >> >> >>> maybe they split openblas.so into separate sections for an unclear > >> >> >>> to me reason, but most applications I know uses cblas, blas and > >> >> >>> lapack, so they need to link them all anyway. > >> >> >>> > >> >> >>> > >> >> >>>> > >> >> >>>> As for LAPACK, in OpenBLAS, they have implemented a handful of > >> >> >>>> functions to use OpenBLAS internals to make it parallel, but most > >> >> >>>> of the LAPACK functions are from Netlib's Reference > >> >> >>>> implementation. So, when ArchLinux is using LAPACK from netlib, > >> >> >>>> they are losing out on only a few LAPACK functions. > >> >> >>>> > >> >> >>>> Isuru > >> >> >>>> > >> >> >>>> On Wed, Jan 22, 2020 at 9:27 AM Dima Pasechnik <[email protected]> > >> >> >>>> wrote: > >> >> >>>>> > >> >> >>>>> > >> >> >>>>> > >> >> >>>>> On Wed, 22 Jan 2020, 15:12 Vincent Delecroix, > >> >> >>>>> <[email protected]> wrote: > >> >> >>>>>> > >> >> >>>>>> Le 22/01/2020 à 15:41, Dima Pasechnik a écrit : > >> >> >>>>>> > On Wed, Jan 22, 2020 at 2:31 PM Vincent Delecroix > >> >> >>>>>> > <[email protected]> wrote: > >> >> >>>>>> >> > >> >> >>>>>> >> I had the same annoying trouble... Is it really a bug in > >> >> >>>>>> >> arch? I don't > >> >> >>>>>> >> find anywhere where libopenblas is supposed to provide the > >> >> >>>>>> >> LAPACK > >> >> >>>>>> >> interface. To me it is just true for SageMath and false for > >> >> >>>>>> >> Archlinux. > >> >> >>>>>> > > >> >> >>>>>> > well, a "bug", in the sense that their openblas configuration > >> >> >>>>>> > just > >> >> >>>>>> > does not make any sense, > >> >> >>>>>> > because if you install a highly optimised BLAS implementation, > >> >> >>>>>> > which can also > >> >> >>>>>> > provided a highly optimised CBLAS and LAPACK, in a sane world > >> >> >>>>>> > you'd install them > >> >> >>>>>> > too. But Arch does not do this, it instead provides dog-slow > >> >> >>>>>> > CBLAS and > >> >> >>>>>> > LAPACK, built > >> >> >>>>>> > from another (reference, i.e. no assembler, no optimisation) > >> >> >>>>>> > implementation. > >> >> >>>>>> > This causes Sage's logic, which expects that openblas will also > >> >> >>>>>> > provide CBLAS and LAPACK, to fail. > >> >> >>>>>> > > >> >> >>>>>> > Yes, it is possible to make tests to detect this silly setup, > >> >> >>>>>> > but I > >> >> >>>>>> > think that everyone's time > >> >> >>>>>> > is much better spent if Arch provided a full build of > >> >> >>>>>> > openblas, with > >> >> >>>>>> > CBLAS and LAPACK, > >> >> >>>>>> > instead. > >> >> >>>>>> > >> >> >>>>>> Hence there are two bugs > >> >> >>>>>> > >> >> >>>>>> * a "logical" bug in archlinux that provides a slow lapack for > >> >> >>>>>> (apparently) no good reason > >> >> >>>>>> > >> >> >>>>>> * a structural bug in SageMath that assumes that openblas does > >> >> >>>>>> install the LAPACK interface > >> >> >>>>>> > >> >> >>>>>> > >> >> >>>>>> I tend to agree that the resolution of any of these two would > >> >> >>>>>> solve > >> >> >>>>>> the compilation. But in an ideal world we would just solve the > >> >> >>>>>> two. > >> >> >>>>> > >> >> >>>>> > >> >> >>>>> a meaningful resolution of this would be to test openblas for > >> >> >>>>> lapack and cblas capacities. > >> >> >>>>> > >> >> >>>>> this is relatively easy, and would result in openblas on Arch > >> >> >>>>> being built by Sage. > >> >> >>>>> > >> >> >>>>> as far as expanding this to other implementations of blas/lapack, > >> >> >>>>> it is harder. > >> >> >>>>>> > >> >> >>>>>> > >> >> >>>>>> > >> >> >>>>>> >> > >> >> >>>>>> >> Le 13/01/2020 à 11:44, Dima Pasechnik a écrit : > >> >> >>>>>> >>> On Sat, Jan 11, 2020 at 11:35 AM arojas <[email protected]> > >> >> >>>>>> >>> wrote: > >> >> >>>>>> >>>> > >> >> >>>>>> >>>> El sábado, 11 de enero de 2020, 11:23:32 (UTC+1), Dima > >> >> >>>>>> >>>> Pasechnik escribió: > >> >> >>>>>> >>>>> > >> >> >>>>>> >>>>> > >> >> >>>>>> >>>>> > >> >> >>>>>> >>>>> On Sat, 11 Jan 2020, 10:19 arojas, <[email protected]> > >> >> >>>>>> >>>>> wrote: > >> >> >>>>>> >>>>>> > >> >> >>>>>> >>>>>> I suspect this may be due to our openblas package only > >> >> >>>>>> >>>>>> providing libblas.so (not libcblas.so or liblapack.so, > >> >> >>>>>> >>>>>> which in our case come from the netlib blas). > >> >> >>>>>> >>>>> > >> >> >>>>>> >>>>> > >> >> >>>>>> >>>>> Are you saying that your libopenblas.so does not provide > >> >> >>>>>> >>>>> everything that is in openblas by default? > >> >> >>>>>> >>>>> > >> >> >>>>>> >>>>> This is largely defeating its purpose. > >> >> >>>>>> >>>>> Sage's openblas does provide the cblas and lapack > >> >> >>>>>> >>>>> capabilities. > >> >> >>>>>> >>>> > >> >> >>>>>> >>>> > >> >> >>>>>> >>>> In that case, openblas' spkg-configure should check > >> >> >>>>>> >>>> whether cblas symbols are provided by the system openblas, > >> >> >>>>>> >>>> and if not it should compile sage's one. It shouldn't > >> >> >>>>>> >>>> unconditionally assume that that's always the case. > >> >> >>>>>> >>> > >> >> >>>>>> >>> I maintain that it's a bug in Arch, that libopenblas cannot > >> >> >>>>>> >>> be used as > >> >> >>>>>> >>> a replacement of cblas and lapack, for it defeats its > >> >> >>>>>> >>> purpose. > >> >> >>>>>> >>> Most uses of BLAS routines are either via LAPACK, or from > >> >> >>>>>> >>> C/C++, via cblas. > >> >> >>>>>> >>> > >> >> >>>>>> >>> > >> >> >>>>>> >>> > >> >> >>>>>> >>>> > >> >> >>>>>> >>>> -- > >> >> >>>>>> >>>> You received this message because you are subscribed to the > >> >> >>>>>> >>>> Google Groups "sage-devel" group. > >> >> >>>>>> >>>> To unsubscribe from this group and stop receiving emails > >> >> >>>>>> >>>> from it, send an email to > >> >> >>>>>> >>>> [email protected]. > >> >> >>>>>> >>>> To view this discussion on the web visit > >> >> >>>>>> >>>> https://groups.google.com/d/msgid/sage-devel/a1516e56-ab25-410e-be0f-75a7a6713466%40googlegroups.com. > >> >> >>>>>> >>> > >> >> >>>>>> >> > >> >> >>>>>> >> -- > >> >> >>>>>> >> You received this message because you are subscribed to the > >> >> >>>>>> >> Google Groups "sage-devel" group. > >> >> >>>>>> >> To unsubscribe from this group and stop receiving emails from > >> >> >>>>>> >> it, send an email to [email protected]. > >> >> >>>>>> >> To view this discussion on the web visit > >> >> >>>>>> >> https://groups.google.com/d/msgid/sage-devel/dbf38073-170b-4c7d-63b9-d4429a8794e2%40gmail.com. > >> >> >>>>>> > > >> >> >>>>>> > >> >> >>>>>> -- > >> >> >>>>>> You received this message because you are subscribed to the > >> >> >>>>>> Google Groups "sage-devel" group. > >> >> >>>>>> To unsubscribe from this group and stop receiving emails from > >> >> >>>>>> it, send an email to [email protected]. > >> >> >>>>>> To view this discussion on the web visit > >> >> >>>>>> https://groups.google.com/d/msgid/sage-devel/e3ad609d-8d21-bb10-2876-367fdf1b7e24%40gmail.com. > >> >> >>>>> > >> >> >>>>> -- > >> >> >>>>> You received this message because you are subscribed to the > >> >> >>>>> Google Groups "sage-devel" group. > >> >> >>>>> To unsubscribe from this group and stop receiving emails from it, > >> >> >>>>> send an email to [email protected]. > >> >> >>>>> To view this discussion on the web visit > >> >> >>>>> https://groups.google.com/d/msgid/sage-devel/CAAWYfq3p7x9KmwHLmhjOtuoy_3PdsohHN6RJ0fp0Bzxv-wHODQ%40mail.gmail.com. > >> >> >>>> > >> >> >>>> -- > >> >> >>>> You received this message because you are subscribed to the Google > >> >> >>>> Groups "sage-devel" group. > >> >> >>>> To unsubscribe from this group and stop receiving emails from it, > >> >> >>>> send an email to [email protected]. > >> >> >>>> To view this discussion on the web visit > >> >> >>>> https://groups.google.com/d/msgid/sage-devel/CA%2B01voO3C4pdc5gbU02KzhR1DuzZ0Gwv2n6CoSukODeEqFfEjQ%40mail.gmail.com. > >> >> >>> > >> >> >>> -- > >> >> >>> You received this message because you are subscribed to the Google > >> >> >>> Groups "sage-devel" group. > >> >> >>> To unsubscribe from this group and stop receiving emails from it, > >> >> >>> send an email to [email protected]. > >> >> >>> To view this discussion on the web visit > >> >> >>> https://groups.google.com/d/msgid/sage-devel/CAAWYfq1qxH7SUEWy%2BP3CQ_WeCimK_ByX3Q_HNz4Li%2BvwZCznYA%40mail.gmail.com. > >> >> > > >> >> > -- > >> >> > You received this message because you are subscribed to the Google > >> >> > Groups "sage-devel" group. > >> >> > To unsubscribe from this group and stop receiving emails from it, > >> >> > send an email to [email protected]. > >> >> > To view this discussion on the web visit > >> >> > https://groups.google.com/d/msgid/sage-devel/CA%2B01voO-GZJ9gSgsQ-N4stwfkvonWz2aDU%2Bcpv6ViJ8eN%3D0wNw%40mail.gmail.com. > >> >> > >> >> -- > >> >> You received this message because you are subscribed to the Google > >> >> Groups "sage-devel" group. > >> >> To unsubscribe from this group and stop receiving emails from it, send > >> >> an email to [email protected]. > >> >> To view this discussion on the web visit > >> >> https://groups.google.com/d/msgid/sage-devel/CAAWYfq3D4YGM2HR7SORfjDFhkjQ%3Dm1Jn4CsEwmioF-18poxRdQ%40mail.gmail.com. > >> > > >> > -- > >> > You received this message because you are subscribed to the Google > >> > Groups "sage-devel" group. > >> > To unsubscribe from this group and stop receiving emails from it, send > >> > an email to [email protected]. > >> > To view this discussion on the web visit > >> > https://groups.google.com/d/msgid/sage-devel/CA%2B01voO%2BWkpafvpuUfLmGw8qDZ1qFMwzoA6HFBBBiZWYGdTqpw%40mail.gmail.com. > >> > >> -- > >> You received this message because you are subscribed to the Google Groups > >> "sage-devel" group. > >> To unsubscribe from this group and stop receiving emails from it, send an > >> email to [email protected]. > >> To view this discussion on the web visit > >> https://groups.google.com/d/msgid/sage-devel/CAAWYfq1H_8RGBSN5DW25%3Dvf7SE_TJXeFtVpKSQQU%2BRcsDmFQ5w%40mail.gmail.com. > > > > -- > > You received this message because you are subscribed to the Google Groups > > "sage-devel" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [email protected]. > > To view this discussion on the web visit > > https://groups.google.com/d/msgid/sage-devel/CA%2B01voMM5COObh5FWUd%2BH%2BXEweA_jbCrNdMCRvU3x6KhaYHsxQ%40mail.gmail.com. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAAWYfq0-3TRSDD5uDh3i3HbC4eGiezvGE2LtnhUVB%3DcDeAbayw%40mail.gmail.com.
