Hi, After fighting a long time with this, we managed to get a solution that passes both the "Openblas_matrix_issue" and "BLAS_tester" test suites.
To solve the issue we had to apply a patch and add a new build parameter (USE_SIMPLE_THREADED_LEVEL3=1) to OpenBLAS to make it work with multiple openmp threads. This is how the buildopts line looks like for us: buildopts = ' USE_SIMPLE_THREADED_LEVEL3=1 BINARY=64 USE_THREAD=1 USE_OPENMP=1 CC="$CC" FC="$F77" DYNAMIC_ARCH=1' And the patch, we got it from this commit on the OpenBLAS repo: https://github.com/xianyi/OpenBLAS/commit/b14f44d2adbe1ec8ede0cdf06fb8b09f3c4b6e43 (you can get the patch by adding .patch at the end of the URL) Regards, Carlos On Mon, May 27, 2019 at 6:15 PM Pablo Escobar Lopez < [email protected]> wrote: > Hi, > > did anyone found a working patch or workaround for the matrix issue when > using OpenBLAS-0.3.1 ? > > After a lot of try&error I couldn't pass the tests in > https://github.com/eylenth/Openblas_matrix_issue when using > https://github.com/easybuilders/easybuild-easyconfigs/blob/master/easybuild/easyconfigs/o/OpenBLAS/OpenBLAS-0.3.1-GCC-7.3.0-2.30.eb > . > No matter what patches, toolchainopts or buildopts I use (and I have tried > few different combinations) . Is anyone able to pass the tests using > openblas-0.3.1 ? > > I could pass the tests using openblas-0.3.5 but upgrading my foss/2018b > toolchain would be quite messy because I use RPATH. The less intrusive > solution for my users would be to be able to patch openblas-0.3.1 somehow > but I couldn't find a working solution. Any suggestions? > > regards, > Pablo. > > p.s. in a related topic, IMHO unless there is a proper workaround I would > suggest to stop providing openblas-0.3.1 with easybuild. Right now we are > distributing a broken library > > > On Tue, May 7, 2019 at 6:34 PM Mikael Öhman <[email protected]> wrote: > >> Hi Thomas, >> >> I can also confirm these issues. I tried rebuilding OpenBLAS+R after the >> fix in #7180, but I still saw the same problems. >> Very large matrix-matrix multiplications randomly gave the wrong result. >> Very large errors. The larger the matrix, the more frequent the errors. >> >> In the end, I compiled an intel-version (but I had to remove a few >> extensions that didn't build) and removed my Foss version from our >> installations. >> >> Perhaps it's related to hardware; I saw this on happen skylake servers. I >> haven't had time to check if this >> https://github.com/easybuilders/easybuild-easyconfigs/issues/8197 >> also affects 0.3.1 >> >> Best regards, Mikael >> >> >> On Tue, May 7, 2019 at 6:12 PM Thomas Eylenbosch < >> [email protected]> wrote: >> >>> Hello >>> >>> >>> >>> Some of our end users reported a calculation issue with matrices when >>> they are working with a foss/2018b module >>> >>> >>> >>> I reproduced this error with Python and R that are compiled with the >>> foss/2018b toolchain, the output returns unexcepted results. >>> >>> Then I reproduced this error with Python and R that are compiled with >>> the foss/2016b toolchain , then it gives me the expected behavior. >>> >>> >>> >>> You can reproduce this error with the following github repository: >>> >>> https://github.com/eylenth/Openblas_matrix_issue >>> >>> >>> >>> I have also tried to recompile the OpenBLAS-0.3.1-GCC-7.3.0-2.30.eb >>> easyconfig file with “toolchainopts = {'vectorize': False}” ( cfr. >>> https://github.com/easybuilders/easybuild-easyconfigs/issues/7180) >>> >>> But is still giving me unexpected behavior >>> >>> >>> >>> Can someone try to reproduce the error with the R/Python(foss/2018b) >>> modules. Or can someone give me feedback on this? >>> >>> >>> >>> Thank you in advance. >>> >>> >>> >>> Met vriendelijke groet / Kind regards / Beste Grüße >>> >>> *Thomas Eylenbosch* >>> >>> *Ext: Gluo N.V.* >>> >>> >>> >>> BASF Agricultural Solutions Belgium NV >>> >>> Technologiepark 101 >>> >>> B-9052 Ghent (Zwijnaarde) >>> >>> BELGIUM >>> >>> E-mail: *[email protected] >>> <[email protected]>* >>> >>> [image: cid:[email protected]] >>> >>> BASF Agricultural Solutions Belgium NV, Registered Office: 9052 Gent, >>> Belgium >>> >>> Registration: RPR Gent: 0685.756.742 >>> >>> >>> >> > > -- > Pablo Escobar López > Linux/HPC systems engineer > sciCORE, University of Basel > SIB Swiss Institute of Bioinformatics >
