Hi Pablo,
Thanks for the good news! For those of us who are not experts, could
you kindly describe the commands required to update our currently
installed OpenBLAS modules correctly? Are there any caveats with
updating an existing module in a toolchain (foss)?
We currently have these OpenBLAS modules installed:
$ ml av OpenBLAS/
-------------------------- /home/modules/modules/all
---------------------------
OpenBLAS/0.2.20-GCC-6.4.0-2.28 OpenBLAS/0.3.5-GCC-8.2.0-2.31.1 (D)
OpenBLAS/0.3.1-GCC-7.3.0-2.30
So is the update command simply this one?
eb --rebuild OpenBLAS-0.3.5-GCC-8.2.0-2.31.1.eb
OpenBLAS-0.3.1-GCC-7.3.0-2.30.eb OpenBLAS-0.2.20-GCC-6.4.0-2.28.eb
Is the --force flag also required?
I suppose that we do not have to rebuild the ScaLAPACK modules as well?
Thanks a lot,
Ole
On 5/28/19 5:14 PM, Pablo Escobar Lopez wrote:
thank you Carlos! You did a great job figuring out this fix :)
I can confirm that after applying this patch in our cluster the issue
seems to be solved for us. Now we pass these tests with
"OpenBLAS/0.3.1-GCC-7.3.0-2.30":
https://github.com/eylenth/Openblas_matrix_issue
https://github.com/xianyi/BLAS-Tester
I also got a confirmation from a colleague in our user support team that
a problem he was trying to debug with some R code is solved after this
fix was applied.
I have sent a PR with the fix upstream:
https://github.com/easybuilders/easybuild-easyconfigs/pull/8396
In case anyone else test the workaround it would be nice if you report
in the mailing list or in the pull request in github if it's working
fine for you too.
regards,
Pablo
On Tue, May 28, 2019 at 2:32 PM Carlos Fenoy <[email protected]
<mailto:[email protected]>> wrote:
Hi,
After fighting a long time with this, we managed to get a solution
that passes both the "Openblas_matrix_issue" and "BLAS_tester" test
suites.
To solve the issue we had to apply a patch and add a new build
parameter (USE_SIMPLE_THREADED_LEVEL3=1) to OpenBLAS to make it work
with multiple openmp threads.
This is how the buildopts line looks like for us:
buildopts = ' USE_SIMPLE_THREADED_LEVEL3=1 BINARY=64 USE_THREAD=1
USE_OPENMP=1 CC="$CC" FC="$F77" DYNAMIC_ARCH=1'
And the patch, we got it from this commit on the OpenBLAS repo:
https://github.com/xianyi/OpenBLAS/commit/b14f44d2adbe1ec8ede0cdf06fb8b09f3c4b6e43
(you
can get the patch by adding .patch at the end of the URL)
Regards,
Carlos
On Mon, May 27, 2019 at 6:15 PM Pablo Escobar Lopez
<[email protected] <mailto:[email protected]>>
wrote:
Hi,
did anyone found a working patch or workaround for the matrix
issue when using OpenBLAS-0.3.1 ?
After a lot of try&error I couldn't pass the tests in
https://github.com/eylenth/Openblas_matrix_issue when using
https://github.com/easybuilders/easybuild-easyconfigs/blob/master/easybuild/easyconfigs/o/OpenBLAS/OpenBLAS-0.3.1-GCC-7.3.0-2.30.eb
.
No matter what patches, toolchainopts or buildopts I use (and I
have tried few different combinations) . Is anyone able to pass
the tests using openblas-0.3.1 ?
I could pass the tests using openblas-0.3.5 but upgrading my
foss/2018b toolchain would be quite messy because I use RPATH.
The less intrusive solution for my users would be to be able to
patch openblas-0.3.1 somehow but I couldn't find a working
solution. Any suggestions?
regards,
Pablo.
p.s. in a related topic, IMHO unless there is a proper
workaround I would suggest to stop providing openblas-0.3.1 with
easybuild. Right now we are distributing a broken library
On Tue, May 7, 2019 at 6:34 PM Mikael Öhman <[email protected]
<mailto:[email protected]>> wrote:
Hi Thomas,
I can also confirm these issues. I tried rebuilding
OpenBLAS+R after the fix in #7180, but I still saw the same
problems.
Very large matrix-matrix multiplications randomly gave the
wrong result. Very large errors. The larger the matrix, the
more frequent the errors.
In the end, I compiled an intel-version (but I had to remove
a few extensions that didn't build) and removed my Foss
version from our installations.
Perhaps it's related to hardware; I saw this on happen
skylake servers. I haven't had time to check if this
https://github.com/easybuilders/easybuild-easyconfigs/issues/8197
also affects 0.3.1
Best regards, Mikael
On Tue, May 7, 2019 at 6:12 PM Thomas Eylenbosch
<[email protected]
<mailto:[email protected]>> wrote:
Hello____
__ __
Some of our end users reported a calculation issue with
matrices when they are working with a foss/2018b module____
__ __
I reproduced this error with Python and R that are
compiled with the foss/2018b toolchain, the output
returns unexcepted results. ____
Then I reproduced this error with Python and R that are
compiled with the foss/2016b toolchain , then it gives
me the expected behavior. ____
__ __
You can reproduce this error with the following github
repository: ____
https://github.com/eylenth/Openblas_matrix_issue____
__ __
I have also tried to recompile the
OpenBLAS-0.3.1-GCC-7.3.0-2.30.eb easyconfig file with
“toolchainopts = {'vectorize': False}” ( cfr.
https://github.com/easybuilders/easybuild-easyconfigs/issues/7180)____
But is still giving me unexpected behavior____
__ __
Can someone try to reproduce the error with the
R/Python(foss/2018b) modules. Or can someone give me
feedback on this?____
__ __
Thank you in advance. ____
__ __
Met vriendelijke groet / Kind regards / Beste Grüße*____*
*Thomas Eylenbosch____*
/Ext: Gluo N.V.//____/
__ __
BASF Agricultural Solutions Belgium NV____
Technologiepark 101____
B-9052 Ghent (Zwijnaarde)____
BELGIUM____
E-mail: *[email protected]
<mailto:[email protected]>*____
cid:[email protected]____
BASF Agricultural Solutions Belgium NV, Registered
Office: 9052 Gent, Belgium____
Registration: RPR Gent: 0685.756.742____
__ __
--
Pablo Escobar López
Linux/HPC systems engineer
sciCORE, University of Basel
SIB Swiss Institute of Bioinformatics
--
Pablo Escobar López
Linux/HPC systems engineer
sciCORE, University of Basel
SIB Swiss Institute of Bioinformatics
