On 29/08/14 17:44, Trey Dockendorf wrote:
Kenneth,
Thanks for finding the cause. I was able to successfully build using your
suggestions.
How do I verify that the LAPACK functionality is included in OpenBLAS with or without the
Easyconfig provided LAPACK actually being applied? I saw your issue on OpenBLAS about
this [1] and noticed that when building OpenBLAS 0.2.10 (and 0.2.8) that there is no
file named "liblapack.*" present in the source after build or in the install
directory.
The LAPACK functionality is embedded in the libopenblas.a library.
You can check with 'nm' whether the LAPACK symbols are there:
nm <prefix>/softwareLAPACK/3.4.2-gompi-1.5.12-no-OFED/lib/liblapack.a |
grep zgetrf
U zgetrf_
U zgetrf_
zgetrf.o:
0000000000000000 T zgetrf_
U zgetrf_
nm
<prefix>/software/OpenBLAS/0.2.8-gompi-1.5.14-LAPACK-3.5.0/lib/libopenblas.a
| grep zgetrf
zgetrf.o:
0000000000000000 T zgetrf_
U zgetrf_parallel
U zgetrf_single
U zgetrf_
zgetrf_single.o:
0000000000000000 T zgetrf_single
zgetrf_parallel.o:
0000000000000bf0 T zgetrf_parallel
U zgetrf_single
U zgetrf_
U zgetrf_
lapacke_zgetrf.o:
0000000000000000 T LAPACKE_zgetrf
U LAPACKE_zgetrf_work
lapacke_zgetrf_work.o:
0000000000000000 T LAPACKE_zgetrf_work
U zgetrf_
K.
Thanks,
- Trey
[1]: https://github.com/xianyi/OpenBLAS/issues/203
=============================
Trey Dockendorf
Systems Analyst I
Texas A&M University
Academy for Advanced Telecommunications and Learning Technologies
Phone: (979)458-2396
Email: [email protected]
Jabber: [email protected]
----- Original Message -----
From: "Kenneth Hoste" <[email protected]>
To: [email protected]
Sent: Thursday, August 28, 2014 3:14:43 AM
Subject: Re: [easybuild] OpenBLAS with buildopts UTEST_CHECK=1
Hi Trey,
I can reproduce your problem on our end, thanks for the info.
From the CUnit configure script:
ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS
conftest.$ac_ext $LIBS >&5'
So, the "libopenblasp-r0.2.10.a" part is passed via $LIBS.
However, EasyBuild doesn't define $LIBS as such. It does define
$LIBS,
but differently:
== 2014-08-27 15:43:32,732 main.environment INFO Environment
variable LIBS set to -lm -lpthread
So, this is quite weird, but I've seen behavior similar to this when
$LIBS is involved.
My suspicion is that the OpenBLAS installation procedure somehow acts
differently when $LIBS is set from the outside, breaking the CUnit
configure by passing down 'libopenblasp-r0.2.10.a' that only makes
sense
for OpenBLAS.
You can resolve this problem by unsetting $LIBS before starting the
OpenBLAS procedure:
prebuildopts = "unset LIBS && "
I verified this, it works:
CUnit - A unit testing framework for C - Version 2.1-2
http://cunit.sourceforge.net/
Suite: Level1 Test Suite
Test: Testing srot when incx || incy == 0 ...passed
Test: Testing drot when incx || incy == 0 ...passed
Test: Testing csrot when incx || incy == 0 ...passed
Test: Testing zdrot when incx || incy == 0 ...passed
Test: Testing sswap with incx || incy == 0 ...passed
Test: Testing dswap with incx || incy == 0 ...passed
Test: Testing cswap with incx || incy == 0 ...passed
Test: Testing zswap with incx || incy == 0 ...passed
Test: Testing saxpy with incx || incy == 0 ...passed
Test: Testing daxpy with incx || incy == 0 ...passed
Test: Testing caxpy with incx || incy == 0 ...passed
Test: Testing zaxpy with incx || incy == 0 ...passed
Test: Testing zdotu with n == 1 ...passed
Test: Testing zdotu with input x & y offset == 1 ...passed
Test: Testing drotmg ...passed
Test: Testing dsdot with n == 1 ...passed
Test: Testing samax ...passed
Test: Testing fork safety ...passed
Run Summary: Type Total Ran Passed Failed Inactive
suites 1 1 n/a 0 0
tests 18 18 18 0 0
asserts 1000177 1000177 1000177 0 n/a
One thing I don't particularly like about this whole situation is
that
OpenBLAS auto-downloads the CUnit source tarball and then builds it
itself.
The proper way would be to install a separate module for CUnit, and
provide that as a (build) dependency to OpenBLAS...
Then this problem should not arise either to start with...
Another issue I noticed while browsing your OpenBLAS is that the
LAPACK
source that is being provided is not being used at all...
It's just using a 'lapack-netlib' that is shipped with the OpenBLAS
sources, as opposed to the one being provided.
I've opened
https://github.com/hpcugent/easybuild-easyconfigs/issues/1050 to
track
this one down.
regards,
Kenneth
On 27/08/14 23:31, Trey Dockendorf wrote:
The debug log was too big to send (15.6MB), resending compressed.
- Trey
=============================
Trey Dockendorf
Systems Analyst I
Texas A&M University
Academy for Advanced Telecommunications and Learning Technologies
Phone: (979)458-2396
Email: [email protected]
Jabber: [email protected]
----- Original Message -----
From: "Trey Dockendorf" <[email protected]>
To: [email protected]
Sent: Wednesday, August 27, 2014 4:21:22 PM
Subject: Re: [easybuild] OpenBLAS with buildopts UTEST_CHECK=1
I can't build 0.2.9, as I need the NO_AVX2 flag added in 0.2.10.
I
get the same failure with 0.2.8 which I've also successfully built
outside EB.
Attached is the log from running 'eb
OpenBLAS-0.2.10-GCC-4.8.3-LAPACK-3.5.0.eb --robot=. --force
--debug'
as well as the easyconfig.
Builds just fine without UTEST_CHECK=1, but be nice to be able to
use
the build-in OpenBLAS tests during build.
Thanks,
- Trey
=============================
Trey Dockendorf
Systems Analyst I
Texas A&M University
Academy for Advanced Telecommunications and Learning Technologies
Phone: (979)458-2396
Email: [email protected]
Jabber: [email protected]
----- Original Message -----
From: "Kenneth Hoste" <[email protected]>
To: [email protected]
Sent: Wednesday, August 27, 2014 2:53:15 PM
Subject: Re: [easybuild] OpenBLAS with buildopts UTEST_CHECK=1
On 27/08/14 21:45, Trey Dockendorf wrote:
Kenneth,
Thanks for response, below is output from config.log [1] under
build root's "utest/CUnit-2.1-2". Comparing with config.log
from
a successful build when run interactively [2] it seems that the
line "./libopenblasp-r0.2.10.a >&5" should not be there.
Hmm, weird, I have absolutely no clue where that is coming from.
Can you collect a full debug log with EasyBuild (use --debug),
and
send
it to us?
Please include your easyconfig file as well.
Are you running into similar issues with OpenBLAS 0.2.9 with the
same
build options?
I'm not aware of anyone having tried OpenBLAS 0.2.10 yet with EB,
but
I
know 0.2.9 does work (although without the UTEST_CHECK enabled).
regards,
Kenneth
[1] Failing under Easybuild
gcc version 4.8.3 (GCC)
configure:3379: $? = 0
configure:3368: gcc -V >&5
gcc: error: unrecognized command line option '-V'
gcc: fatal error: no input files
compilation terminated.
configure:3379: $? = 1
configure:3368: gcc -qversion >&5
gcc: error: unrecognized command line option '-qversion'
gcc: fatal error: no input files
compilation terminated.
configure:3379: $? = 1
configure:3399: checking whether the C compiler works
configure:3421: gcc -O2 -march=native -O2 -DUTEST_CHECK
-DSANITY_CHECK -DREFNAME=f_ -Wall -m64 -DF_INTERFACE_GFORT -fPIC
-DDYNAMIC_ARCH -DNO_AVX2 -DSMP_SERVER -DNO_WARMUP
-DMAX_CPU_NUMBER=64 -DASMNAME= -DASMFNAME=_ -DNAME=_ -DCNAME=
-DCHAR_NAME="_" -DCHAR_CN
AME="" -DNO_AFFINITY -I.
-I/dev/shm/OpenBLAS/0.2.10/GCC-4.8.3-LAPACK-3.5.0/OpenBLAS-0.2.10/utest/CUnit-2.1-2/include
-O2 -DUTEST_CHECK -DSANITY_CHECK -DREFNAME=libcunitf_ -Wall -m64
-DF_INTERFACE_GFORT -fPIC -DDYNAMIC_ARCH -DNO_AVX2 -DSMP_SERVER
-DNO_WARMUP
-DMAX_CPU_NUMBER=64 -DASMNAME=libcunit -DASMFNAME=libcunit_
-DNAME=libcunit_ -DCNAME=libcunit -DCHAR_NAME="libcunit_"
-DCHAR_CNAME="libcunit" -DNO_AFFINITY -I.. -DRELEASE=@RELEASE@
-Wall -W -pedantic -Wshadow -ansi
-I/dev/shm/OpenBLAS/0.2.10/GCC-4.8.3-LAPACK
-3.5.0/OpenBLAS-0.2.10/utest/CUnit-2.1-2/CUnit/Headers
-L/apps/easybuild/software/Core/GCC/4.8.3/lib64
-L/apps/easybuild/software/Core/GCC/4.8.3/lib
-L/dev/shm/OpenBLAS/0.2.10/GCC-4.8.3-LAPACK-3.5.0/OpenBLAS-0.2.10/utest/CUnit-2.1-2/CUnit/Sources
conftest.c
../libopenblasp-r0.2.10.a >&5
gcc: error: ../libopenblasp-r0.2.10.a: No such file or directory
configure:3425: $? = 1
configure:3463: result: no
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "CUnit"
| #define PACKAGE_TARNAME "cunit"
| #define PACKAGE_VERSION "2.1-2"
| #define PACKAGE_STRING "CUnit 2.1-2"
| #define PACKAGE_BUGREPORT ""
| #define PACKAGE_URL ""
| #define PACKAGE "CUnit"
| #define VERSION "2.1-2"
| /* end confdefs.h. */
|
| int
| main ()
| {
|
| ;
| return 0;
| }
configure:3468: error: in
`/dev/shm/OpenBLAS/0.2.10/GCC-4.8.3-LAPACK-3.5.0/OpenBLAS-0.2.10/utest/CUnit-2.1-2':
configure:3470: error: C compiler cannot create executables
See `config.log' for more details
[2] Succeeds when run manually:
gcc version 4.8.3 (GCC)
configure:3379: $? = 0
configure:3368: gcc -V >&5
gcc: error: unrecognized command line option '-V'
gcc: fatal error: no input files
compilation terminated.
configure:3379: $? = 1
configure:3368: gcc -qversion >&5
gcc: error: unrecognized command line option '-qversion'
gcc: fatal error: no input files
compilation terminated.
configure:3379: $? = 1
configure:3399: checking whether the C compiler works
configure:3421: gcc
-I/dev/shm/OpenBLAS/0.2.10/GCC-4.8.3-LAPACK-3.5.0/OpenBLAS-0.2.10/utest/CUnit-2.1-2/include
-O2 -DUTEST_CHECK -DSANITY_CHECK -DREFNAME=libcunitf_ -Wall -m64
-DF_INTERFACE_GFORT -fPIC -DDYNAMIC_ARCH -DNO_AVX2 -DSMP_SERVER
-DNO_WARMUP -DMAX_CPU_NUMBER=64 -DASMNAME=libcunit
-DASMFNAME=libcunit_ -DNAME=libcunit_ -DCNAME=libcunit
-DCHAR_NAME="libcunit_" -DCHAR_CNAME="libcunit" -DNO_AFFINITY
-I..
-DRELEASE=@RELEASE@ -Wall -W -pedantic -Wshadow -ansi
-I/dev/shm/OpenBLAS/0.2.10/GCC-4.8.3-LAPACK-3.5.0/OpenBLAS-0.2.10/utest/CUnit-2.1-2/CUnit/Headers
-L/dev/shm/OpenBLAS/0.2.10/GCC-4.8.3-LAPACK-3.5.0/OpenBLAS-0.2.10/utest/CUnit-2.1-2/CUnit/Sources
conftest.c >&5
configure:3425: $? = 0
configure:3473: result: yes
=============================
Trey Dockendorf
Systems Analyst I
Texas A&M University
Academy for Advanced Telecommunications and Learning
Technologies
Phone: (979)458-2396
Email: [email protected]
Jabber: [email protected]
----- Original Message -----
From: "Kenneth Hoste" <[email protected]>
To: [email protected]
Sent: Wednesday, August 27, 2014 2:31:25 PM
Subject: Re: [easybuild] OpenBLAS with buildopts UTEST_CHECK=1
Hi Trey,
On 27/08/14 21:25, Trey Dockendorf wrote:
I'm having an issue replicating in Easybuild what I can do
interactively when building OpenBLAS.
Manually running 'make BINARY=64 DYNAMIC_ARCH=1 USE_THREAD=1
NUM_THREADS=64 NO_AFFINITY=1 NO_AVX2=1 UTEST_CHECK=1' on the
source gives me no issue.
Using as a base the Easyconfig for
OpenBLAS-0.2.9-GCC-4.8.3-LAPACK-3.5.0 [1] where I only changed
the
following:
version = '0.2.10'
maxparallel = 1 # OpenBLAS will parallel build by default
AFAIK
buildopts = 'BINARY=64 DYNAMIC_ARCH=1 USE_THREAD=1
NUM_THREADS=64
NO_AFFINITY=1 NO_AVX2=1 UTEST_CHECK=1'
$ eb OpenBLAS-0.2.10-GCC-4.8.3-LAPACK-3.5.0.eb --robot=.
# From the log file when it fails
<snip>
checking for gcc... gcc
checking whether the C compiler works... no
configure: error: in
`/dev/shm/OpenBLAS/0.2.10/GCC-4.8.3-LAPACK-3.5.0/OpenBLAS-0.2.10/utest/CUnit-2.1-2':
configure: error: C compiler cannot create executables
See `config.log' for more details
Did you check this config.log file? The real reason will be in
there
somewhere (not at the very bottom though).
Look for the string "checking whether the C compiler works".
regards,
Kenneth
make[1]: ***
[/dev/shm/OpenBLAS/0.2.10/GCC-4.8.3-LAPACK-3.5.0/OpenBLAS-0.2.10/utest/CUnit-2.1-2/lib/libcunit.a]
Error 77
make[1]: Leaving directory
`/dev/shm/OpenBLAS/0.2.10/GCC-4.8.3-LAPACK-3.5.0/OpenBLAS-0.2.10/utest'
make: *** [tests] Error 2
== 2014-08-27 14:09:57,601 main.easyblock.<module> WARNING
build
failed (first 300 chars): EasyBuild crashed with an error (at
easybuild/tools/run.py:382 in parse_cmd_output): cmd " make -j
1
BINARY=64 DYNAMIC_ARCH=1 USE_THREAD=1 NUM_THREADS=64
NO_AFFINITY=1
NO_AVX2=1 UTEST_CHECK=1" exited with exitcode 2 and output:
Thanks,
- Trey
[1] :
https://github.com/hpcugent/easybuild-easyconfigs/blob/master/easybuild/easyconfigs/o/OpenBLAS/OpenBLAS-0.2.9-GCC-4.8.3-LAPACK-3.5.0.eb
=============================
Trey Dockendorf
Systems Analyst I
Texas A&M University
Academy for Advanced Telecommunications and Learning
Technologies
Phone: (979)458-2396
Email: [email protected]
Jabber: [email protected]
