It is not the first time I have this problem and my aim was now to try to solve
it instead of ignoring tests. The environment seams coherent (see below).
I'll try to run in debug mode to investigate where the code hangs.
Patrick
[begou@kareline tutorials]$ make ex19
*mpicc* -o ex19.o -c -Wall -Wwrite-strings -Wno-strict-aliasing
-Wno-unknown-pragmas -fvisibility=hidden -g3
-I/kareline/data/begou/YALES2_1.0.0/PREREQUIS/petsc-git/include
-I/kareline/data/begou/YALES2_1.0.0/PREREQUIS/petsc-git/GCC48/include `pwd`/ex19.c
mpicc -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas
-fvisibility=hidden -g3 -o ex19 ex19.o
-L/kareline/data/begou/YALES2_1.0.0/PREREQUIS/petsc-git/GCC48/lib -lpetsc
-llapack -lblas -lX11 -lhwloc -lssl -lcrypto -L/opt/openmpi173-GCC48-node/lib
-L/opt/GCC48c/lib/gcc/x86_64-unknown-linux-gnu/4.8.1 -L/opt/GCC48c/lib64
-L/opt/GCC48c/lib -lmpi_usempi -lmpi_mpifh -lgfortran -lm -lgfortran -lm
-lquadmath -lm -lmpi_cxx -lstdc++ -L/opt/openmpi173-GCC48-node/lib
-L/opt/GCC48c/lib/gcc/x86_64-unknown-linux-gnu/4.8.1 -L/opt/GCC48c/lib64
-L/opt/GCC48c/lib -ldl -lmpi -lgcc_s -lpthread -ldl
/bin/rm -f ex19.o
[begou@kareline tutorials]$ *which mpiexec**
**/opt/openmpi173-GCC48-node/bin/mpiexec*
[begou@kareline tutorials]$ *mpicc --showme**
**gcc -I/opt/openmpi173-GCC48-node/include -pthread
-L/opt/openmpi173-GCC48-node/lib -lmpi*
[begou@kareline tutorials]$ *ldd ./ex19*
linux-vdso.so.1 => (0x00007ffe771ea000)
liblapack.so.3 => /usr/lib64/atlas/liblapack.so.3 (0x00007f5ac8596000)
libblas.so.3 => /usr/lib64/libblas.so.3 (0x00007f5ac833e000)
libX11.so.6 => /usr/lib64/libX11.so.6 (0x0000003c66600000)
libhwloc.so.5 => /usr/lib64/libhwloc.so.5 (0x0000003dde600000)
libssl.so.10 => /usr/lib64/libssl.so.10 (0x0000003c71600000)
libcrypto.so.10 => /usr/lib64/libcrypto.so.10 (0x0000003c69a00000)
libmpi_usempi.so.1 => */opt/openmpi173-GCC48-node/lib/libmpi_usempi.so.1*
(0x00007f5ac80a4000)
libmpi_mpifh.so.2 => */opt/openmpi173-GCC48-node/lib/libmpi_mpifh.so.2*
(0x00007f5ac7e5a000)
libgfortran.so.3 => /opt/GCC48c/lib64/libgfortran.so.3 (0x00007f5ac7b43000)
libm.so.6 => /lib64/libm.so.6 (0x0000003c63200000)
libquadmath.so.0 => /opt/GCC48c/lib64/libquadmath.so.0 (0x00007f5ac7907000)
libmpi_cxx.so.1 => */opt/openmpi173-GCC48-node/lib/libmpi_cxx.so.1*
(0x00007f5ac76ed000)
libstdc++.so.6 => /opt/GCC48c/lib64/libstdc++.so.6 (0x00007f5ac73e4000)
libdl.so.2 => /lib64/libdl.so.2 (0x0000003c63a00000)
libmpi.so.1 => */opt/openmpi173-GCC48-node/lib/libmpi.so.1*
(0x00007f5ac7115000)
libgcc_s.so.1 => /opt/GCC48c/lib64/libgcc_s.so.1 (0x00007f5ac6eff000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003c63600000)
libc.so.6 => /lib64/libc.so.6 (0x0000003c62e00000)
libf77blas.so.3 => /usr/lib64/atlas/libf77blas.so.3 (0x00007f5ac6cdf000)
libcblas.so.3 => /usr/lib64/atlas/libcblas.so.3 (0x00007f5ac6abe000)
libxcb.so.1 => /usr/lib64/libxcb.so.1 (0x0000003c66200000)
libnuma.so.1 => /usr/lib64/libnuma.so.1 (0x0000003ddee00000)
libpci.so.3 => /lib64/libpci.so.3 (0x0000003ddea00000)
libxml2.so.2 => /usr/lib64/libxml2.so.2 (0x0000003c6e200000)
libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x0000003c70a00000)
libkrb5.so.3 => /lib64/libkrb5.so.3 (0x0000003c70e00000)
libcom_err.so.2 => /lib64/libcom_err.so.2 (0x0000003c69e00000)
libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x0000003c6f200000)
libz.so.1 => /lib64/libz.so.1 (0x0000003c64200000)
libopen-rte.so.6 => */opt/openmpi173-GCC48-node/lib/libopen-rte.so.6*
(0x00007f5ac684a000)
libopen-pal.so.6 => */opt/openmpi173-GCC48-node/lib/libopen-pal.so.6*
(0x00007f5ac6575000)
librt.so.1 => /lib64/librt.so.1 (0x0000003c63e00000)
libnsl.so.1 => /lib64/libnsl.so.1 (0x0000003c71a00000)
libutil.so.1 => /lib64/libutil.so.1 (0x0000003c6ba00000)
/lib64/ld-linux-x86-64.so.2 (0x0000003c62a00000)
libatlas.so.3 => /usr/lib64/atlas/libatlas.so.3 (0x00007f5ac5f18000)
libXau.so.6 => /usr/lib64/libXau.so.6 (0x0000003c66a00000)
libresolv.so.2 => /lib64/libresolv.so.2 (0x0000003c64e00000)
libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x0000003c6fe00000)
libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x0000003c6f600000)
libselinux.so.1 => /lib64/libselinux.so.1 (0x0000003c64600000)
Matthew Knepley a écrit :
On Fri, Jan 6, 2017 at 2:39 AM, Patrick Begou
<[email protected]
<mailto:[email protected]>> wrote:
Hi Matthew,
Launching manualy ex19 shows only one process consuming cpu time, after
952mn I've killed the job this morning.
[begou@kareline tutorials]$ make ex19
mpicc -o ex19.o -c -Wall -Wwrite-strings -Wno-strict-aliasing
-Wno-unknown-pragmas -fvisibility=hidden -g3
-I/kareline/data/begou/YALES2_1.0.0/PREREQUIS/petsc-git/include
-I/kareline/data/begou/YALES2_1.0.0/PREREQUIS/petsc-git/GCC48/include
`pwd`/ex19.c
mpicc -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas
-fvisibility=hidden -g3 -o ex19 ex19.o
-L/kareline/data/begou/YALES2_1.0.0/PREREQUIS/petsc-git/GCC48/lib -lpetsc
-llapack -lblas -lX11 -lhwloc -lssl -lcrypto
-L/opt/openmpi173-GCC48-node/lib
-L/opt/GCC48c/lib/gcc/x86_64-unknown-linux-gnu/4.8.1 -L/opt/GCC48c/lib64
-L/opt/GCC48c/lib -lmpi_usempi -lmpi_mpifh -lgfortran -lm -lgfortran -lm
-lquadmath -lm -lmpi_cxx -lstdc++ -L/opt/openmpi173-GCC48-node/lib
-L/opt/GCC48c/lib/gcc/x86_64-unknown-linux-gnu/4.8.1 -L/opt/GCC48c/lib64
-L/opt/GCC48c/lib -ldl -lmpi -lgcc_s -lpthread -ldl
/bin/rm -f ex19.o
[begou@kareline tutorials]$ mpiexec -n 2 ./ex19 -snes_monitor
top command shows:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
32184 begou 20 0 249m 7152 5132 R 99.8 0.0 952:15.97 ex19
32183 begou 20 0 71676 3508 2264 S 0.0 0.0 0:00.04 mpiexec
32185 begou 20 0 185m 7132 5124 S 0.0 0.0 0:00.04 ex19
looks like the first process waiting for something that never occur in MPI
communication....
1000s of people run this every day, so I am skeptical of that explanation.
However, this could happen if the 'mpiexec' in your path does not match the
MPI libraries that PETSc is linked to.
Matt
Patrick
Matthew Knepley a écrit :
On Thu, Jan 5, 2017 at 6:31 AM, Patrick Begou
<[email protected]
<mailto:[email protected]>> wrote:
I am unable to run any test on petsc. It looks like if the ex19 run
freeze on the server as it do not use any cpu time and pstree shows
sshd---bash-+-gedit
`-make---sh-+-gmake---sh---gmake---sh---mpiexec---ex19
`-tee
I've tested petsc-3.7.5.tar.gz and the latest sources on the Git
repository.
All make is doing is running ex19, which you can do by hand. What do you
get for
cd $PETSC_DIR
cd src/snes/examples/tutorials
make ex19
mpiexec -n 2 ./ex19 -snes_monitor
Thanks,
Matt
Setup from the Git repo:
./configure
--prefix=/kareline/data/begou/YALES2_1.0.0/PREREQUIS/petsc-git-binaries
\
--PETSC_ARCH=GCC48 \
--PETSC_DIR=/kareline/data/begou/YALES2_1.0.0/PREREQUIS/petsc-git \
--with-shared-libraries=0 \
--with-fortran-interfaces=1 \
--with-fortran-kernels=1 \
--with-cc=mpicc \
--with-fc=mpif90 \
--with-cxx=mpicxx
make PETSC_DIR=/kareline/data/begou/YALES2_1.0.0/PREREQUIS/petsc-git
PETSC_ARCH=GCC48 all
make PETSC_DIR=/kareline/data/begou/YALES2_1.0.0/PREREQUIS/petsc-git
PETSC_ARCH=GCC48 install
make
PETSC_DIR=/kareline/data/begou/YALES2_1.0.0/PREREQUIS/petsc-git-binaries
PETSC_ARCH="" test
In the log file I've just:
Running test examples to verify correct installation
Using
PETSC_DIR=/kareline/data/begou/YALES2_1.0.0/PREREQUIS/petsc-git-binaries
and PETSC_ARCH=
I'm using:
gcc version 4.8.1
Open MPI: 1.7.3 (build with gcc 4.8.1)
(This environment is in production for a while for many local
software and works fine)
Any suggestion is welcome
Patrick
--
===================================================================
| Equipe M.O.S.T. | |
| Patrick BEGOU | mailto:[email protected]
<mailto:[email protected]> |
| LEGI | |
| BP 53 X | Tel 04 76 82 51 35 |
| 38041 GRENOBLE CEDEX | Fax 04 76 82 52 71 |
===================================================================
--
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which
their experiments lead.
-- Norbert Wiener
--
===================================================================
| Equipe M.O.S.T. | |
| Patrick BEGOU |mailto:[email protected]
<mailto:[email protected]> |
| LEGI | |
| BP 53 X | Tel 04 76 82 51 35 |
| 38041 GRENOBLE CEDEX | Fax 04 76 82 52 71 |
===================================================================
--
What most experimenters take for granted before they begin their experiments
is infinitely more interesting than any results to which their experiments
lead. -- Norbert Wiener
--
===================================================================
| Equipe M.O.S.T. | |
| Patrick BEGOU | mailto:[email protected] |
| LEGI | |
| BP 53 X | Tel 04 76 82 51 35 |
| 38041 GRENOBLE CEDEX | Fax 04 76 82 52 71 |
===================================================================
