Shouldn't I see a mixture of MPI-libraries when calling ldd if I mix versions of MPI?
I also removed all calls to armadillo, and accordingly all references to it in the compilation, but the behavior is still unchanged. Regards, Roland Am 13.10.21 um 12:36 schrieb Matthew Knepley: > On Wed, Oct 13, 2021 at 6:32 AM Roland Richter > <[email protected]> wrote: > > Yes, the first part (which works) consists out of a compilation > line and a linking line, while the second command is a combination > of compile- and linking line. > > The link line in the first does not tell us anything because MPI is > not even present. It is being pulled in I presume from libarmadillo, > which we cannot see. It still > seems most likely, as Stefano said, that you are mixing versions of MPI. > > Thanks, > > Matt > > > Am 13.10.21 um 12:26 schrieb Matthew Knepley: >> On Wed, Oct 13, 2021 at 5:53 AM Roland Richter >> <[email protected]> wrote: >> >> Hei, >> >> I noticed a difference in when the program is running, and >> when not. The code works fine if I compile it via a >> CMake-file and load PETSc there. If I use the compilation >> line which is included in the Makefiles, then the code will >> fail with the mentioned error. The cmake-generated >> compilation line (including armadillo, because my test sample >> contained armadillo-code) is >> >> One of these is a compile command and the other is a link command. >> >> Matt >> >> //opt/intel/oneapi/mpi/2021.4.0/bin/mpicxx -D__INSDIR__="" >> -I/include -I/opt/petsc/include -I/opt/armadillo/include >> -std=c++0x -g -MD -MT CMakeFiles/main.dir/source/main.cpp.o >> -MF CMakeFiles/main.dir/source/main.cpp.o.d -o >> CMakeFiles/main.dir/source/main.cpp.o -c source/main.cpp// >> ///opt/intel/oneapi/mpi/2021.4.0/bin/mpicxx -rdynamic >> CMakeFiles/main.dir/source/main.cpp.o -o main_short >> -Wl,-rpath,/opt/petsc/lib:/opt/armadillo/lib64 >> /opt/petsc/lib/libpetsc.so /opt/armadillo/lib64/libarmadillo.so / >> >> Meanwhile, the original compilation line from PETSc is >> >> /mpicxx -mavx2 -march=native -O3 -fPIC -fopenmp >> -I/opt/petsc/include -I/opt/armadillo/include >> -I/opt/intel/oneapi/mkl/latest/include -I/opt/fftw3/include >> -I/opt/hdf5/include -I/opt/boost/include source/main.cpp >> -Wl,-rpath,/opt/petsc/lib -L/opt/petsc/lib >> -Wl,-rpath,/opt/petsc/lib -L/opt/petsc/lib >> -L/opt/intel/oneapi/mkl/latest/lib/intel64 >> -Wl,-rpath,/opt/fftw3/lib64 -L/opt/fftw3/lib64 >> -Wl,-rpath,/opt/armadillo/lib64 -L/opt/armadillo/lib64 >> -Wl,-rpath,/opt/intel/oneapi/mkl/latest/lib/intel64 >> -Wl,-rpath,/opt/hdf5/lib -L/opt/hdf5/lib >> -Wl,-rpath,/opt/intel/oneapi/mpi/2021.4.0/lib/release >> -L/opt/intel/oneapi/mpi/2021.4.0/lib/release >> -Wl,-rpath,/opt/intel/oneapi/mpi/2021.4.0/lib >> -L/opt/intel/oneapi/mpi/2021.4.0/lib >> -Wl,-rpath,/usr/lib64/gcc/x86_64-suse-linux/11 >> -L/usr/lib64/gcc/x86_64-suse-linux/11 >> -Wl,-rpath,/opt/intel/oneapi/vpl/2021.6.0/lib >> -L/opt/intel/oneapi/vpl/2021.6.0/lib >> -Wl,-rpath,/opt/intel/oneapi/tbb/2021.4.0/lib/intel64/gcc4.8 >> -L/opt/intel/oneapi/tbb/2021.4.0/lib/intel64/gcc4.8 >> -Wl,-rpath,/opt/intel/oneapi/mpi/2021.4.0/libfabric/lib >> -L/opt/intel/oneapi/mpi/2021.4.0/libfabric/lib >> -Wl,-rpath,/opt/intel/oneapi/mkl/2021.4.0/lib/intel64 >> -L/opt/intel/oneapi/mkl/2021.4.0/lib/intel64 >> -Wl,-rpath,/opt/intel/oneapi/ipp/2021.4.0/lib/intel64 >> -L/opt/intel/oneapi/ipp/2021.4.0/lib/intel64 >> -Wl,-rpath,/opt/intel/oneapi/ippcp/2021.4.0/lib/intel64 >> -L/opt/intel/oneapi/ippcp/2021.4.0/lib/intel64 >> -Wl,-rpath,/opt/intel/oneapi/dnnl/2021.4.0/cpu_dpcpp_gpu_dpcpp/lib >> -L/opt/intel/oneapi/dnnl/2021.4.0/cpu_dpcpp_gpu_dpcpp/lib >> -Wl,-rpath,/opt/intel/oneapi/dal/2021.4.0/lib/intel64 >> -L/opt/intel/oneapi/dal/2021.4.0/lib/intel64 >> >> -Wl,-rpath,/opt/intel/oneapi/compiler/2021.4.0/linux/compiler/lib/intel64_lin >> -L/opt/intel/oneapi/compiler/2021.4.0/linux/compiler/lib/intel64_lin >> -Wl,-rpath,/opt/intel/oneapi/compiler/2021.4.0/linux/lib >> -L/opt/intel/oneapi/compiler/2021.4.0/linux/lib >> -Wl,-rpath,/opt/intel/oneapi/clck/2021.4.0/lib/intel64 >> -L/opt/intel/oneapi/clck/2021.4.0/lib/intel64 >> -Wl,-rpath,/opt/intel/oneapi/ccl/2021.4.0/lib/cpu_gpu_dpcpp >> -L/opt/intel/oneapi/ccl/2021.4.0/lib/cpu_gpu_dpcpp >> -Wl,-rpath,/usr/x86_64-suse-linux/lib >> -L/usr/x86_64-suse-linux/lib -larmadillo -lpetsc -lHYPRE >> -lcmumps -ldmumps -lsmumps -lzmumps -lmumps_common -lpord >> -lmkl_scalapack_lp64 -lmkl_blacs_intelmpi_lp64 -lspqr >> -lumfpack -lklu -lcholmod -lbtf -lccolamd -lcolamd -lcamd >> -lamd -lsuitesparseconfig -lsuperlu -lsuperlu_dist -lEl >> -lElSuiteSparse -lpmrrr -lfftw3_mpi -lfftw3 -lp4est -lsc >> -lmkl_intel_lp64 -lmkl_core -lmkl_intel_thread -liomp5 -ldl >> -lpthread -lptesmumps -lptscotchparmetis -lptscotch >> -lptscotcherr -lesmumps -lscotch -lscotcherr -lhdf5_hl -lhdf5 >> -lparmetis -lmetis -lm -lz -lmuparser -lX11 -lstdc++ -ldl >> -lmpifort -lmpi -lrt -lpthread -lgfortran -lm -lgfortran -lm >> -lgcc_s -lquadmath -lrt -lquadmath -lstdc++ -ldl -o main_long/ >> >> Both executables have the same libraries linked to them, but >> in a different order when comparing them with ldd. >> >> Does that explain the observed behavior? >> >> Thanks, >> >> regards, >> >> Roland >> >> Am 11.10.21 um 15:13 schrieb Roland Richter: >>> >>> Hei, >>> >>> the following code works fine: >>> >>> #include <iostream> >>> #include <petsc.h> >>> >>> static char help[] = "Solves 2D Poisson equation using >>> multigrid.\n\n"; >>> int main(int argc,char **argv) { >>> PetscInitialize(&argc,&argv,(char*)0,help); >>> std::cout << "Hello World\n"; >>> PetscFinalize(); >>> return 0; >>> } >>> >>> Regards, >>> >>> Roland >>> >>> Am 11.10.21 um 14:34 schrieb Stefano Zampini: >>>> Can you try with a simple call that only calls >>>> PetscInitialize/Finalize? >>>> >>>> >>>>> On Oct 11, 2021, at 3:30 PM, Roland Richter >>>>> <[email protected]> wrote: >>>>> >>>>> At least according to configure.log mpiexec was defined as >>>>> >>>>> Checking for program >>>>> /opt/intel/oneapi/mpi/2021.4.0//bin/mpiexec...found >>>>> Defined make macro "MPIEXECEXECUTABLE" >>>>> to "/opt/intel/oneapi/mpi/2021.4.0/bin/mpiexec" >>>>> >>>>> When running ex19 with this mpiexec it fails with the >>>>> usual error, even though all configuration steps worked >>>>> fine. I attached the configuration log. >>>>> >>>>> Regards, >>>>> >>>>> Roland >>>>> >>>>> Am 11.10.21 um 14:24 schrieb Stefano Zampini: >>>>>> You are most probably using a different mpiexec then the >>>>>> one used to compile petsc. >>>>>> >>>>>> >>>>>> >>>>>>> On Oct 11, 2021, at 3:23 PM, Roland Richter >>>>>>> <[email protected]> wrote: >>>>>>> >>>>>>> I tried either /./ex19/ (SNES-example), /mpirun ./ex19/ >>>>>>> or /mpirun -n 1 ./ex19/, all with the same result. >>>>>>> >>>>>>> Regards, >>>>>>> >>>>>>> Roland >>>>>>> >>>>>>> Am 11.10.21 um 14:22 schrieb Matthew Knepley: >>>>>>>> On Mon, Oct 11, 2021 at 8:07 AM Roland Richter >>>>>>>> <[email protected]> wrote: >>>>>>>> >>>>>>>> Hei, >>>>>>>> >>>>>>>> at least in gdb it fails with >>>>>>>> >>>>>>>> Attempting to use an MPI routine before >>>>>>>> initializing MPICH >>>>>>>> [Inferior 1 (process 7854) exited with code 01] >>>>>>>> (gdb) backtrace >>>>>>>> No stack. >>>>>>>> >>>>>>>> >>>>>>>> What were you running? If it never makes it into PETSc >>>>>>>> code, I am not sure what we are >>>>>>>> doing to cause this. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Matt >>>>>>>> >>>>>>>> >>>>>>>> Regards, >>>>>>>> >>>>>>>> Roland >>>>>>>> >>>>>>>> Am 11.10.21 um 13:57 schrieb Matthew Knepley: >>>>>>>>> On Mon, Oct 11, 2021 at 5:24 AM Roland Richter >>>>>>>>> <[email protected]> wrote: >>>>>>>>> >>>>>>>>> Hei, >>>>>>>>> >>>>>>>>> I compiled PETSc with Intel MPI (MPICH) and >>>>>>>>> GCC as compiler (i.e. using >>>>>>>>> Intel OneAPI together with the supplied >>>>>>>>> mpicxx-compiler). Compilation >>>>>>>>> and installation worked fine, but running the >>>>>>>>> tests resulted in the >>>>>>>>> error "Attempting to use an MPI routine before >>>>>>>>> initializing MPICH". A >>>>>>>>> simple test program (attached) worked fine >>>>>>>>> with the same combination. >>>>>>>>> >>>>>>>>> What could be the reason for that? >>>>>>>>> >>>>>>>>> >>>>>>>>> Hi Roland, >>>>>>>>> >>>>>>>>> Can you get a stack trace for this error using the >>>>>>>>> debugger? >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Matt >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks! >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> >>>>>>>>> Roland Richter >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> What most experimenters take for granted before >>>>>>>>> they begin their experiments is infinitely more >>>>>>>>> interesting than any results to which their >>>>>>>>> experiments lead. >>>>>>>>> -- Norbert Wiener >>>>>>>>> >>>>>>>>> https://www.cse.buffalo.edu/~knepley/ >>>>>>>>> <http://www.cse.buffalo.edu/~knepley/> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> What most experimenters take for granted before they >>>>>>>> begin their experiments is infinitely more interesting >>>>>>>> than any results to which their experiments lead. >>>>>>>> -- Norbert Wiener >>>>>>>> >>>>>>>> https://www.cse.buffalo.edu/~knepley/ >>>>>>>> <http://www.cse.buffalo.edu/~knepley/> >>>>>> >>>>> <configure.log> >>>> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to >> which their experiments lead. >> -- Norbert Wiener >> >> https://www.cse.buffalo.edu/~knepley/ >> <http://www.cse.buffalo.edu/~knepley/> > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > <http://www.cse.buffalo.edu/~knepley/>
