I would suggest installing regular 32bit int blas/lapack - And then using it with --with-blaslapack-lib option
[we don't know what -fdefault-integer-8 does with --download-fblaslapack - if it really creates --known-64-bit-blas-indices variant of blas/lapack or not] Satish On Tue, 21 Jan 2020, Smith, Barry F. via petsc-users wrote: > > I would avoid OpenBLAS it just introduces one new variable that could > introduce problems. > > PetscErrorCode is ALWAYS 32 bit, PetscInt becomes 64 bit with > --with-64-bit-indices, PETScMPIInt is ALWAYS 32 bit, PetscBLASInt is > usually 32 bit unless you build with a special BLAS that supports 64 bit > indices. > > In theory the ex5f should be fine, we test it all the time with all > possible values of the integer. Please redo the ./configure with > --with-64-bit-indices --download-fblaslapack and send the configure.log this > provides the most useful information on the decisions configure has made. > > Barry > > > > On Jan 21, 2020, at 4:28 AM, Дмитрий Мельничук > > <[email protected]> wrote: > > > > > First you need to figure out what is triggering: > > > > > C:/MPI/Bin/mpiexec.exe: error while loading shared libraries: ?: cannot > > > open shared object file: No such file or directory > > > > > Googling it finds all kinds of suggestions for Linux. But Windows? Maybe > > > the debugger will help. > > > > > Second > > > VecNorm_Seq line 221 > > > /cygdrive/d/Computational_geomechanics/installation/petsc-barry/src/vec/vec/impls/seq/bvec2.c > > > > > > > Debugger is best to find out what is triggering this. Since it is the C > > > side of things it would be odd that the Fortran change affects it. > > > > > Barry > > > > > > I am in the process of finding out the causes of these errors. > > > > I'm inclined to the fact that BLAS has still some influence on what is > > happening. > > Because testing of 32-bit version of PETSc gives such weird error with > > mpiexec.exe, but Fortran example ex5f completes succeccfully. > > > > I need to say that my solver compiled with 64-bit version of PETSc failed > > with Segmentation Violation error (the same as ex5f) when calling > > KSPSolve(Krylov,Vec_F,Vec_U,ierr). > > During the execution KSPSolve appeals to VecNorm_Seq in bvec2.c. Also > > VecNorm_Seq uses several types of integer: PetscErrorCode, PetscInt, > > PetscBLASInt. > > I suspect that PetscBLASInt may conflict with PetscInt. > > Also I noted that execution of KSPSolve() does not even start , so > > arguments (Krylov,Vec_F,Vec_U,ierr) cannot be passed to KSPSolve(). > > (inserted fprint() in the top of KSPSolve and saw no output) > > > > > > So I tried to configure PETSc with --download-fblaslapack > > --with-64-bit-blas-indices, but got an error that > > > > fblaslapack does not support -with-64-bit-blas-indices > > > > Switching to flags --download-openblas -with-64-bit-blas-indices was > > unsuccessfully too because of error: > > > > Error during download/extract/detection of OPENBLAS: > > Unable to download openblas > > Could not execute "['git clone https://github.com/xianyi/OpenBLAS.git > > /cygdrive/d/Computational_geomechanics/installation/petsc-barry/arch-mswin-c-debug/externalpackages/git.openblas']": > > fatal: destination path > > '/cygdrive/d/Computational_geomechanics/installation/petsc-barry/arch-mswin-c-debug/externalpackages/git.openblas' > > already exists and is not an empty directory. > > Unable to download package OPENBLAS from: > > git://https://github.com/xianyi/OpenBLAS.git > > * If URL specified manually - perhaps there is a typo? > > * If your network is disconnected - please reconnect and rerun ./configure > > * Or perhaps you have a firewall blocking the download > > * You can run with --with-packages-download-dir=/adirectory and ./configure > > will instruct you what packages to download manually > > * or you can download the above URL manually, to /yourselectedlocation > > and use the configure option: > > --download-openblas=/yourselectedlocation > > Unable to download openblas > > Could not execute "['git clone https://github.com/xianyi/OpenBLAS.git > > /cygdrive/d/Computational_geomechanics/installation/petsc-barry/arch-mswin-c-debug/externalpackages/git.openblas']": > > fatal: destination path > > '/cygdrive/d/Computational_geomechanics/installation/petsc-barry/arch-mswin-c-debug/externalpackages/git.openblas' > > already exists and is not an empty directory. > > Unable to download package OPENBLAS from: > > git://https://github.com/xianyi/OpenBLAS.git > > * If URL specified manually - perhaps there is a typo? > > * If your network is disconnected - please reconnect and rerun ./configure > > * Or perhaps you have a firewall blocking the download > > * You can run with --with-packages-download-dir=/adirectory and ./configure > > will instruct you what packages to download manually > > * or you can download the above URL manually, to /yourselectedlocation > > and use the configure option: > > --download-openblas=/yourselectedlocation > > Could not locate downloaded package OPENBLAS in > > /cygdrive/d/Computational_geomechanics/installation/petsc-barry/arch-mswin-c-debug/externalpackages > > > > But I checked the last location (.../externalpackages) and saw that > > OpenBLAS downloaded and unzipped. > > > > > > > > Kind regards, > > Dmitry Melnichuk > > > > > > 20.01.2020, 16:32, "Smith, Barry F." <[email protected]>: > > > > First you need to figure out what is triggering: > > > > C:/MPI/Bin/mpiexec.exe: error while loading shared libraries: ?: cannot > > open shared object file: No such file or directory > > > > Googling it finds all kinds of suggestions for Linux. But Windows? Maybe > > the debugger will help. > > > > Second > > > > > > > > VecNorm_Seq line 221 > > /cygdrive/d/Computational_geomechanics/installation/petsc-barry/src/vec/vec/impls/seq/bvec2.c > > > > Debugger is best to find out what is triggering this. Since it is the C > > side of things it would be odd that the Fortran change affects it. > > > > Barry > > > > > > > > > > > > > > > > On Jan 20, 2020, at 4:43 AM, Дмитрий Мельничук > > <[email protected]> wrote: > > > > Thank you so much for your assistance! > > > > As far as I have been able to find out, the errors "Type mismatch in > > argument ‘ierr’" have been successfully fixed. > > But execution of command "make PETSC_DIR=/cygdrive/d/... > > PETSC_ARCH=arch-mswin-c-debug check" leads to the appereance of > > Segmantation Violation error. > > > > I compiled PETSc with Microsoft MPI v10. > > Does it make sense to compile PETSc with another MPI implementation (such > > as MPICH) in order to resolve the issue? > > > > Error message: > > Running test examples to verify correct installation > > Using > > PETSC_DIR=/cygdrive/d/Computational_geomechanics/installation/petsc-barry > > and PETSC_ARCH=arch-mswin-c-debug > > Possible error running C/C++ src/snes/examples/tutorials/ex19 with 1 MPI > > process > > See http://www.mcs.anl.gov/petsc/documentation/faq.html > > C:/MPI/Bin/mpiexec.exe: error while loading shared libraries: ?: cannot > > open shared object file: No such file or directory > > Possible error running C/C++ src/snes/examples/tutorials/ex19 with 2 MPI > > processes > > See http://www.mcs.anl.gov/petsc/documentation/faq.html > > C:/MPI/Bin/mpiexec.exe: error while loading shared libraries: ?: cannot > > open shared object file: No such file or directory > > Possible error running Fortran example src/snes/examples/tutorials/ex5f > > with 1 MPI process > > See http://www.mcs.anl.gov/petsc/documentation/faq.html > > [0]PETSC ERROR: > > ------------------------------------------------------------------------ > > [0]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, > > probably memory access out of range > > [0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger > > [0]PETSC ERROR: or see > > https://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind > > [0]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X > > to find memory corruption errors > > [0]PETSC ERROR: likely location of problem given in stack below > > [0]PETSC ERROR: --------------------- Stack Frames > > ------------------------------------ > > [0]PETSC ERROR: Note: The EXACT line numbers in the stack are not > > available, > > [0]PETSC ERROR: INSTEAD the line number of the start of the function > > [0]PETSC ERROR: is given. > > [0]PETSC ERROR: [0] VecNorm_Seq line 221 > > /cygdrive/d/Computational_geomechanics/installation/petsc-barry/src/vec/vec/impls/seq/bvec2.c > > [0]PETSC ERROR: [0] VecNorm line 213 > > /cygdrive/d/Computational_geomechanics/installation/petsc-barry/src/vec/vec/interface/rvector.c > > [0]PETSC ERROR: [0] SNESSolve_NEWTONLS line 144 > > /cygdrive/d/Computational_geomechanics/installation/petsc-barry/src/snes/impls/ls/ls.c > > [0]PETSC ERROR: [0] SNESSolve line 4375 > > /cygdrive/d/Computational_geomechanics/installation/petsc-barry/src/snes/interface/snes.c > > [0]PETSC ERROR: --------------------- Error Message > > -------------------------------------------------------------- > > [0]PETSC ERROR: Signal received > > [0]PETSC ERROR: See https://www.mcs.anl.gov/petsc/documentation/faq.html > > for trouble shooting. > > [0]PETSC ERROR: Petsc Development GIT revision: unknown GIT Date: unknown > > [0]PETSC ERROR: ./ex5f on a arch-mswin-c-debug named DESKTOP-R88IMOB by > > useruser Mon Jan 20 09:18:34 2020 > > [0]PETSC ERROR: Configure options --with-cc=x86_64-w64-mingw32-gcc > > --with-cxx=x86_64-w64-mingw32-g++ --with-fc=x86_64-w64-mingw32-gfortran > > --with-mpi-include=/cygdrive/c/MPISDK/Include > > --with-mpi-lib=/cygdrive/c/MPISDK/Lib/libmsmpi.a > > --with-mpi-mpiexec=/cygdrive/c/MPI/Bin/mpiexec.exe --with-debugging=yes > > -CFLAGS=-O2 -CXXFLAGS=-O2 -FFLAGS="-O2 -static-libgfortran -static > > -lpthread -fno-range-check -fdefault-integer-8" --download-fblaslapack > > --with-shared-libraries=no --with-64-bit-indices --force > > [0]PETSC ERROR: #1 User provided function() line 0 in unknown file > > > > job aborted: > > [ranks] message > > > > [0] application aborted > > aborting MPI_COMM_WORLD (comm=0x44000000), error 50152059, comm rank 0 > > > > ---- error analysis ----- > > > > [0] on DESKTOP-R88IMOB > > ./ex5f aborted the job. abort code 50152059 > > > > ---- error analysis ----- > > Completed test examples > > > > Kind regards, > > Dmitry Melnichuk > > > > 19.01.2020, 07:47, "Smith, Barry F." <[email protected]>: > > > > Dmitry, > > > > I have completed and tested the branch > > barry/2020-01-15/support-default-integer-8 it is undergoing testing now > > https://gitlab.com/petsc/petsc/merge_requests/2456 > > > > Please give it a try. Note that MPI has no support for integer > > promotion so YOU must insure that any MPI calls from Fortran pass 4 byte > > integers not promoted 8 byte integers. > > > > I have tested it with recent versions of MPICH and OpenMPI, it is > > fragile at compile time and may fail to compile with different versions of > > MPI. > > > > Good luck, > > > > Barry > > > > I do not recommend this approach for integer promotion in Fortran. > > Just blindly promoting all integers can often lead to problems. I recommend > > using the kind mechanism of > > Fortran to insure that each variable is the type you want, you can > > recompile with different options to promote the kind declared variables you > > wish. Of course this is more intrusive and requires changes to the Fortran > > code. > > > > > > On Jan 15, 2020, at 7:00 AM, Дмитрий Мельничук > > <[email protected]> wrote: > > > > Hello all! > > > > At present time I need to compile solver called Defmod > > (https://bitbucket.org/stali/defmod/wiki/Home), which is written in Fortran > > 95. > > Defmod uses PETSc for solving linear algebra system. > > Solver compilation with 32-bit version of PETSc does not cause any > > problem. > > But solver compilation with 64-bit version of PETSc produces an error > > with size of ierr PETSc variable. > > > > 1. For example, consider the following statements written in Fortran: > > > > > > PetscErrorCode :: ierr_m > > PetscInt :: ierr > > ... > > ... > > call VecDuplicate(Vec_U,Vec_Um,ierr) > > call VecCopy(Vec_U,Vec_Um,ierr) > > call VecGetLocalSize(Vec_U,j,ierr) > > call VecGetOwnershipRange(Vec_U,j1,j2,ierr_m) > > > > > > As can be seen first three subroutunes require ierr to be size of > > INTEGER(8), while the last subroutine (VecGetOwnershipRange) requires ierr > > to be size of INTEGER(4). > > Using the same integer format gives an error: > > > > There is no specific subroutine for the generic ‘vecgetownershiprange’ at > > (1) > > > > 2. Another example is: > > > > > > call MatAssemblyBegin(Mat_K,Mat_Final_Assembly,ierr) > > CHKERRA(ierr) > > call MatAssemblyEnd(Mat_K,Mat_Final_Assembly,ierr) > > > > > > I am not able to define an appropriate size if ierr in CHKERRA(ierr). If > > I choose INTEGER(8), the error "Type mismatch in argument ‘ierr’ at (1); > > passed INTEGER(8) to INTEGER(4)" occurs. > > If I define ierr as INTEGER(4), the error "Type mismatch in argument > > ‘ierr’ at (1); passed INTEGER(4) to INTEGER(8)" appears. > > > > > > 3. If I change the sizes of ierr vaiables as error messages require, the > > compilation completed successfully, but an error occurs when calculating > > the RHS vector with following message: > > > > [0]PETSC ERROR: Out of range index value -4 cannot be negative > > > > > > Command to configure 32-bit version of PETSc under Windows 10 using > > Cygwin: > > ./configure --with-cc=x86_64-w64-mingw32-gcc > > --with-cxx=x86_64-w64-mingw32-g++ --with-fc=x86_64-w64-mingw32-gfortran > > --download-fblaslapack --with-mpi-include=/cygdrive/c/MPISDK/Include > > --with-mpi-lib=/cygdrive/c/MPISDK/Lib/libmsmpi.a > > --with-mpi-mpiexec=/cygdrive/c/MPI/Bin/mpiexec.exe --with-debugging=yes > > -CFLAGS='-O2' -CXXFLAGS='-O2' -FFLAGS='-O2 -static-libgfortran -static > > -lpthread -fno-range-check' --with-shared-libraries=no > > > > Command to configure 64-bit version of PETSc under Windows 10 using > > Cygwin: > > ./configure --with-cc=x86_64-w64-mingw32-gcc > > --with-cxx=x86_64-w64-mingw32-g++ --with-fc=x86_64-w64-mingw32-gfortran > > --download-fblaslapack --with-mpi-include=/cygdrive/c/MPISDK/Include > > --with-mpi-lib=/cygdrive/c/MPISDK/Lib/libmsmpi.a > > --with-mpi-mpiexec=/cygdrive/c/MPI/Bin/mpiexec.exe --with-debugging=yes > > -CFLAGS='-O2' -CXXFLAGS='-O2' -FFLAGS='-O2 -static-libgfortran -static > > -lpthread -fno-range-check -fdefault-integer-8' --with-shared-libraries=no > > --with-64-bit-indices --known-64-bit-blas-indices > > > > > > Kind regards, > > Dmitry Melnichuk > > > > > >
