Likely you can fix the problem by adding #if defined(PETSC_HAVE_MKL_INTEL_ILP64) #define MKL_ILP64 #endif
before the #include <mkl_spblas.h> in src/mat/impls/aij/seq/aijmkl/aijmkl.c and src/mat/impls/baij/seq/baijmkl/baijmkl.c Please let us know if this resolves the problem. Barry > On Sep 25, 2022, at 12:37 AM, Bro H <[email protected]> wrote: > > Barry, thank you for answering. I did some further testing. My MKL > version is 20220002 as detected by PETSc. I tried to compile one of > the examples distributed with MKL > (${MKLROOT}/latest/examples/examples_core_c.tgz/c/sparse_blas/source/sparse_csr.c) > that contains calls to mkl_sparse_d_mv function from Sparse BLAS, > among others. It compiled and ran without errors with 64-bit indices. > > I used the following command to compile this example on Debian 11: > > gcc `mkl_link_tool --quiet -c gnu_c -o gomp -i ilp64 -opts` > sparse_csr.c `mkl_link_tool --quiet -c gnu_c -o gomp -i ilp64 -libs` > > mkl_link_tool is bundled with MKL. It returns: > > `mkl_link_tool --quiet -c gnu_c -o gomp -i ilp64 -opts` = -DMKL_ILP64 > -m64 -I"${MKLROOT}/include" > > `mkl_link_tool --quiet -c gnu_c -o gomp -i ilp64 -opts` = > -L${MKLROOT}/lib/intel64 -Wl,--no-as-needed -lmkl_intel_ilp64 > -lmkl_gnu_thread -lmkl_core -lgomp -lpthread -lm -ldl > > So in theory it should work when PetscInt is 64 bits, am I right? The > only problem is with the build scripts or source code in PETSc? > > Per your suggestion, I have tried to change requires32bitint to 0 in > mkl_sparse.py and mkl_sparse_optimize.py routines. PETSc used 64-bit > indices, but configuration also set MKL to 32-bit indices, so I added > --with-64-bit-blas-indices. I then ran into "Cannot use mkl_sparse > with 64-bit BLAS/Lapack indices" error, and so I tried to change > requires32bitintblas to 0 in config/BuildSystem/config/package.py. > Configuration was successful with both PETSc and MKL set to 64-bit > indices, however, it looks like PETSc was still using 32-bit indices > during compilation of aijmkl implementation, with warnings such as: > > petsc-3.17.4/src/mat/impls/aij/seq/aijmkl/aijmkl.c:282:98: warning: > passing argument 7 of ‘mkl_sparse_d_export_csr’ from incompatible > pointer type > > Are there any other modifications required to make PETSc work with > 64-bit MKL Sparse BLAS? > > On Sun, Sep 25, 2022 at 3:54 AM Barry Smith <[email protected]> wrote: >> >> >> It is possible they recently added support for using it with 64 bit >> integers. You would need to through their documents to see how to get >> mkl_spblas.h. to use 64 bit integers and if the library for 64 bit integer >> has a different name that would need to be linked to. >> >> You would need to remove the requires32bitint = 1 from the various >> config/BuildSystem/config/packages/mkl_*.py routines to get configure to >> accept MKL sparse with 64 bit integers. >> >> >>> On Sep 24, 2022, at 11:04 AM, Bro H <[email protected]> wrote: >>> >>> Hello. >>> >>> I would like to build PETSc with support for MKL Sparse BLAS, so that >>> I can use MATAIJMKL for improved performance, but I also need to use >>> 64 bit indices. I'm trying to build using the following parameters: >>> >>> ./configure --force --prefix="/opt/libs/petsc" --with-precision=double >>> --with-64-bit-indices >>> --with-blas-lapack-dir=/opt/intel/oneapi/mkl/latest/ >>> --with-mkl_sparse=1 --with-mkl_sparse_optimize=1 --with-debugging=0 >>> --with-shared-libraries=1 --with-cxx=0 --with-mpi=1 --with-hdf5=1 >>> --with-hdf5-dir="${HDF5_ROOT}" --with-openmp=1 >>> >>> Configuration fails when using --with-64-bit-indices simultaneously >>> with --with-mkl_sparse=1 and/or --with-mkl_sparse_optimize=1 with the >>> following possible errors: >>> >>> "Cannot use mkl_sparse with 64 bit integers, it is not coded for this >>> capability" >>> "Cannot use mkl_sparse_optimize with 64 bit integers, it is not coded >>> for this capability" >>> >>> But doesn't the latest version of MKL Sparse BLAS support 64 bit >>> indices? It is using MKL_INT in mkl_spblas.h. >>> >>> Is it possible to build PETSc with 64 bit indices and MKL Sparse BLAS? >>
