Sorry, forgot to send a copy to petsc-users mailing list when I first replied. My first reply is below.
On Sun, Sep 25, 2022 at 11:37 AM Bro H <[email protected]> wrote: > > Barry, thank you for answering. I did some further testing. My MKL > version is 20220002 as detected by PETSc. I tried to compile one of > the examples distributed with MKL > (${MKLROOT}/latest/examples/examples_core_c.tgz/c/sparse_blas/source/sparse_csr.c) > that contains calls to mkl_sparse_d_mv function from Sparse BLAS, > among others. It compiled and ran without errors with 64-bit indices. > > I used the following command to compile this example on Debian 11: > > gcc `mkl_link_tool --quiet -c gnu_c -o gomp -i ilp64 -opts` > sparse_csr.c `mkl_link_tool --quiet -c gnu_c -o gomp -i ilp64 -libs` > > mkl_link_tool is bundled with MKL. It returns: > > `mkl_link_tool --quiet -c gnu_c -o gomp -i ilp64 -opts` = -DMKL_ILP64 > -m64 -I"${MKLROOT}/include" > > `mkl_link_tool --quiet -c gnu_c -o gomp -i ilp64 -opts` = > -L${MKLROOT}/lib/intel64 -Wl,--no-as-needed -lmkl_intel_ilp64 > -lmkl_gnu_thread -lmkl_core -lgomp -lpthread -lm -ldl > > So in theory it should work when PetscInt is 64 bits, am I right? The > only problem is with the build scripts or source code in PETSc? > > Per your suggestion, I have tried to change requires32bitint to 0 in > mkl_sparse.py and mkl_sparse_optimize.py routines. PETSc used 64-bit > indices, but configuration also set MKL to 32-bit indices, so I added > --with-64-bit-blas-indices. I then ran into "Cannot use mkl_sparse > with 64-bit BLAS/Lapack indices" error, and so I tried to change > requires32bitintblas to 0 in config/BuildSystem/config/package.py. > Configuration was successful with both PETSc and MKL set to 64-bit > indices, however, it looks like PETSc was still using 32-bit indices > during compilation of aijmkl implementation, with warnings such as: > > petsc-3.17.4/src/mat/impls/aij/seq/aijmkl/aijmkl.c:282:98: warning: > passing argument 7 of ‘mkl_sparse_d_export_csr’ from incompatible > pointer type > > Are there any other modifications required to make PETSc work with > 64-bit MKL Sparse BLAS? > > On Sun, Sep 25, 2022 at 3:54 AM Barry Smith <[email protected]> wrote: > > > > > > It is possible they recently added support for using it with 64 bit > > integers. You would need to through their documents to see how to get > > mkl_spblas.h. to use 64 bit integers and if the library for 64 bit integer > > has a different name that would need to be linked to. > > > > You would need to remove the requires32bitint = 1 from the various > > config/BuildSystem/config/packages/mkl_*.py routines to get configure to > > accept MKL sparse with 64 bit integers. > > > > > > > On Sep 24, 2022, at 11:04 AM, Bro H <[email protected]> wrote: > > > > > > Hello. > > > > > > I would like to build PETSc with support for MKL Sparse BLAS, so that > > > I can use MATAIJMKL for improved performance, but I also need to use > > > 64 bit indices. I'm trying to build using the following parameters: > > > > > > ./configure --force --prefix="/opt/libs/petsc" --with-precision=double > > > --with-64-bit-indices > > > --with-blas-lapack-dir=/opt/intel/oneapi/mkl/latest/ > > > --with-mkl_sparse=1 --with-mkl_sparse_optimize=1 --with-debugging=0 > > > --with-shared-libraries=1 --with-cxx=0 --with-mpi=1 --with-hdf5=1 > > > --with-hdf5-dir="${HDF5_ROOT}" --with-openmp=1 > > > > > > Configuration fails when using --with-64-bit-indices simultaneously > > > with --with-mkl_sparse=1 and/or --with-mkl_sparse_optimize=1 with the > > > following possible errors: > > > > > > "Cannot use mkl_sparse with 64 bit integers, it is not coded for this > > > capability" > > > "Cannot use mkl_sparse_optimize with 64 bit integers, it is not coded > > > for this capability" > > > > > > But doesn't the latest version of MKL Sparse BLAS support 64 bit > > > indices? It is using MKL_INT in mkl_spblas.h. > > > > > > Is it possible to build PETSc with 64 bit indices and MKL Sparse BLAS? > >
