Giuseppe, The move to Kokkos is still in progress, so right now we have a mix of Kokkos and CUDA. This works because under the hood Kokkos uses CUDA to run on Nvidia GPUs. So far we have changed the build system and the distributed vectors. I have a branch where the MatrixFree infrastructure uses Kokkos. I still need to clean up some of the code before I create the PR. The SparseMatrix code won't be moved to Kokkos for a while but you can mix CUDA and Kokkos so it shouldn't be a problem.
Best, Bruno Le lun. 8 mai 2023 à 15:08, Giuseppe Orlando <[email protected]> a écrit : > Dear Bruno, > > thanks for the kind response. I will certainly have a look to the files > that you suggested me about the CUDAWrappers::SparseMatrix (and also the > other tests included therein). > > Just as curiosity (if I am not inappropriate and you can answer me): are > these new GPU facilities with Kokkos already and usable ailable anin the > development repository? Because, for instance, the step-64 tutorial source > code has been converted from ‘step-64.cu’ to ‘step-64.cc’, the same holds > for the CUDA tests folder that you linked me and, finally, I could find > some ‘Kokkos keywords’, but I cannot figure out from a rapid glance if it > can be already executed with a version of deal.II installed with > ‘DEAL_II_WITH_KOKKOS’ flag on or one has to wait the new release. > > > > Best > > > > Giuseppe > > > > *Da: *Bruno Turcksin <[email protected]> > *Inviato: *lunedì 8 maggio 2023 16:11 > *A: *deal.II User Group <[email protected]> > *Oggetto: *[deal.II] Re: CUDA facilities > > > > Giuseppe, > > > > I am not sure if google sent my previous message, so I will send it again. > I apologize if you get it twice. > > > > We are currently replacing the CUDA code with Kokkos. These will allow us > to support Nvidia, AMD, and Intel GPUs. It will also make it easier to add > new functionalities. Unfortunately there will be a very limited number of > non-backward compatible changes. > > > > On Monday, May 8, 2023 at 5:50:22 AM UTC-4 [email protected] wrote: > > > > 1. Searching for the documentation, I noticed that routines such as > CUDAWrappers::FEFaceEvalution or CUDAWrappers::MatrixFree::loop are not > defined. Am I correct? Does this mean, for instance, that a Discontinuous > Galerkin extension of step-64 tutorial with a matrix free formulation is > not feasible at the moment? > > That's correct. Currently we only support continuous finite elements with > a single component. Support for multiple components is on my todo list. > > > > 2. I noticed instead that a CUDAWrappers::SparseMatrix class is defined. > However, it seems to me that it can be initialized only with SparseMatrix > class. Does it mean that it cannot be combined with MPI sparse matrices > (i.e. PETSc or Trilinos)? Moreover, is there somewhere an example of use of > such class, so as to check e.g. the proper use Utilities::CUDA::Handle for > the 'reinit' function? > > > > CUDAWrappers::SparseMatrix is the CUDA version of the SparseMatrix class > and so it has the same limitations. If you want MPI sparse matrices, you > need to use PETSc or Trilinos. They support GPU through Kokkos. > > For examples using CUDAWrappers::SparseMatrix look at the solver, > sparse_matrix, and precondition tests here > <https://github.com/dealii/dealii/tree/dealii-9.4/tests/cuda>. > > > > Best, > > > > Bruno > > -- > The deal.II project is located at http://www.dealii.org/ > For mailing list/forum options, see > https://groups.google.com/d/forum/dealii?hl=en > --- > You received this message because you are subscribed to a topic in the > Google Groups "deal.II User Group" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/dealii/Fgm4g2TjOrQ/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/dealii/6ea874b3-8758-47f5-b6a5-a225ddbddd00n%40googlegroups.com > <https://groups.google.com/d/msgid/dealii/6ea874b3-8758-47f5-b6a5-a225ddbddd00n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > > > -- > The deal.II project is located at http://www.dealii.org/ > For mailing list/forum options, see > https://groups.google.com/d/forum/dealii?hl=en > --- > You received this message because you are subscribed to a topic in the > Google Groups "deal.II User Group" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/dealii/Fgm4g2TjOrQ/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/dealii/0F7FB6EA-24F4-4146-B1BA-FC856DD4BC18%40hxcore.ol > <https://groups.google.com/d/msgid/dealii/0F7FB6EA-24F4-4146-B1BA-FC856DD4BC18%40hxcore.ol?utm_medium=email&utm_source=footer> > . > -- The deal.II project is located at http://www.dealii.org/ For mailing list/forum options, see https://groups.google.com/d/forum/dealii?hl=en --- You received this message because you are subscribed to the Google Groups "deal.II User Group" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/dealii/CAGVt9eMfXmG8B27x554OcdrOpC_8VL%2B0%2BRyTj4KbQHWGf0MAmw%40mail.gmail.com.
