This is loosely a follow up to [1]. In this thread a few potential ways for making GPU assembly work with PETSc were discussed and to me the two most promising appeared to be: 1) Create a PETSc matrix from a pre-assembled CSR structure, or 2) Preallocate a PETSc matrix and get the handle to pass the row pointer, column indices and values array to a custom assembly routine.
To recap what Lawrence already said in [1]: We're working on PyOP2 [2] a tool chain for parallel (FEM) computations on unstructured meshes with support for multiple hardware platforms, currently CPU + MPI + OpenMP and CUDA. We are using PETSc (MPI)AIJ on the CPU, and thinking how to make multi-GPU via MPI work, preferably also using PETSc. We compute local assembly matrices on the GPU and a crucial requirement is that the matrix *only* lives in device device, we want to avoid any host <-> device data transfers. So far we have been using CUSP with a custom (generated) assembly into our own CUSP-compatible CSR data structure for a single GPU. Since CUSP doesn't give us multi-GPU solvers out of the box we'd rather use existing infrastructure that works rather than rolling our own. At the time of [1] supporting GPU assembly in one form or the other was on the roadmap, but the implementation direction seemed to not have been finally decided. Was there any progress since then or anything to add to the discussion? Is there even (experimental) code we might be able to use? Note that we're using petsc4py to interface to PETSc. A cursory look through petsc-dev and currently active petsc branches didn't reveal anything obvious, but apologies if I missed something. Florian [1]: https://lists.mcs.anl.gov/mailman/htdig/petsc-dev/2012-November/009941.html [2]: https://github.com/OP2/PyOP2 -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2913 bytes Desc: S/MIME Cryptographic Signature URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130502/ffc9caa1/attachment.bin>
