Hi Jose,
> We are trying to do some GPU developments on the SLEPc side, and we
would need a way of placing the array of a VECCUSP vector, providing the
GPU address. Specifically, what we want to do is have a large Vec on GPU
and slice it in several smaller Vecs.
For the GetArray/RestoreArray we have all possibilities:
- VecGetArray: gets the pointer to the buffer stored in CPU memory
- VecCUSPGetArray*: returns a CUSPARRAY object that contains some info,
including the buffer allocated in GPU memory
- VecCUSPGetCUDAArray*: returns a raw pointer of the GPU buffer
The problem comes with PlaceArray equivalents. Using VecPlaceArray we can
provide a new pointer to CPU memory. We wanted to implement the equivalent
thing for GPU, but we found difficulties due to Thrust. If we wanted to provide
a VecCUSPPlaceCUDAArray the problem is that Thrust does not allow wrapping an
exisiting GPU buffer with a CUSPARRAY object (when creating a CUSPARRAY it
always allocates new memory). On the other hand, VecCUSPPlaceArray is possible
to implement, but the problem is that one should provide a CUSPARRAY obtained
from a VecCUSPGetArray* without modification (it is not possible to do pointer
arithmetic with a CUSPARRAY).
As far as I can see from browsing the documentation and the web there is
indeed no such option. Ouch.
Any thoughts?
I'll soon expose the CUDA backend of ViennaCL in PETSc, which will offer
such functionality. Is this an option? It will take 2-3 weeks, though.
Best regards,
Karli
