I see. Again to confirm, are the following lines (in proc2) the right way to actually copy data? x_gpu = gpuarray.GPUArray(shape, dtype, gpudata=drv.IPCMemoryHandle(h)) y_gpu = gpuarray.zeros_like(x_gpu) drv.memcpy_peer(y_gpu.ptr, x_gpu.ptr, x_gpu.dtype.itemsize * x_gpu.size, ctx, ctx)
It seems working for me, I'm kind of not sure about to set dest_ctx and src_ctx to be the same. Thanks a lot!!! On Wed, Oct 8, 2014 at 2:05 AM, Lev Givon <[email protected]> wrote: > Received from Gavin Weiguang Ding on Tue, Oct 07, 2014 at 06:40:46PM EDT: > > Hi Lev, > > > > Thanks a lot for the example! It really helps! > > > > Just to make sure I understand the example in the right way: > > > > The actual p2p data transfer happens when executing the following line? > > x_gpu = gpuarray.GPUArray(shape, dtype, gpudata=drv.IPCMemoryHandle(h)) > > > > So, if I use different devices in proc1 and proc2, then > > x_gpu = gpuarray.GPUArray(shape, dtype, gpudata=drv.IPCMemoryHandle(h)) > > would actually transfer the data from device 1 to device 2 through p2p? > > > > My GPU server has 3 gpus, and only device 1 and 2 can do p2p. > > What I observed is that > > the example works when > > 1. proc1 and proc2 using the same device > > 2. proc1 and proc2 using device 1 and 2 respectively or vice versa > > > > it doesn't work when > > proc1 using device 0 and proc2 using 1 or 2 and vice versa > > it also returns the error of > > "LogicError: cuIpcOpenMemHandle failed: invalid/unknown error code" > > > > So I guess that's the desired behaviour? and IPCMemoryHandle only works > > when p2p (or within same GPU) is possible? > > > > Thanks a lot! > > CUDA's IPC mechanism doesn't actually copy any data; it makes it possible > to share a device pointer created in one process with some other process. > You > still need to transfer the data from one location to the other. > -- > Lev Givon > Bionet Group | Neurokernel Project > http://www.columbia.edu/~lev/ > http://lebedov.github.io/ > http://neurokernel.github.io/ > >
_______________________________________________ PyCUDA mailing list [email protected] http://lists.tiker.net/listinfo/pycuda
