> Is there support in the standard OFED for GPU direct? Or is it only supported
> by
> the MLNX_OFED package Mellanox provides?
[Tom]
It may depend on whether you are using a fairly recent Linux distro and CUDA
4.0 or newer. In that case you may not need GPUDirect software from Mellanox
to get GPUDirect v1 levels of performance.
I realize that you probably have Mellanox HCAs, but this answer for QLogic, now
Intel, IB HCAs may also apply to Mellanox:
The following applies to the QLogic OFED+ 7.1 (most recent) distribution :
If you are running on any Linux distribution supported by this QLogic
InfiniBand software release*, then you must use NVIDIA CUDA 4.0 or newer to
have a supported solution from NVIDIA. To get optimal performance with the
qib InfiniBand driver in this software release in applications which use CUDA
4.x,
all you need to do is to set the environment variable:
CUDA_NIC_INTEROP=1
and to propagate that variable to the compute nodes running the application.
The preceding instructions should also work with the qib driver in the recent
OFED 1.5.4.1 release -- and probably for the Mellanox driver too.
If you need to use CUDA 3.x and/or older Linux releases than say RHEL 5.6 or
SLES 11 SP1, then you probably need the closed-source GPUDirect software from
Mellanox. QLogic had a different way to provide the GPUDirect capability on
these older CUDA/Linux releases.
-Tom Elken
Intel InfiniBand (formerly QLogic)
* RHEL 5.6, 5.7 or RHEL 6.0 or newer
SLES 11 SP1 or SLES 10 SP4
>
> I searched the mailing list and found some talk about GPU direct here:
> http://lists.openfabrics.org/pipermail/ewg/2011-February/016202.html
> but it indicated it might be a closed source thing from Mellanox only.
>
> Are there any other sources of information about this that I should read?
>
> Thanks,
> Rick
> _______________________________________________
> ewg mailing list
> [email protected]
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
_______________________________________________
ewg mailing list
[email protected]
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
