FWIW: now that I am out of Intel, we are planning on upping the PMIx support for GPUs in general, so I expect we'll be including this one. Support will include providing info on capabilities (for both local and remote devices), distances from every proc to each of its local GPUs, affinity settings, etc.
> On Feb 11, 2021, at 10:57 AM, Atchley, Scott {Leadership Computing} via devel > <devel@lists.open-mpi.org> wrote: > >> On Feb 11, 2021, at 1:56 PM, Atchley, Scott <e...@ornl.gov> wrote: >> >>> On Feb 11, 2021, at 1:11 PM, Jeff Squyres (jsquyres) via devel >>> <devel@lists.open-mpi.org> wrote: >>> >>> <snip> >>> >>> That being said, we just added the AVX MPI_Op component -- equivalent >>> components could be added for CUDA and/or AMD's GPU (what API does it use >>> -- OpenCL?). >> >> AMD’s API is HIP: >> >> https://rocmdocs.amd.com/en/latest/Programming_Guides/HIP-GUIDE.html >> >> It is an abstraction of CUDA that allows compiling to AMD or NVIDIA GPUs. > > I should have added that there is an ECP project to port it to Intel GPUs as > well.