Thanks. This is very useful information. I will experiment with the collectives and see if I can get any tentative results in our HPX backend.
I'm also interested in GPUDirect support and interfacing via libfabric - or at least having a simple async interface of the same style that allows me to coordinate transfers from GPU->GPU/CPU etc - I see that this has been discussed at ofiwg meetings , so I'll keep my ear to the ground for more news. Cheers JB ________________________________ From: Hefty, Sean <[email protected]> Sent: 27 November 2019 18:38:47 To: Biddiscombe, John A.; '[email protected]'; [email protected] Subject: RE: libfabric release v1.9.0 is now available > Where can I find further information on collectives? - I have been waiting > for these > for a while and was considering switching from libfabric to UCX as they have > a lot of > activity on their mailing list and I saw progress on collectives. I am > willing to make > contributions to libfabric collectives if there is work to be done. For now, > I'd > welcome any info on how to start testing experimenting with them. See: https://ofiwg.github.io/libfabric/master/man/fi_collective.3.html Collective support was discussed a few times in the OFIWG. These target switch based collectives. The rxm provider has implemented 2-3 of the collectives, but that is actively being expanded. I would guess that most or all collectives that have been defined will likely be available in a 1.10 release (target is March). Since the collectives target offloaded support, there will likely be limitations on the software implementation. The primary goal of the software implementation is to verify the API and allow application development. Real collective optimizations would still be the responsibility of MPI. - Sean
_______________________________________________ ofiwg mailing list [email protected] https://lists.openfabrics.org/mailman/listinfo/ofiwg
