Thanks Sean, just to clarify, this question that I asked right now has nothing to do with GASNet (I was trying to avoid a new email chain). I was just wondering if I can build OFI with PSM provider, and use it with MPICH. From your answer, it seems that I can.
On Fri, Jun 22, 2018 at 4:53 PM Hefty, Sean <[email protected]> wrote: > OFI does work over PSM. However, the psm and psm2 providers do not > support the same OFI feature set. You can see the differences here: > > https://github.com/ofiwg/libfabric/wiki/Provider-Feature-Matrix-master > > I'm not familiar enough with the GasNET implementation to know what > feature it needs. > > I'm copying the developer who maintains both the psm and psm2 providers. > He can provide more details on what differences there are between psm and > psm2, and *might* know if GasNET should work over the psm provider. > Unfortunately, the developer who ported GasNET to OFI is not available. > > - Sean > > > > -----Original Message----- > > From: Sayan Ghosh [mailto:[email protected]] > > Sent: Friday, June 22, 2018 4:39 PM > > To: Hefty, Sean <[email protected]> > > Cc: [email protected] > > Subject: Re: [ofiwg] GASNet with OFI on PSM provider > > > > Hi Sean - Sorry if it is evident, but I just wanted to clarify > > something. Since OFI has a PSM provider, can I assume that it works? I > > am asking because you said "OFI targets Omni Path". > > > > On Wed, Jun 20, 2018 at 12:08 PM Hefty, Sean <[email protected] > > <mailto:[email protected]> > wrote: > > > > > > To clarify, I meant GASNet to TrueScale through OFI. GASNet > > should have a psm-conduit. OFI targets Omni-Path. > > > > > > > > - Sean > > > > > > > > > > > > > > > > This is my understanding as well. I don’t believe GASNet is a > > target application for TrueScale or PSM. The high performance > > interface for TrueScale is PSM, rather than verbs, and targets older > > MPI implementations. > > > > > > > > - Sean > > > > > > > > > > > > From: ofiwg [mailto:[email protected]] On > > Behalf Of Sayan Ghosh > > Sent: Wednesday, June 20, 2018 9:17 AM > > To: [email protected] > > <mailto:[email protected]> > > Subject: Re: [ofiwg] GASNet with OFI on PSM provider > > > > > > > > Just wanted to clarify that a discussion with GASNet developers > > revealed that their PSM provider compatibility over OFI is only for > > PSM2 (since TrueScale is an old technology). > > > > https://groups.google.com/a/lbl.gov/forum/#!topic/gasnet- > > users/1Y_mDmdcCDw > > > > > > > > On Tue, Jun 19, 2018 at 10:44 PM Sayan Ghosh <[email protected] > > <mailto:[email protected]> > wrote: > > > > I am trying to install GASNet 1.30.0 using OFI on an Intel > > TrueScale network (PSM provider), and although OFI fi_info detects PSM > > provider (see attached output of fi_info), when I mention it during > > GASNet configure (config.log attached, please see line #10126), GASNet > > warns: > > > > > > > > WARNING: Invalid or unsupported OFI provider specified via > > the --with-ofi-provider flag. > > > > Defaulting to runtime provider feature detection for ofi- > > conduit. > > > > > > > > > > GASNet configuration: > > > > > > > > CC=icc CXX=icpc MPI_CC=/home/sghosh/builds/mvapich222- > > intel-psm/bin/mpicc MPI_CFLAGS="-I/home/sghosh/builds/mvapich222- > > intel-psm/include" MPI_LIBS="-L/home/sghosh/builds/mvapich222-intel- > > psm/lib -lpciaccess -lnuma -lxml2" ./configure -- > > prefix=/home/sghosh/builds/gasnet-ofi --enable-ofi --with-ofi- > > home=/home/sghosh/builds/libfabric --disable-psm --disable-ibv --with- > > ofi-spawner=mpi --with-ofi-provider=psm > > > > > > > > GASNet throws this warning during application execution: > > > > > > > > WARNING: Using OFI provider (verbs;ofi_rxm), which has not > > been validated to provide > > > > WARNING: acceptable GASNet performance. You should > > consider using a more > > > > WARNING: hardware-appropriate GASNet conduit. See ofi- > > conduit/README. > > > > *** FATAL ERROR: The statically-determined value for > > GASNETC_OFI_HAS_MR_SCALABLE=1 does > > > > not match the memory registration support that the > > (verbs;ofi_rxm) provider reported. > > > > This could happen if a provider that previously only > > supported FI_MR_BASIC > > > > added support for FI_MR_SCALABLE, or if the wrong > > provider was selected at runtime. > > > > Use configure option --disable-ofi-mr-scalable to > > correct this. > > > > > > > > Instead of picking up psm, GASNet picks up verbs. I have > > reported this issue on GASNet user list as well. Please let me know if > > I can try some options in building libfabrics for PSM provider, that > > may fix this issue. > > > > > > > > -- > > > > Thanks, > > > > Sayan | https://sg0.github.io/ > > > > > > > > > > > > > > -- > > > > Sayan | https://sg0.github.io/ > > > > > > > > -- > > > > Sayan | https://sg0.github.io/ > > -- Sayan | https://sg0.github.io/
_______________________________________________ ofiwg mailing list [email protected] http://lists.openfabrics.org/mailman/listinfo/ofiwg
