I meant MPICH with OFI for TrueScale. On Mon, Jun 25, 2018 at 12:35 PM Sayan Ghosh <[email protected]> wrote:
> Thanks, in my preliminary evaluation of MPICH with OFI, I don't see an > issue. > > On Mon, Jun 25, 2018 at 9:28 AM Hefty, Sean <[email protected]> wrote: > >> Actually copying the psm/psm2 provider maintainer this time. But, MPICH >> should work. >> >> - Sean >> >> > -----Original Message----- >> > From: ofiwg [mailto:[email protected]] On Behalf Of >> > Hefty, Sean >> > Sent: Friday, June 22, 2018 4:53 PM >> > To: Sayan Ghosh <[email protected]> >> > Cc: [email protected] >> > Subject: Re: [ofiwg] GASNet with OFI on PSM provider >> > >> > OFI does work over PSM. However, the psm and psm2 providers do not >> > support the same OFI feature set. You can see the differences here: >> > >> > https://github.com/ofiwg/libfabric/wiki/Provider-Feature-Matrix-master >> > >> > I'm not familiar enough with the GasNET implementation to know what >> > feature it needs. >> > >> > I'm copying the developer who maintains both the psm and psm2 >> > providers. He can provide more details on what differences there are >> > between psm and psm2, and *might* know if GasNET should work over the >> > psm provider. Unfortunately, the developer who ported GasNET to OFI >> > is not available. >> > >> > - Sean >> > >> > >> > > -----Original Message----- >> > > From: Sayan Ghosh [mailto:[email protected]] >> > > Sent: Friday, June 22, 2018 4:39 PM >> > > To: Hefty, Sean <[email protected]> >> > > Cc: [email protected] >> > > Subject: Re: [ofiwg] GASNet with OFI on PSM provider >> > > >> > > Hi Sean - Sorry if it is evident, but I just wanted to clarify >> > > something. Since OFI has a PSM provider, can I assume that it works? >> > I >> > > am asking because you said "OFI targets Omni Path". >> > > >> > > On Wed, Jun 20, 2018 at 12:08 PM Hefty, Sean <[email protected] >> > > <mailto:[email protected]> > wrote: >> > > >> > > >> > > To clarify, I meant GASNet to TrueScale through OFI. GASNet >> > should >> > > have a psm-conduit. OFI targets Omni-Path. >> > > >> > > >> > > >> > > - Sean >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > This is my understanding as well. I don’t believe GASNet is a >> > target >> > > application for TrueScale or PSM. The high performance interface >> > for >> > > TrueScale is PSM, rather than verbs, and targets older MPI >> > > implementations. >> > > >> > > >> > > >> > > - Sean >> > > >> > > >> > > >> > > >> > > >> > > From: ofiwg [mailto:[email protected]] On >> > Behalf Of >> > > Sayan Ghosh >> > > Sent: Wednesday, June 20, 2018 9:17 AM >> > > To: [email protected] >> > > <mailto:[email protected]> >> > > Subject: Re: [ofiwg] GASNet with OFI on PSM provider >> > > >> > > >> > > >> > > Just wanted to clarify that a discussion with GASNet developers >> > > revealed that their PSM provider compatibility over OFI is only for >> > > PSM2 (since TrueScale is an old technology). >> > > >> > > https://groups.google.com/a/lbl.gov/forum/#!topic/gasnet- >> > > users/1Y_mDmdcCDw >> > > >> > > >> > > >> > > On Tue, Jun 19, 2018 at 10:44 PM Sayan Ghosh <[email protected] >> > > <mailto:[email protected]> > wrote: >> > > >> > > I am trying to install GASNet 1.30.0 using OFI on an Intel >> > TrueScale >> > > network (PSM provider), and although OFI fi_info detects PSM >> > provider >> > > (see attached output of fi_info), when I mention it during GASNet >> > > configure (config.log attached, please see line #10126), GASNet >> > > warns: >> > > >> > > >> > > >> > > WARNING: Invalid or unsupported OFI provider specified via >> > the >> > > --with-ofi-provider flag. >> > > >> > > Defaulting to runtime provider feature detection for ofi- >> > conduit. >> > > >> > > >> > > >> > > >> > > GASNet configuration: >> > > >> > > >> > > >> > > CC=icc CXX=icpc MPI_CC=/home/sghosh/builds/mvapich222- >> > > intel-psm/bin/mpicc MPI_CFLAGS="-I/home/sghosh/builds/mvapich222- >> > > intel-psm/include" MPI_LIBS="-L/home/sghosh/builds/mvapich222-intel- >> > > psm/lib -lpciaccess -lnuma -lxml2" ./configure -- >> > > prefix=/home/sghosh/builds/gasnet-ofi --enable-ofi --with-ofi- >> > > home=/home/sghosh/builds/libfabric --disable-psm --disable-ibv -- >> > with- >> > > ofi-spawner=mpi --with-ofi-provider=psm >> > > >> > > >> > > >> > > GASNet throws this warning during application execution: >> > > >> > > >> > > >> > > WARNING: Using OFI provider (verbs;ofi_rxm), which has not >> > been >> > > validated to provide >> > > >> > > WARNING: acceptable GASNet performance. You should >> > consider using a >> > > more >> > > >> > > WARNING: hardware-appropriate GASNet conduit. See ofi- >> > > conduit/README. >> > > >> > > *** FATAL ERROR: The statically-determined value for >> > > GASNETC_OFI_HAS_MR_SCALABLE=1 does >> > > >> > > not match the memory registration support that the >> > > (verbs;ofi_rxm) provider reported. >> > > >> > > This could happen if a provider that previously only >> > supported >> > > FI_MR_BASIC >> > > >> > > added support for FI_MR_SCALABLE, or if the wrong >> > provider was >> > > selected at runtime. >> > > >> > > Use configure option --disable-ofi-mr-scalable to >> > correct this. >> > > >> > > >> > > >> > > Instead of picking up psm, GASNet picks up verbs. I have >> > reported >> > > this issue on GASNet user list as well. Please let me know if I can >> > > try some options in building libfabrics for PSM provider, that may >> > fix >> > > this issue. >> > > >> > > >> > > >> > > -- >> > > >> > > Thanks, >> > > >> > > Sayan | https://sg0.github.io/ >> > > >> > > >> > > >> > > >> > > >> > > >> > > -- >> > > >> > > Sayan | https://sg0.github.io/ >> > > >> > > >> > > >> > > -- >> > > >> > > Sayan | https://sg0.github.io/ >> > >> > _______________________________________________ >> > ofiwg mailing list >> > [email protected] >> > http://lists.openfabrics.org/mailman/listinfo/ofiwg >> > > > -- > Sayan | https://sg0.github.io/ > -- Sayan | https://sg0.github.io/
_______________________________________________ ofiwg mailing list [email protected] http://lists.openfabrics.org/mailman/listinfo/ofiwg
