Hi Gilles,
My main concern is that if the user specifies InfiniBand per Jeff’s proposal,
will they get or not get sm (of any flavor) and self.
Scott
On Oct 21, 2015, at 9:00 AM, Gilles Gouaillardet
<gilles.gouaillar...@gmail.com> wrote:
> Scott and all,
>
> two btl are optimized (and work only) for intra node communications : sm and
> vader
>
> by "sm" I am not sure you mean the sm btl, or any/both sm and vader btl.
>
> from an user point of view, and to disambiguate this, maybe we should use the
> term "shm"
> (which means sm and/or vader btl for ompi developers)
>
> Cheers,
>
> Gilles
>
>
> On Wednesday, October 21, 2015, Atchley, Scott <atchle...@ornl.gov> wrote:
> On Oct 20, 2015, at 4:45 PM, Jeff Squyres (jsquyres) <jsquy...@cisco.com>
> wrote:
>
> > On Oct 20, 2015, at 3:42 PM, Jeff Squyres (jsquyres) <jsquy...@cisco.com>
> > wrote:
> >>
> >> I'm guessing we'll talk about this at the Feb dev meeting, but we need to
> >> think about this a bit before hand. Here's a little more fuel for the
> >> fire: let's at least specify the problem space a bit more precisely...
> >
> > I'm replying to my own mail because I wanted a separate email for a
> > (half-baked) proposal.
> >
> > How about something like this:
> >
> > mpirun --[enable|disable]
> > NETWORK_TYPE[:QUALIFIER][,NETWORK_TYPE[:QUALIFIER]]*
> >
> > 1. Both forms take a comma-delimited list of 1 or more items.
> >
> > 2. --enable would work similar to our "include" MCA params: OMPI will
> > *only* use the network type(s) listed.
>
> Jeff,
>
> In this scenario, will the user still need to “enable” off-node network, sm,
> and self? Or do you assume sm and self?
>
> Scott
>
>
> >
> > 3. --disable would work similar to our "exclude" MCA params: OMPI will use
> > all network types *except* those listed.
> >
> > --> Alternative, if "--enable" and "--disable" are too general, we could
> > use "--net" and "--nonet", or something like that. Suggestions welcome.
> >
> > 4. NETWORK_TYPE values can be registered via an OPAL API, e.g.:
> >
> > // In the TCP BTL
> > opal_register_network_type("eth", ...some TCP BTL callback function...);
> > // In the usnic BTL
> > opal_register_network_type("eth", ...some usnic BTL callback function...);
> >
> > // In the openib BTL
> > opal_register_network_type("ib", ...some openib BTL callback function...);
> > // In the MXM MTL
> > opal_register_network_type("ib", ...some MXM MTL callback function...);
> >
> > The main idea, though, is that various components can register themselves
> > to these network type names that can be specified on the
> > mpirun/orterun/oshrun command lines. Once a user specifies a network type,
> > I'm not quite sure what OMPI does next (e.g., what will that callback
> > function pointer do?), ...but I thought I'd at least capture these thoughts
> > far. :-)
> >
> > We can even allow synonyms:
> >
> > opal_register_network_synonym("eth", "ethernet");
> > opal_register_network_synonym("ib", "infiniband");
> >
> > 5. ":QUALIFIER" is optional for each NETWORK_TYPE specified, and can be
> > used to disambiguate when a given network type can be reached multiple ways
> > in OMPI. E.g., it can help choose between the openib BTL, the MXM MTL, and
> > the Yalla PML. E.g.:
> >
> > mpirun --enable ib:btl
> > mpirun --enable ib:mtl
> > mpirun --enable ib:yalla
> >
> > That being said, I don't like these names (btl, mtl, yalla) -- they mean
> > nothing to non-OMPI experts. But I like the idea that a QUALIFIER can help
> > choose between the different possibilities.
> >
> > mpirun --enable eth:tcp
> > mpirun --enable eth:usnic
> >
> > These QUALIFIER values are a *little* better, but not much. And usnic is
> > going to get confusing when it starts supporting the OFI MTL tag matching
> > interface (i.e., you'll be able to use usNIC support via the usnic BTL and
> > the OFI MTL).
> >
> > ...thoughts?
> >
> > --
> > Jeff Squyres
> > jsquy...@cisco.com
> > For corporate legal information go to:
> > http://www.cisco.com/web/about/doing_business/legal/cri/
> >
> > _______________________________________________
> > devel mailing list
> > de...@open-mpi.org
> > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
> > Link to this post:
> > http://www.open-mpi.org/community/lists/devel/2015/10/18210.php
> >
>
> _______________________________________________
> devel mailing list
> de...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
> Link to this post:
> http://www.open-mpi.org/community/lists/devel/2015/10/18227.php
> _______________________________________________
> devel mailing list
> de...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
> Link to this post:
> http://www.open-mpi.org/community/lists/devel/2015/10/18228.php
