Hi Durga as an alternative you could implement a libfabric provider for your network. In theory, if you can implement the reliable datagram endpoint type on your network and a tag matching mechanism, you could then just use the ofi mtl and not have to do much if anything in open mpi or mpich etc.
https://github.com/ofiwg/libfabric You may also want to see if the open ucx tl model might work for your network. It may be less work than implementing a libfabric provider. good luck Howard ---------- sent from my smart phonr so no good type. Howard On Feb 4, 2016 6:00 AM, "Jeff Squyres (jsquyres)" <jsquy...@cisco.com> wrote: > +1 on what Gilles said. :-) > > Check out this part of the v1.10 README file: > > https://github.com/open-mpi/ompi-release/blob/v1.10/README#L585-L625 > > Basically: > > - PML is the back-end to functions like MPI_Send and MPI_Recv. > - The ob1 PML uses BTL plugins in a many-of-many relationship to > potentially utilize multiple networks. > - The cm PML uses matching-style network APIs in CM plugins to utilize a > single underlying network. > - The yalla PML was written by Mellanox as a replacement for cm and ob1, > in that it directly utilizes the MXM network library without going through > any of the abstractions in ob1 and cm. It was written at a time when cm > was not well optimized, and basically just added a latency penalty before > dispatching to the underlying MTL module. Since then, cm has been > optimized such that its abstraction penalty before invoking the underlying > MTL module is negligible. > > So the question really comes down to: > > - if you have a network stack API that does MPI-style matching, you should > write an MTL. > - if not, you should write a BTL > > Does that help? > > > > On Feb 4, 2016, at 2:29 AM, Gilles Gouaillardet <gil...@rist.or.jp> > wrote: > > > > Durga, > > > > did you confuse PML and MTL ? > > > > basically, a BTL (Byte Transport Layer ?) is used with "primitive" > interconnects that can only send bytes. > > (e.g. if you need to transmit a tagged message, it is up to you > send/recv the tag and manually match the tag on the receiver side so you > can put the message into the right place) > > on the other hand, MTL (Message Transport Layer ?) can be used with more > advanced interconnects, that can "natively" send/recv (tagged) messages. > > > > for example, with infiniband, you can use the openib BTL, or the mxm MTL > > (note the openib BTL only requires the free ibverbs libraries > > and mxm MTL requires proprietary extensions provided by mellanox) > > > > a good starting point is the video Jeff posted at > https://www.open-mpi.org/video/?category=internals > > > > Cheers, > > > > Gilles > > > > On 2/4/2016 2:20 PM, dpchoudh . wrote: > >> Hi developers > >> > >> I am trying to add support for a new (proprietary) RDMA capable fabric > >> to OpenMPI and have the following question: > >> > >> As I understand, some networks are implemented as a PML framework and > >> some are implemented as a BTL framework. It seems there is even > >> overlap as Myrinet seems to exist in both. > >> > >> My question is: what is the difference between these two frameworks? > >> When adding support for a new fabric, what factors one should consider > >> when choosing between one type of framework over the other? > >> > >> And, with apologies for asking a summary question: is there any kind > >> of documentation and/or book that explains all the internal details of > >> the implementation (which looks little like voodoo to a newcomer like > >> me)? > >> > >> Thanks for your help. > >> > >> Durga Choudhury > >> > >> Life is complex. It has real and imaginary parts. > >> _______________________________________________ > >> devel mailing list > >> de...@open-mpi.org > >> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel > >> Link to this post: > http://www.open-mpi.org/community/lists/devel/2016/02/18544.php > >> > > > > _______________________________________________ > > devel mailing list > > de...@open-mpi.org > > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel > > Link to this post: > http://www.open-mpi.org/community/lists/devel/2016/02/18545.php > > > -- > Jeff Squyres > jsquy...@cisco.com > For corporate legal information go to: > http://www.cisco.com/web/about/doing_business/legal/cri/ > > _______________________________________________ > devel mailing list > de...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel > Link to this post: > http://www.open-mpi.org/community/lists/devel/2016/02/18549.php >
