Or, are you proposing that rdma_cm try to separate 2 cases. One where 2 sides each trying to set up a connection to another side, vs. where 2 sides are trying to set up 1 connection but each side issuing a connection request? Isn't it easier to handle in MPI which has a unique rank so only one side issues a connection request? Thanks,
Arkady Kanevsky email: [EMAIL PROTECTED] Network Appliance Inc. phone: 781-768-5395 1601 Trapelo Rd. - Suite 16. Fax: 781-895-1195 Waltham, MA 02451 central phone: 781-768-5300 > -----Original Message----- > From: Or Gerlitz [mailto:[EMAIL PROTECTED] > Sent: Wednesday, December 19, 2007 6:34 AM > To: Sean Hefty > Cc: OpenFabrics General > Subject: Re: [ofa-general] peer to peer connections support > > Sean Hefty wrote: > > Peer to peer connection was never fully implemented in the > ib_cm. I > > don't think it would be that hard to implement at that > level, and it > > shouldn't require API changes. > > With you below comment of "CM needs to know the connection > model selected by the app" I am somehow confused. With > reading your other comments, I see two options here based on > whether the implementation differentiate between peer-to-peer > SIDs to client/server SIDs: > > if there's no difference, then also in the peer-to-peer > model, the application must first tell the CM to listen on a > SID and its up to the CM to break the symmetry and decide who > sends the REP and who ignores the REQ. > > if there is a diff, then peer-to-peer SIDs are in a different > domain then client/server SIDs. > > > Support at the rdma_cm level may require an API change. There's no > > easy way for the rdma_cm to know if it should invoke the IB > > peer-to-peer connection model. I'm not even sure how one > peer would > > know the other peer's port number, unless well known ports > are used on both sides. > > Why there should be a difference between the rdma-cm to the > cm? if in the cm you have a model without API change, > wouldn't it apply also to the rdma-cm? > > >> Such support would be useful in symmetric schemes such as > MPIs that > >> open connections on demand and more applications where > each party can > >> both accept and initiate connections. For example, I > understand that > >> some work is done now at the open mpi community to use the > rdma-cm as > >> a possible channel for connection establishment. > > > I would need to better understand the expected usage model, > like how > > the peers find each other, but this is something that could > be added if needed. > > I think that in the MPI world each rank gets a SID from the > local CM and they exchange the SIDs out-of-band, then > connections are opened. If its a connection-on-demand scheme, > then when ever the rank process calls > mpi_send() to peer for which the local MPI library does not > have a connection, it tries to connect. So if this happens > "at once" between some pair of ranks, there should be a way > to form one connection out of these two connecting requests. > My thinking/motivation is that support of this scheme should > be in the IB stack (cm and rdma-cm) level and not in the > specific MPI implementation level. > > Jeff, Jon, any comments? > > Or. > > > _______________________________________________ > general mailing list > [email protected] > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > _______________________________________________ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
