Hi Jeff, Thank you for your prompt response. I have a query related to MPI_INIT here. What is the underlying transport mechanism does OpenMPI uses to exchange service-id/lid via MPI_INIT, is it TCP/IP socket ?
Thanks & Regards, Ramu Message: 2 > Date: Mon, 22 Aug 2011 17:33:19 -0400 > From: Jeff Squyres <jsquy...@cisco.com> > Subject: Re: [OMPI devel] Regarding Connection establishment in > OpenMPI > To: Open MPI Developers <de...@open-mpi.org> > Message-ID: <2399c470-7f91-49d4-a463-a8994691a...@cisco.com> > Content-Type: text/plain; charset=us-ascii > > On Aug 22, 2011, at 9:35 AM, Bhargava Ramu Kavati wrote: > > > I am trying to explore the details of connection establishment in OpenMPI > using libibcm/librdmacm. > > Note that the IB community has given up on ibcm. Our support of it is > incomplete; I wouldn't look at it as an example. > > > In the code, I could not find how OpenMPI app is getting service-id/lid > of remote node to which it wants to connect. > > In the normal case, we pass that information during MPI_INIT. It's a > global gather / broadcast operation that we refer to as the "modex" (module > exchange). I.e., each openib BTL module instance publishes its address > information in the modex and sends it. Near the end of MPI_INIT, each MPI > process receives the modex broadcast and caches it. > > During connection establishment, an MPI process will look in its modex > cache to find the connection information for the peer process that it wants > to connect to. > > > Also, I did not see any query in the code related to service_record_get > from SA. Can you please desribe what is happening OR Am I missing something > here ? > > IIRC, we don't currently use the SA because of its serialization and other > resource bottlenecks (this is a hand-waving answer; I don't remember the > exact reasons for not using the SA, but there were many discussions between > the MPI and OpenFabrics communities a long time ago. The SA issues were not > resolved to the MPI community's liking, IIRC, but this was a long time ago, > and I don't even work for an IB vendor any more, so I might not be > remembering this correctly...). > > -- > Jeff Squyres > jsquy...@cisco.com > For corporate legal information go to: > http://www.cisco.com/web/about/doing_business/legal/cri/ > > >
