Kyle: > > Amit - > The config file you sent looks like it should work for IB without tcp? > Which config options are you setting for the other cases?
All of my IO servers and Metadata server use the same config file, that I posted in my reply. I am not sure what you mean by "which config options am I setting for other cases"? If you mean while running the configure script, then I am not using any other config options apart from what I posted earlier. > > Also, to answer your other question, BMI_ib used to use tcp (not > bmi_tcp) to setup connections and do administrative stuff, but you > should not ever see 'file data' go over your Ethernet connection. Okay. In that case connection setup should work without Ethernet interface being up. Which I don't see happening with the options I used to configure PVFS2. As far as I notice when I use regular copy command, not pvfs2-cp: I see the traffic going through the Ethernet interface. IMPORTANT THING TO NOTE: I have disabled Ethernet interfaces completely on my PVFS2 IO & Metadata servers. Mysterious and interesting...!!! Thank you, Amit > > Kyle > > > > On Thu, Aug 28, 2008 at 1:14 PM, Kumar, Amit H. <[EMAIL PROTECTED]> > wrote: > > My config config options: > > > > ./configure --prefix=/opt/pvfs2 --enable-karma --with- > kernel=/usr/src/linux-2.6 --with-openib=/usr --with-openib- > includes=/usr/include --with-openib-libs=/usr/lib64 CFLAGS=-g --enable- > nptl-workaround=yes --without-bmi-tcp > > > > Thanks!!! > > Amit > > > > -----Original Message----- > > From: Kyle Schochenmaier [mailto:[EMAIL PROTECTED] > > Sent: Thursday, August 28, 2008 1:55 PM > > To: Kumar, Amit H. > > Cc: [email protected] > > Subject: Re: [Pvfs2-developers] RE: PVFS2 seems tcp inclined ? > > > > Amit - > > Can you send a copy of your .conf file so I can take a closer look at > > what order you have things setup. > > I'm interested in the BMI_Modules section and the alias' sections for > > each of your above configurations. > > Also, did you configure/build with `--without-bmi-tcp'? (as > suggested > > for IB users) > > > > ~Kyle > > > > On Thu, Aug 28, 2008 at 12:15 PM, Kumar, Amit H. <[EMAIL PROTECTED]> > wrote: > >> Hello Again ... > >> > >> > >> > >> Can anyone please help me understand this ?? > >> > >> > >> > >> Thank you, > >> > >> Amit > >> > >> > >> > >> From: [EMAIL PROTECTED] > >> [mailto:[EMAIL PROTECTED] On Behalf > Of > >> Kumar, Amit H. > >> Sent: Tuesday, August 26, 2008 2:07 PM > >> To: '[email protected]' > >> Subject: [Pvfs2-developers] PVFS2 seems tcp inclined ? > >> > >> > >> > >> Hi All, > >> > >> > >> > >> On compute nodes when I have both Ethernet and IB interface up data > traffic > >> gets routed through the Ethernet interface as opposed to IB, though > pvfs2 is > >> built with IB, and tcp_bmi disabled. > >> > >> Also to note, when I disable Ethernet interface, it routes the > traffic > >> through IB interface. > >> > >> > >> > >> Is this how it should work ??? Any thoughts please. > >> > >> > >> > >> Thank you, > >> > >> Amit > >> > >> > >> > >> _______________________________________________ > >> Pvfs2-developers mailing list > >> [email protected] > >> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers > >> > >> > > > > > > > > -- > > Kyle Schochenmaier > > > > > > -- > Kyle Schochenmaier > _______________________________________________ > Pvfs2-developers mailing list > [email protected] > http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers _______________________________________________ Pvfs2-developers mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
