Looks like I forgot to CC pvfs2-dev a message or two ago, I'll put this back on the list.
On Mon, Sep 1, 2008 at 6:30 PM, Kumar, Amit H. <[EMAIL PROTECTED]> wrote: > Kyle: > Sorry for the delayed response. >>>> Also, to answer your other question, BMI_ib used to use tcp (not >>>> bmi_tcp) to setup connections and do administrative stuff, but you >>>> should not ever see 'file data' go over your Ethernet connection. >>> >>> Okay. In that case connection setup should work without Ethernet interface >>> being up. >>> Which I don't see happening with the options I used to configure PVFS2. >>I'm not sure if bmi_ib uses IB to do its connection management, there >>was talk of porting it to use openIB's connection manager but I think >>I was the only one who was serious about doing it, and thus it never >>got done to my knowledge. > > I disabled ethernet interface just to test that, and I found it doesn't use > bmi_ib for connection management. > I didnt think this was implemented yet. (as it was my potential job to do it) >>> >>> As far as I notice when I use regular copy command, not pvfs2-cp: I see the >>> traffic going >>> through the Ethernet interface. IMPORTANT THING TO NOTE: I have disabled >>> Ethernet >>> interfaces completely on my PVFS2 IO & Metadata servers. Mysterious and >>> interesting...!!! >>> >>Just to be clear, when you use pvfs2-cp, you do not get traffic over >>the ethernet device? > > I do get traffic over the ethernet device. Though I noticed one problem with > pvfs2-cp as far as the installation I have. > when I do # pvfs2-cp source local_filesystem_file path_to_pvfs2_FS/. > The pvfs2-cp command fails with a Segmentation Fault. > Where As when I chage directory to PVFS2 filesystem and then copy as shown > below, it succeeds: > [/scratch/pvfs2 #] pvfs2-cp path_to_sourcefilesystem_file ./. > > Any idea ?? Does this sound like a bug ??? Can you provide a backtrace? Also I believe there are a couple fixes to IB that have occured since 2.7.1 that dealt with possible segfaults. Can you please try the latest cvs and see if you get the same segfault issues? Lets start with that if its possible, just to make sure we're not stepping on our own toes here. Kyle > >>Is it possible that you somehow have a bmi_tcp client mounted somewhere? > > No I have double checked it. > >>What version of pvfs2 & kernel are you using, and do you have >>infiniband setup properly? > > PVFS2: version 2.7.1 > Kernel: 2.6.18-53.1.14.el5 > -- Kyle Schochenmaier _______________________________________________ Pvfs2-developers mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
