So I did the rebuild with -g and start the debugger and after a few
hundred "n" I figured something was up. I started the server as normal
(no gdb) and it started just fine. I was sure I had done a "make clean"
before trying the IB stuff, but maybe not. Sorry about the false alarm
there and thanks for the quick answers!
My problem now seems to be on the client side. I do a "modprobe pvfs2",
start the client and then try to mount the filesystem.
[EMAIL PROTECTED] tmp]# mount -t pvfs2 ib://compute-0-0:3334/pvfs2-fs
/mnt/pvfs2
mount: Connection refused
The client logs show
[EMAIL PROTECTED] tmp]# pvfs2-ping -m /mnt/pvfs2
(1) Parsing tab file...
(2) Initializing system interface...
(3) Initializing each file system found in tab file: /etc/pvfs2tab...
PVFS2 servers: ib://compute-0-0:3334
Storage name: pvfs2-fs
Local mount point: /mnt/pvfs2
[E 15:24:28.563484] Warning: ib_tcp_client_connect: connect to server
compute-0-0:3334: Connection refused.
[E 15:24:28.563628] Receive immediately failed: Connection refused
[E 15:24:28.563725] msgpair failed, will retry: Connection refused
I'm throwing in my /etc/pvfs2-fs.conf file here
<Defaults>
UnexpectedRequests 50
EventLogging none
LogStamp datetime
BMIModules bmi_ib
FlowModules flowproto_multiqueue
PerfUpdateInterval 1000
ServerJobBMITimeoutSecs 30
ServerJobFlowTimeoutSecs 30
ClientJobBMITimeoutSecs 300
ClientJobFlowTimeoutSecs 300
ClientRetryLimit 5
ClientRetryDelayMilliSecs 2000
</Defaults>
<Aliases>
Alias compute-0-0 ib://compute-0-0:3335
Alias compute-0-1 ib://compute-0-1:3335
Alias compute-0-2 ib://compute-0-2:3335
Alias compute-0-3 ib://compute-0-3:3335
</Aliases>
<Filesystem>
Name pvfs2-fs
ID 1110091704
RootHandle 1048576
<MetaHandleRanges>
Range compute-0-0 4-858993461
</MetaHandleRanges>
<DataHandleRanges>
Range compute-0-0 858993462-1717986919
Range compute-0-1 1717986920-2576980377
Range compute-0-2 2576980378-3435973835
Range compute-0-3 3435973836-4294967293
</DataHandleRanges>
<StorageHints>
TroveSyncMeta yes
TroveSyncData no
</StorageHints>
</Filesystem>
-----Original Message-----
From: Pete Wyckoff [mailto:[EMAIL PROTECTED]
Sent: Tuesday, March 20, 2007 2:43 PM
To: Carlson, Timothy S
Cc: [email protected]
Subject: Re: [Pvfs2-users] error starting server compiled for
infiniband(topspin)
[EMAIL PROTECTED] wrote on Tue, 20 Mar 2007 13:59 -0700:
> I've built and installed pvfs-2.6.2 on x86_64 (RHEL 4) with
> Cisco/Topspin cards and software stack.
>
> topspin-ib-rhel4-3.2.0-118
> topspin-ib-mod-rhel4-2.6.9-42.ELsmp-3.2.0-118
>
> ./configure --with-kernel=/usr/src/kernels/2.6.9-42.EL-smp-x86_64
> --with-ib=/usr/local/topspin
> --with-ib-includes=/usr/local/topspin/include/vapi
>
> Everything builds fine and I install with make install make
> kmod_install
>
> However after configuring the metadata and IO nodes, I try to start
> the server and get this in the server log.
>
> [D 03/20 13:25] PVFS2 Server version 2.6.2 starting.
> [E 03/20 13:25] src/io/bmi/bmi.c line 1841: Error: no method available
> for bmi_ib.
> [E 03/20 13:25] [bt] /usr/local/sbin/pvfs2-server [0x433a50]
> [E 03/20 13:25] [bt]
> /usr/local/sbin/pvfs2-server(BMI_initialize+0x13e) [0x433f1e]
> [E 03/20 13:25] [bt] /usr/local/sbin/pvfs2-server [0x40de0d]
> [E 03/20 13:25] [bt] /usr/local/sbin/pvfs2-server(main+0xaa1)
> [0x40f6a1]
> [E 03/20 13:25] [bt]
> /lib64/tls/libc.so.6(__libc_start_main+0xdb) [0x39ce01c3fb]
> [E 03/20 13:25] [bt]
> /usr/local/sbin/pvfs2-server(aio_cancel64+0x16a) [0x40d4aa] [E 03/20
> 13:25] Error: BMI_initialize: Protocol not available [E 03/20 13:25]
> Error: Could not initialize server interfaces; aborting.
> [E 03/20 13:25] Error: Could not initialize server; aborting.
Recompile with debugging:
make clean
CFLAGS=-g ./configure ...
make
make install
and cd to where your server expects to run. Then do:
gdb --args pvfs2-server -d fs.conf server.conf
b BMI_ib_initialize
run
then use "n" to walk through the function and try to find out where it
is failing. I would have expected an error message, but do not see one
above.
Your build and ldd output look okay. Your fs.conf has lines like:
BMIModules bmi_ib
...
Alias myhost1 ib://myhost1:3335
or there would have been different messages, but check anyway.
If you get a bit closer, let us know and we'll figure out the problem.
-- Pete
P.S. Can you get exchange to turn off html email?
_______________________________________________
Pvfs2-users mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
