Hi Tim,
It looks like ib support isn't getting compiled in to pvfs, even
though you've specified paths and the server appears to link with the
vapi lib. Could you send the config.log and pvfs2-config.h files
generated from the build?
Thanks,
-sam
On Mar 20, 2007, at 3:59 PM, Carlson, Timothy S wrote:
I've built and installed pvfs-2.6.2 on x86_64 (RHEL 4) with Cisco/
Topspin cards and software stack.
topspin-ib-rhel4-3.2.0-118
topspin-ib-mod-rhel4-2.6.9-42.ELsmp-3.2.0-118
./configure --with-kernel=/usr/src/kernels/2.6.9-42.EL-smp-x86_64 --
with-ib=/usr/local/topspin --with-ib-includes=/usr/local/topspin/
include/vapi
Everything builds fine and I install with
make install
make kmod_install
However after configuring the metadata and IO nodes, I try to start
the server and get this in the server log.
[D 03/20 13:25] PVFS2 Server version 2.6.2 starting.
[E 03/20 13:25] src/io/bmi/bmi.c line 1841: Error: no method
available for bmi_ib.
[E 03/20 13:25] [bt] /usr/local/sbin/pvfs2-server [0x433a50]
[E 03/20 13:25] [bt] /usr/local/sbin/pvfs2-server
(BMI_initialize+0x13e) [0x433f1e]
[E 03/20 13:25] [bt] /usr/local/sbin/pvfs2-server [0x40de0d]
[E 03/20 13:25] [bt] /usr/local/sbin/pvfs2-server(main
+0xaa1) [0x40f6a1]
[E 03/20 13:25] [bt] /lib64/tls/libc.so.6(__libc_start_main
+0xdb) [0x39ce01c3fb]
[E 03/20 13:25] [bt] /usr/local/sbin/pvfs2-server
(aio_cancel64+0x16a) [0x40d4aa]
[E 03/20 13:25] Error: BMI_initialize: Protocol not available
[E 03/20 13:25] Error: Could not initialize server interfaces;
aborting.
[E 03/20 13:25] Error: Could not initialize server; aborting.
I saw a similar post in the February archives, but that was for
Infinipath and I didn't see a final answer to the question that was
posted. Any ideas here? The pvfs2-server binary is built against
the topspin libraries.
[EMAIL PROTECTED] tmp]# ldd /usr/local/sbin/pvfs2-server
libdb-4.2.so => /lib64/tls/libdb-4.2.so (0x00000039cf900000)
libpthread.so.0 => /lib64/tls/libpthread.so.0
(0x00000039ce900000)
librt.so.1 => /lib64/tls/librt.so.1 (0x00000039cf100000)
libvapi.so => /usr/local/topspin/lib64/libvapi.so
(0x0000002a95573000)
libmtl_common.so => /usr/local/topspin/lib64/
libmtl_common.so (0x0000002a95693000)
libmosal.so => /usr/local/topspin/lib64/libmosal.so
(0x0000002a95795000)
libmpga.so => /usr/local/topspin/lib64/libmpga.so
(0x0000002a958a4000)
libdl.so.2 => /lib64/libdl.so.2 (0x00000039ce300000)
libc.so.6 => /lib64/tls/libc.so.6 (0x00000039ce000000)
/lib64/ld-linux-x86-64.so.2 (0x00000039cde00000)
libg2c.so.0 => /usr/lib64/libg2c.so.0 (0x0000002a959a6000)
libm.so.6 => /lib64/tls/libm.so.6 (0x00000039ce500000)
The following ib modules are loaded into the running kernel.
[EMAIL PROTECTED] tmp]# lsmod | grep ib
ts_ib_dm_client 31776 1 ts_srp_host
ts_ib_sa_client 44688 2 ts_srp_host,ts_ib_dm_client
ts_ib_client_query 23072 3
ts_srp_host,ts_ib_dm_client,ts_ib_sa_client
ts_ib_cm 59128 1 ts_srp_host
ts_kernel_poll 28496 3
ts_ib_dm_client,ts_ib_client_query,ts_ib_cm
ts_ib_useraccess 18728 0
ts_ib_tavor 34752 3
mod_vip 461096 3 ts_ib_tavor,mod_rhh,mod_thh
mlxsys 166380 4
ts_ib_useraccess,mod_rhh,mod_thh,mod_vip
ts_ib_mad 37144 4
ts_ib_client_query,ts_ib_cm,ts_ib_useraccess,ts_ib_tavor
ts_ib_core 65688 7
ts_srp_host,ts_ib_dm_client,ts_ib_sa_client,ts_ib_cm,ts_ib_useraccess,
ts_ib_tavor,ts_ib_mad
ts_ib_packet_lib 212992 3 ts_ib_tavor,ts_ib_mad,ts_ib_core
ts_kernel_services 25664 11
ts_srp_host,ts_ib_dm_client,ts_ib_sa_client,ts_ib_client_query,ts_ib_c
m,ts_kernel_poll,ts_ib_useraccess,ts_ib_tavor,ts_ib_mad,ts_ib_core,ts_
ib_packet_lib
Tim
Tim Carlson
Voice: (509) 376 3423
Email: [EMAIL PROTECTED]
Pacific Northwest National Laboratory
HPCaNS: High Performance Computing and Networking Services
_______________________________________________
Pvfs2-users mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
_______________________________________________
Pvfs2-users mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
