I've built and installed pvfs-2.6.2 on x86_64 (RHEL 4) with
Cisco/Topspin cards and software stack.
topspin-ib-rhel4-3.2.0-118
topspin-ib-mod-rhel4-2.6.9-42.ELsmp-3.2.0-118
./configure --with-kernel=/usr/src/kernels/2.6.9-42.EL-smp-x86_64
--with-ib=/usr/local/topspin
--with-ib-includes=/usr/local/topspin/include/vapi
Everything builds fine and I install with
make install
make kmod_install
However after configuring the metadata and IO nodes, I try to start the
server and get this in the server log.
[D 03/20 13:25] PVFS2 Server version 2.6.2 starting.
[E 03/20 13:25] src/io/bmi/bmi.c line 1841: Error: no method available
for bmi_ib.
[E 03/20 13:25] [bt] /usr/local/sbin/pvfs2-server [0x433a50]
[E 03/20 13:25] [bt]
/usr/local/sbin/pvfs2-server(BMI_initialize+0x13e) [0x433f1e]
[E 03/20 13:25] [bt] /usr/local/sbin/pvfs2-server [0x40de0d]
[E 03/20 13:25] [bt] /usr/local/sbin/pvfs2-server(main+0xaa1)
[0x40f6a1]
[E 03/20 13:25] [bt]
/lib64/tls/libc.so.6(__libc_start_main+0xdb) [0x39ce01c3fb]
[E 03/20 13:25] [bt]
/usr/local/sbin/pvfs2-server(aio_cancel64+0x16a) [0x40d4aa]
[E 03/20 13:25] Error: BMI_initialize: Protocol not available
[E 03/20 13:25] Error: Could not initialize server interfaces; aborting.
[E 03/20 13:25] Error: Could not initialize server; aborting.
I saw a similar post in the February archives, but that was for
Infinipath and I didn't see a final answer to the question that was
posted. Any ideas here? The pvfs2-server binary is built against the
topspin libraries.
[EMAIL PROTECTED] tmp]# ldd /usr/local/sbin/pvfs2-server
libdb-4.2.so => /lib64/tls/libdb-4.2.so (0x00000039cf900000)
libpthread.so.0 => /lib64/tls/libpthread.so.0
(0x00000039ce900000)
librt.so.1 => /lib64/tls/librt.so.1 (0x00000039cf100000)
libvapi.so => /usr/local/topspin/lib64/libvapi.so
(0x0000002a95573000)
libmtl_common.so => /usr/local/topspin/lib64/libmtl_common.so
(0x0000002a95693000)
libmosal.so => /usr/local/topspin/lib64/libmosal.so
(0x0000002a95795000)
libmpga.so => /usr/local/topspin/lib64/libmpga.so
(0x0000002a958a4000)
libdl.so.2 => /lib64/libdl.so.2 (0x00000039ce300000)
libc.so.6 => /lib64/tls/libc.so.6 (0x00000039ce000000)
/lib64/ld-linux-x86-64.so.2 (0x00000039cde00000)
libg2c.so.0 => /usr/lib64/libg2c.so.0 (0x0000002a959a6000)
libm.so.6 => /lib64/tls/libm.so.6 (0x00000039ce500000)
The following ib modules are loaded into the running kernel.
[EMAIL PROTECTED] tmp]# lsmod | grep ib
ts_ib_dm_client 31776 1 ts_srp_host
ts_ib_sa_client 44688 2 ts_srp_host,ts_ib_dm_client
ts_ib_client_query 23072 3
ts_srp_host,ts_ib_dm_client,ts_ib_sa_client
ts_ib_cm 59128 1 ts_srp_host
ts_kernel_poll 28496 3
ts_ib_dm_client,ts_ib_client_query,ts_ib_cm
ts_ib_useraccess 18728 0
ts_ib_tavor 34752 3
mod_vip 461096 3 ts_ib_tavor,mod_rhh,mod_thh
mlxsys 166380 4 ts_ib_useraccess,mod_rhh,mod_thh,mod_vip
ts_ib_mad 37144 4
ts_ib_client_query,ts_ib_cm,ts_ib_useraccess,ts_ib_tavor
ts_ib_core 65688 7
ts_srp_host,ts_ib_dm_client,ts_ib_sa_client,ts_ib_cm,ts_ib_useraccess,ts
_ib_tavor,ts_ib_mad
ts_ib_packet_lib 212992 3 ts_ib_tavor,ts_ib_mad,ts_ib_core
ts_kernel_services 25664 11
ts_srp_host,ts_ib_dm_client,ts_ib_sa_client,ts_ib_client_query,ts_ib_cm,
ts_kernel_poll,ts_ib_useraccess,ts_ib_tavor,ts_ib_mad,ts_ib_core,ts_ib_p
acket_lib
Tim
Tim Carlson
Voice: (509) 376 3423
Email: [EMAIL PROTECTED]
Pacific Northwest National Laboratory
HPCaNS: High Performance Computing and Networking Services
_______________________________________________
Pvfs2-users mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
