To my knowledge there has been no pvfs2 support written for the topspin
library?
The error there is saying the server is confused on how to startup all
the IB stuff.
I've also seen the following error when the config files are misconfigured.
[E 03/20 13:25] src/io/bmi/bmi.c line 1841: Error: no method available
for bmi_ib.
I could be wrong though on this, as I'm not familiar with the
topspin/vapi stack. I dont know if anyone else has used the topspin
stack yet here.. It seems to be building.. Maybe pete can shed some
light on this.
Kyle
Carlson, Timothy S wrote:
I've built and installed pvfs-2.6.2 on x86_64 (RHEL 4) with
Cisco/Topspin cards and software stack.
topspin-ib-rhel4-3.2.0-118
topspin-ib-mod-rhel4-2.6.9-42.ELsmp-3.2.0-118
./configure --with-kernel=/usr/src/kernels/2.6.9-42.EL-smp-x86_64
--with-ib=/usr/local/topspin
--with-ib-includes=/usr/local/topspin/include/vapi
Everything builds fine and I install with
make install
make kmod_install
However after configuring the metadata and IO nodes, I try to start
the server and get this in the server log.
[D 03/20 13:25] PVFS2 Server version 2.6.2 starting.
[E 03/20 13:25] src/io/bmi/bmi.c line 1841: Error: no method available
for bmi_ib.
[E 03/20 13:25] [bt] /usr/local/sbin/pvfs2-server [0x433a50]
[E 03/20 13:25] [bt]
/usr/local/sbin/pvfs2-server(BMI_initialize+0x13e) [0x433f1e]
[E 03/20 13:25] [bt] /usr/local/sbin/pvfs2-server [0x40de0d]
[E 03/20 13:25] [bt] /usr/local/sbin/pvfs2-server(main+0xaa1)
[0x40f6a1]
[E 03/20 13:25] [bt]
/lib64/tls/libc.so.6(__libc_start_main+0xdb) [0x39ce01c3fb]
[E 03/20 13:25] [bt]
/usr/local/sbin/pvfs2-server(aio_cancel64+0x16a) [0x40d4aa]
[E 03/20 13:25] Error: BMI_initialize: Protocol not available
[E 03/20 13:25] Error: Could not initialize server interfaces; aborting.
[E 03/20 13:25] Error: Could not initialize server; aborting.
I saw a similar post in the February archives, but that was for
Infinipath and I didn't see a final answer to the question that was
posted. Any ideas here? The pvfs2-server binary is built against the
topspin libraries.
[EMAIL PROTECTED] tmp]# ldd /usr/local/sbin/pvfs2-server
libdb-4.2.so => /lib64/tls/libdb-4.2.so (0x00000039cf900000)
libpthread.so.0 => /lib64/tls/libpthread.so.0
(0x00000039ce900000)
librt.so.1 => /lib64/tls/librt.so.1 (0x00000039cf100000)
libvapi.so => /usr/local/topspin/lib64/libvapi.so
(0x0000002a95573000)
libmtl_common.so => /usr/local/topspin/lib64/libmtl_common.so
(0x0000002a95693000)
libmosal.so => /usr/local/topspin/lib64/libmosal.so
(0x0000002a95795000)
libmpga.so => /usr/local/topspin/lib64/libmpga.so
(0x0000002a958a4000)
libdl.so.2 => /lib64/libdl.so.2 (0x00000039ce300000)
libc.so.6 => /lib64/tls/libc.so.6 (0x00000039ce000000)
/lib64/ld-linux-x86-64.so.2 (0x00000039cde00000)
libg2c.so.0 => /usr/lib64/libg2c.so.0 (0x0000002a959a6000)
libm.so.6 => /lib64/tls/libm.so.6 (0x00000039ce500000)
The following ib modules are loaded into the running kernel.
[EMAIL PROTECTED] tmp]# lsmod | grep ib
ts_ib_dm_client 31776 1 ts_srp_host
ts_ib_sa_client 44688 2 ts_srp_host,ts_ib_dm_client
ts_ib_client_query 23072 3
ts_srp_host,ts_ib_dm_client,ts_ib_sa_client
ts_ib_cm 59128 1 ts_srp_host
ts_kernel_poll 28496 3
ts_ib_dm_client,ts_ib_client_query,ts_ib_cm
ts_ib_useraccess 18728 0
ts_ib_tavor 34752 3
mod_vip 461096 3 ts_ib_tavor,mod_rhh,mod_thh
mlxsys 166380 4 ts_ib_useraccess,mod_rhh,mod_thh,mod_vip
ts_ib_mad 37144 4
ts_ib_client_query,ts_ib_cm,ts_ib_useraccess,ts_ib_tavor
ts_ib_core 65688 7
ts_srp_host,ts_ib_dm_client,ts_ib_sa_client,ts_ib_cm,ts_ib_useraccess,ts_ib_tavor,ts_ib_mad
ts_ib_packet_lib 212992 3 ts_ib_tavor,ts_ib_mad,ts_ib_core
ts_kernel_services 25664 11
ts_srp_host,ts_ib_dm_client,ts_ib_sa_client,ts_ib_client_query,ts_ib_cm,ts_kernel_poll,ts_ib_useraccess,ts_ib_tavor,ts_ib_mad,ts_ib_core,ts_ib_packet_lib
Tim
Tim Carlson
Voice: (509) 376 3423
Email: [EMAIL PROTECTED]
Pacific Northwest National Laboratory
HPCaNS: High Performance Computing and Networking Services
!DSPAM:46004c76230776908510162!
------------------------------------------------------------------------
_______________________________________________
Pvfs2-users mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
!DSPAM:46004c76230776908510162!
--
Kyle Schochenmaier
[EMAIL PROTECTED]
Research Assistant, Dr. Brett Bode
AmesLab - US Dept.Energy
Scalable Computing Laboratory
_______________________________________________
Pvfs2-users mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
