thanks for the advice. Those modules were not autoloaded, but could be
loaded manually.
I now have this profile:
------------------------------------
$ lsmod |egrep '(ib|rdma)'
ib_umad 12363 0
ib_ucm 12496 0
rdma_ucm 11779 0
ib_uverbs 31902 2 ib_ucm,rdma_ucm
rdma_cm 31030 1 rdma_ucm
ib_cm 40316 2 ib_ucm,rdma_cm
iw_cm 9385 1 rdma_cm
ib_sa 22006 2 rdma_cm,ib_cm
ib_addr 6766 1 rdma_cm
zlib_deflate 21834 1 btrfs
libcrc32c 1244 1 btrfs
ib_mthca 140934 0
ib_mad 41321 4 ib_umad,ib_cm,ib_sa,ib_mthca
ib_core 64935 10
ib_umad,ib_ucm,rdma_ucm,ib_uverbs,rdma_cm,ib_cm,iw_cm,ib_sa,ib_mthca,ib_mad
------------------------------------
and semi-magically, on one of the nodes, ibv_devinfo reports:
------------------------------------
root@pbs3:~
661 $ ibv_devinfo
hca_id: mthca0
transport: InfiniBand (0)
fw_ver: 1.0.800
node_guid: 0006:6a00:9800:6e55
sys_image_guid: 0006:6a00:9800:6e55
vendor_id: 0x066a
vendor_part_id: 25204
hw_ver: 0xA0
board_id: MT_0230000002
phys_port_cnt: 1
port: 1
state: PORT_INIT (2)
max_mtu: 2048 (4)
active_mtu: 512 (2)
sm_lid: 0
port_lid: 0
port_lmc: 0x00
------------------------------------
However, 2 other nodes are still mute - there's obviously some kind of
software drift between them, but this gives me some purchase to figure
other things out.
Still haven't upgraded the firmware, but this allows me to extract the
PSID and get the right FW image.
Thanks!
Harry
On Monday 07 November 2011 06:44:06 Ben England wrote:
> To Harry Mangalam about Gluster/RDMA:
>
> make sure these modules are loaded
>
> # modprobe -v rdma_ucm
> # modprobe -v ib_uverbs
> # modprobe -v ib_ucm
>
> To run the subnet manager
>
> # modprobe -v ib_umad
>
> Make sure libibverbs and (libmlx4 or libmthca) RPMs are installed.
>
> I don't understand why they appropriate modules aren't loaded
> automatically. Could put something in /etc/modprobe.d/ to make
> this happen maybe? Infiniband should not require troubleshooting
> after 5-10 years of development, it should just work.
> _______________________________________________
> Gluster-users mailing list
> [email protected]
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
--
Harry Mangalam - Research Computing, OIT, Rm 225 MSTB, UC Irvine
[ZOT 2225] / 92697 Google Voice Multiplexer: (949) 478-4487
MSTB Lat/Long: (33.642025,-117.844414) (paste into Google Maps)
--
This signature has been OCCUPIED!
_______________________________________________
Gluster-users mailing list
[email protected]
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
