On Thu, Jul 23, 2009 at 05:27:23PM -0400, Mark J. Pearrow wrote: > The short question is "is there a howto for getting Infiniband and Ubuntu > working?"
I've done this and other similar things, it isn't too hard.. I like to use 2.6.30 these days for ConnectX, this is the base kernel for OFED 1.5 and has a few additional fixes over .28 that comes with Jaunty, but you can do that after getting the user space setup. I don't think there is anything fatal in .28.. When using stock kernels it is apparently best not to use OFED and its patches.. You are going to want to find the following tar files: infiniband-diags-1.5.1.tar.gz libibmad-1.3.1.tar.gz libmlx4-1.0.1.tar.gz libibcm-1.0.4.tar.gz libibumad-1.3.1.tar.gz librdmacm-1.0.8.tar.gz libibverbs-1.1.2.tar.gz opensm-3.3.1.tar.gz I usually just grab them from git rather that fussing: git://git.openfabrics.org/~shefty/libibcm.git git://git.kernel.org/pub/scm/libs/infiniband/libibverbs.git git://git.kernel.org/pub/scm/libs/infiniband/libmlx4.git git://git.openfabrics.org/~shefty/librdmacm.git git://git.openfabrics.org/~sashak/management.git [umad, mad, opensm and diags] I like to install the entire set in an opt directory. Do the following repeatedly: cd XXXX ./autogen.sh LDFLAGS=-L/opt/ofa-1.5/lib CPPFLAGS=-I/opt/ofa-1.5/include ./configure --prefix=/opt/ofa-1.5 make make install In this order: libumad libibmad opensm libibverbs (*) libmlx4 (*) libibcm librdmacm infiniband-diags The ones with a * are already available through ubuntu, but mlx4 should be updated to 1.0.1. You can use 'debian/rules binary' instead of the configure/make/make install sequence to produce a new binary .deb for these two. Once complete you'll have a /opt/ofa-1.5/ full of all the tools and libraries you need. Since the stock Jaunty kernel already includes the kernel side and pci driver autoloading does most of the work all you need is to modprobe a few additional things: ib_ucm ib_ipoib rdma_ucm Thats basically it for the core IB. Ah, one trick, depending on various factors you may not get rpath support out of libtool. I forget how to fix this up - You really want to build all of the above using rpath.. But barring that, you can do 'export LD_LIBRARY_PATH=/opt/ofa-1.5/lib' at the very start to make it all work out. The biggest problem was the ubuntu kernel version. You want at least 2.6.30 for ConnectX cards. Jaunty is only .28.. Oh, do the following to setup the ofa-1.5 directory to make it nice and tidy: mkdir -p /opt/ofa-1.5/share/man (cd /opt/ofa-1.5/ && ln -s share/man man) [Why the variance amongst the packages I have not tracked down] There is probably also a way to do the above, but use the included .spec files to make rpms and alien those to .debs but that has always been too much trouble for me. I'm sure lots of people would be grateful to have Debian packaging :) Jason _______________________________________________ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
