On Tue, 29 Jan 2008, Pawel Dziekonski wrote:
> > On Mon, 28 Jan 2008 at 10:14:22AM -0500, James Lentini wrote: > > > > > > On Sat, 26 Jan 2008, Pawel Dziekonski wrote: > > > > > I pulled Tom's tree from new url and build a kernel. > > > > If you enabled support for INFINIBAND drivers (IB and iWARP support) > > and NFS client/server support, the kernel should be ready to go (run > > "grep RDMA /your_kernel_sources/.config" to confirm that > > CONFIG_SUNRPC_XPRT_RDMA is either m or y). > > > > NFS/RDMA doesn't require OFED be installed. OFED is a release of the > > Linux kernel sources and some userspace libraries/tools. If you are > > > > then I downloaded OFED from > > > http://www.mellanox.com/downloads/NFSoRDMA/OFED-1.2-NFS-RDMA.gz, > > > > I don't know what the above URL contains. The latest code is in Tom > > Tucker's tree (and now NFS server maintainer Bruce Fields tree). It is > > > hi, > > back to subject on a proper mailing list. > > I have a >3 year experience with mellanox hardware and IBGold so I > basically know what OFED is all about. up to now i was only using > IBGold since IB drivers appeared in kernel pretty recently. You'll want to use the mainline kernel's IB drivers for NFS/RDMA. We've been developing the NFS/RDMA software on the OpenFabrics (aka OpenIB) code since it was merged into 2.6.10 in Dec 2004. > currently I have new hardware. I'm running Tom's kernel and already > did some MPI tests. SDP is not working, probably because sdp kernel > modules where not build. ;) I understand that those modules are only > available from ofa-kernel. please correct me if i'm wrong. Correct. SDP has never been submitted to mainline Linux. > system is Scientic Linux 4.5, which is supposed to be a fully > compatible RH4 clone. hardware is Supermicro mobos with Mellanox > MT25204 and Flextronisc switch. > > error log from ofa-kernel build: Is your goal to build a kernel with an NFS/RDMA server? If so, the kernel sources from Tom Tucker's git tree are the ones you want, not the old OFED 1.2-based packages which are out of date. Did you try setting up the NFS/RDMA server on the kernel used for your MPI tests above? > > > make[1]: Entering directory `/usr/src/ib/xprt-switch-2.6' > > > test -e include/linux/autoconf.h -a -e include/config/auto.conf || ( > > > \ > > > echo; \ > > > echo " ERROR: Kernel configuration is invalid."; \ > > > echo " include/linux/autoconf.h or include/config/auto.conf are > > > missing."; \ > > > echo " Run 'make oldconfig && make prepare' on kernel src to fix > > > it."; \ > > > echo; \ > > > /bin/false) > > > > > > obviously, doing 'make oldconfig && make prepare' does not help. > > > anyway, above mentioned files do exist: > > > > > > # ls -la > > > /usr/src/ib/xprt-switch-2.6/{include/linux/autoconf.h,include/config/auto.conf} > > > -rw-r--r-- 1 root root 10156 Jan 25 17:42 > > > /usr/src/ib/xprt-switch-2.6/include/config/auto.conf > > > -rw-r--r-- 1 root root 14733 Jan 25 17:42 > > > /usr/src/ib/xprt-switch-2.6/include/linux/autoconf.h > > > > > > despite of above, compilation continues but fails with: > > > > > > gcc > > > -Wp,-MD,/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/infiniband/core/.mad.o.d > > > -nostdinc -isystem /usr/lib/gcc/x86_64-redhat-linux/3.4.6/include > > > -D__KERNEL__ -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/include > > > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/infiniband/include > > > -Iinclude -include include/linux/autoconf.h -include > > > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/include/linux/autoconf.h -Wall > > > -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing > > > -fno-common -Werror-implicit-function-declaration -Os -m64 > > > -mno-red-zone -mcmodel=kernel -pipe -Wno-sign-compare > > > -fno-asynchronous-unwind-tables -funit-at-a-time -mno-sse -mno-mmx > > > -mno-sse2 -mno-3dnow -maccumulate-outgoing-args -DCONFIG_AS_CFI=1 > > > -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -fomit-frame-pointer > > > -Wdeclaration-after-statement -DMODULE -D"KBUILD_STR(s)=#s" > > > -D"KBUILD_BASENAME=KBUILD_STR(mad)" > > > -D"KBUILD_MODNAME=KBUILD_STR(ib_mad)" -c -o > > > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/infiniband/core/.! tmp > _mad.o /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/infiniband/core/mad.c > > > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/infiniband/core/mad.c: In > > > function `ib_mad_init_module': > > > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/infiniband/core/mad.c:2966: > > > error: too many arguments to function `kmem_cache_create' > > > make[4]: *** > > > [/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/infiniband/core/mad.o] > > > Error 1 > > > make[3]: *** > > > [/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/infiniband/core] Error 2 > > > make[2]: *** [/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/infiniband] > > > Error 2 > > > make[1]: *** [_module_/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2] Error 2 > > > make[1]: Leaving directory `/usr/src/ib/xprt-switch-2.6' > > > make: *** [kernel] Error 2 > > > error: Bad exit status from /var/tmp/rpm-tmp.3877 (%install) > > > > full log: > > > https://cefeid.wcss.wroc.pl/d/tmp/OFED.build.32122.log > > thanks in advance for any help, P > > > -- > Pawel Dziekonski <[EMAIL PROTECTED]> > Wroclaw Centre for Networking & Supercomputing, HPC Department > Politechnika Wr., pl. Grunwaldzki 9, bud. D2/101, 50-377 Wroclaw, POLAND > phone: +48 71 3202043, fax: +48 71 3225797, http://www.wcss.wroc.pl > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > nfs-rdma-devel mailing list > [EMAIL PROTECTED] > https://lists.sourceforge.net/lists/listinfo/nfs-rdma-devel > _______________________________________________ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
