It doesn't appear as if you've come up with a solution to this
problem. I too have run into the same set of issues as you appear to
have here, but I think I have resolved them. I am running SLES 10 SP1
but this should also work for you.
first patch lnet/klnds/o2iblnd/o2iblnd.h You'll need to add the
following line somewhere near the top:
#include <linux/pci.h>
Next you'll need to patch lnet/klnds/o2iblnd/o2iblnd.c. ib_create_cq
uses 6 args in OFED-1.2.5.5 but o2iblnd.c only has 5 in the call. The
new line should look like this:
cq = ib_create_cq(cmid->device,
kiblnd_cq_completion, kiblnd_cq_event, conn,
IBLND_CQ_ENTRIES(), 0);
After applying the two changes mentioned above I have been able to
make though 'make rpms' finishes it is a tad chatty with warning
messages.
On May 15, 12:37 pm, Malcolm Cowe <[EMAIL PROTECTED]> wrote:
> Hi Folks,
>
> Having some trouble with building Lustre 1.6.4.3 with OFED 1.2.5.5 on a
> RHEL 4u4 AS server. Could somebody please help me to understand where
> I've gone wrong? Here's what I have done so far:
>
> 1. Install RHEL 4u4 AS (full installation).
>
> 2. Download the Lustre RPMs from sun.com:
>
> e2fsprogs-1.40.4.cfs1-0redhat.x86_64.rpm
> e2fsprogs-devel-1.40.4.cfs1-0redhat.x86_64.rpm
> kernel-lustre-smp-2.6.9-67.0.4.EL_lustre.1.6.4.3.x86_64.rpm
> kernel-lustre-source-2.6.9-67.0.4.EL_lustre.1.6.4.3.x86_64.rpm
> lustre-1.6.4.3-2.6.9_67.0.4.EL_lustre.1.6.4.3smp.x86_64.rpm
> lustre-modules-1.6.4.3-2.6.9_67.0.4.EL_lustre.1.6.4.3smp.x86_64.rpm
> lustre-source-1.6.4.3-2.6.9_67.0.4.EL_lustre.1.6.4.3smp.x86_64.rpm
> plus: lustre-1.6.4.3.tar.gz
>
> 3. Install kernel-lustre-smp and kernel-lustre-source rpms.
> - Change grub to boot from lustre patched kernel by default.
> - Reboot.
>
> 4. Download OFED distribution from openib.org:
>
> OFED-1.2.5.5.tgz
>
> 5. Extract OFED distribution.
>
> 6. Install OFED:
>
> cd OFED-1.2.5.5/
> ./install.sh
> 2) Install OFED Software
> 3) All packages (all of Basic, HPC)
> [accept defaults for everything, configure IPoIB IP address].
>
> 7. Reboot.
>
> 8. Modify "Module.symvers", removing all references to Infiniband
> modules supplied with the kernel distribution. N.B. Could not find this
> file in the lustre kernel source tree, only in the "-obj" tree.
>
> vi
> /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3-obj/x86_64/smp/Module.symvers
>
> 9. Run "/usr/share/doc/ofed-docs-1.2.5.5/create_Module.symvers.sh" and
> append the resulting file to the existing Module.symvers file:
>
> /usr/share/doc/ofed-docs-1.2.5.5/create_Module.symvers.sh
> cat Module.symvers >>
> /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3-obj/x86_64/smp/Module.symvers
>
> 10. Change into the lustre kernel source and edit the Makefile. Change
> "custom" suffix to "smp" in the variable "EXTRAVERSION".
>
> 11. Change into the lustre kernel source and run the setup commands:
>
> cd /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3
> [linux]$ cp /boot/config-`uname -r` .config
> [linux]$ make oldconfig || make menuconfig
> # For 2.6 kernels
> [linux]$ make include/asm
> [linux]$ make include/linux/version.h
> [linux]$ make SUBDIRS=scripts
>
> 12. Extract the lustre source distribution (using lustre-1.6.4.3.tar.gz
> rather than the RPM):
>
> tar zxf lustre-1.6.4.3.tar.gz
>
> 13. Run the configure script:
>
> cd lustre-1.6.4.3/
> ./configure --with-linux=/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3
> --with-o2ib=/usr/src/ofa_kernel
>
> N.B. Cannot include "--with-linux-obj=" option as the configure script
> exits with an error and a recommendation to run "make config" in the
> linux src tree:
>
> checking for
> /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3-obj/include/linux/autoconf.h...
> no
> configure: error: Run make config in
> /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3.
>
> If I do this, the build fails very early on:
>
> [EMAIL PROTECTED] lustre-1.6.4.3]# make
> test -d CVS || exit 0; \
> list=""; for mod in $list; do \
> perl ./build/kabi -v archive $HOME/nonfree $mod || exit $?; \
> done
> make all-recursive
> make[1]: Entering directory `/root/HPC/build/lustre-1.6.4.3'
> Making all in ldiskfs
> make[2]: Entering directory `/root/HPC/build/lustre-1.6.4.3/ldiskfs'
> test -d CVS || exit 0; \
> list=""; for mod in $list; do \
> perl ./build/kabi -v archive $HOME/nonfree $mod || exit $?; \
> done
> make all-recursive
> make[3]: Entering directory `/root/HPC/build/lustre-1.6.4.3/ldiskfs'
> Making all in .
> make[4]: Entering directory `/root/HPC/build/lustre-1.6.4.3/ldiskfs'
> for dir in ldiskfs ; do \
> make sources -C $dir || exit $? ; \
> done
> make[5]: Entering directory `/root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs'
> rm -rf linux-stage linux sources
> mkdir -p linux-stage/fs/ext3 linux-stage/include/linux
> cp /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/acl.c
> /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/balloc.c
> /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/bitmap.c
> /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/dir.c
> /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/file.c
> /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/fsync.c
> /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/hash.c
> /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/ialloc.c
> /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/inode.c
> /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/ioctl.c
> /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/namei.c
> /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/resize.c
> /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/super.c
> /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/symlink.c
> /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/xattr.c
> /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/xattr_security.c
> /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/xattr_trusted.c
> /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/xattr_user.c
> /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/acl.h
> /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/xattr.h
> /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/Makefile
> linux-stage/fs/ext3
> cp /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/include/linux/ext3_fs.h
> /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/include/linux/ext3_fs_i.h
> /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/include/linux/ext3_fs_sb.h
> /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/include/linux/ext3_jbd.h
> linux-stage/include/linux
> Applying ext3 patches: ext3-wantedi-2.6-rhel4.patch iopen-2.6-rhel4.patch
> export_symbols-ext3-2.6-suse.patch ext3-map_inode_page-2.6-suse.patch
> ext3-ea-in-inode-2.6-rhel4.patch export-ext3-2.6-rhel4.patch
> ext3-include-fixes-2.6-rhel4.patch
> ext3-extents-2.6.9-rhel4.patch ext3-extents-fixes-2.6.9-rhel4.patch
> ext3-extents-multiblock-directio-2.6.9-rhel4.patch
> ext3-extents-search-2.6.9-rhel4.patch ext3-extents-sanity-checks.patch
> ext3-mballoc3-core.patch ext3-mballoc3-rhel4.patch
> ext3-nlinks-2.6.9.patch ext3-ialloc-2.6.patch ext3-lookup-dotdot-2.6.9.patch
> ext3-sector_t-overflow-2.6.9-rhel4.patch ext3-check-jbd-errors-2.6.9.patch
> ext3-uninit-2.6.9.patch ext3-nanosecond-2.6-rhel4.patch
> ext3-unlink-race.patch ext3-mmp-2.6-rhel4.patch
> ext3-fiemap-2.6-sles10.patch
> mkdir linux
> Replacing 'ext3' with 'ldiskfs': acl.h xattr.h acl.c balloc.c bitmap.c dir.c
> file.c fsync.c hash.c ialloc.c inode.c ioctl.c namei.c resize.c super.c
> symlink.c xattr.c xattr_security.c xattr_trusted.c xattr_user.c iopen.c
> iopen.h extents.c mballoc.c group.h
> fiemap.h ext3_fs.h ext3_fs_i.h ext3_fs_sb.h ext3_jbd.h ext3_extents.h
> touch sources
> make[5]: Leaving directory `/root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs'
> make CC="gcc -m64" -C /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3
> \
> -f /root/HPC/build/lustre-1.6.4.3/ldiskfs/build/Makefile
> LUSTRE_LINUX_CONFIG=/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/.config \
> LINUXINCLUDE=' -Iinclude $(if $(KBUILD_SRC),-Iinclude2
> -I$(srctree)/include) -include include/linux/autoconf.h' \
> M=/root/HPC/build/lustre-1.6.4.3/ldiskfs -o tmp_include_depends -o scripts -o
> \
> include/config/MARKER modules
> make[5]: Entering directory `/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3'
> LD /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/built-in.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/acl.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/balloc.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/bitmap.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/dir.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/file.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/fsync.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/hash.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/ialloc.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/inode.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/ioctl.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/namei.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/resize.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/super.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/symlink.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/xattr.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/xattr_security.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/xattr_trusted.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/xattr_user.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/iopen.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/extents.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/mballoc.o
> LD [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/ldiskfs.o
> Building modules, stage 2.
> MODPOST
> CC /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/ldiskfs.mod.o
> LD [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/ldiskfs.ko
> make[5]: Leaving directory `/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3'
> make[4]: Leaving directory `/root/HPC/build/lustre-1.6.4.3/ldiskfs'
> Making all in ldiskfs
> make[4]: Entering directory `/root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs'
> cd .. ; /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/scripts/mod/modpost -m
> -i /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/Module.symvers \
> -o /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/Module.symvers
> ldiskfs/ldiskfs.o
> test -d CVS || exit 0; \
> list=""; for mod in $list; do \
> perl ../build/kabi -v archive $HOME/nonfree $mod || exit $?; \
> done
> make[4]: Leaving directory `/root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs'
> make[3]: Leaving directory `/root/HPC/build/lustre-1.6.4.3/ldiskfs'
> make[2]: Leaving directory `/root/HPC/build/lustre-1.6.4.3/ldiskfs'
> Making all in .
> make[2]: Entering directory `/root/HPC/build/lustre-1.6.4.3'
> for dir in ldiskfs lnet lustre ; do \
> make sources -C $dir || exit $? ; \
> done
> make[3]: Entering directory `/root/HPC/build/lustre-1.6.4.3/ldiskfs'
> for ...
>
> read more ยป
_______________________________________________
Lustre-discuss mailing list
[email protected]
http://lists.lustre.org/mailman/listinfo/lustre-discuss
