Now with attachment. Sorry.

Malcolm Cowe wrote:

Andreas Dilger wrote:
On Oct 06, 2008  10:59 -0400, Brian J. Murrell wrote:
  
On Mon, 2008-10-06 at 15:47 +0100, Malcolm Cowe wrote:
    
With respect to the OFED stack used, we are using the latest official
software stack supplied by Voltaire. The reason for this is that there
is more to OFED than just the kernel modules, including many libraries
and tools,
      
None of these should be necessary for Lustre to use I/B.
    

Also very important to note is that if you are changing the IB stack,
then Lustre also needs to be recompiled to work with the new IB stack.

  
Yes. As a matter of fact, you have anticipated a question I have: how does one re-build Lustre in a safe and consistent manner? I'm working through the docs, but I have come across a problem when I try to run "make rpms" in the Lustre source:

make[4]: *** No rule to make target `/usr/src/redhat/BUILD/lustre-1.6.5.1/ldiskfs/Module.symvers', needed by `Module.symvers'.  Stop.

How do I ensure that the build environment that Lustre requires is properly prepared? I could just hoick a soft link to the Module.symvers file in the kernel tree, but that's a little messy.

I've attached a draft copy of the build process to this message. Again, just looking to sanity check the method, since I'm obviously missing something.

I'm going to rebuild the servers today so that I can provide the debug information that Brian requested.
Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.

_______________________________________________
Lustre-discuss mailing list
[email protected]
http://lists.lustre.org/mailman/listinfo/lustre-discuss
  
Regards,

Malcolm.

--

Malcolm Cowe
Solutions Integration Engineer

Sun Microsystems, Inc.
Blackness Road
Linlithgow, West Lothian EH49 7LR UK
Phone: x73602 / +44 1506 673 602
Email: [EMAIL PROTECTED]

--

Malcolm Cowe
Solutions Integration Engineer

Sun Microsystems, Inc.
Blackness Road
Linlithgow, West Lothian EH49 7LR UK
Phone: x73602 / +44 1506 673 602
Email: [EMAIL PROTECTED]
Creating Lustre Packages From Source for RHEL 4.5 AS, 64-bit
============================================================


Pre-requisites
--------------


RHEL 4.5 AS Full Installation

SUN-supplied Linux RDAC kernel modules installed.

System running on Stock RHEL 4.5 AS kernel with RDAC support.

Quilt Source
Lustre Source
RHEL 4.5 AS Kernel SRPM


1. Download and install Quilt:
        http://download.savannah.gnu.org/releases/quilt/

2. Install the RHEL 4.5 AS Kernel SRPM, found on RHEL 4.5 source CD 4.

3. Change to the Red Hat RPM specs directory and extract the full kernel
        source tree. This will also apply Red Hat's patches to the source:

        cd /usr/src/redhat/SPECS
        rpmbuild -bp kernel-2.6.spec 

4. Install the Lustre sources.

        rpm -ivh 
lustre-source-1.6.5.1-2.6.9_67.0.7.EL_lustre.1.6.5.1smp.x86_64.rpm

5. Prepare the kernel source for the Lustre patches:

        cd /usr/src/redhat/BUILD/kernel-2.6.9/linux-2.6.9
        rm -f patches series 
        ln -s 
/usr/src/lustre-1.6.5.1/lustre/kernel_patches/series/2.6-rhel4.series series
        ln -s /usr/src/lustre-1.6.5.1/lustre/kernel_patches/patches .

6. Apply the Lustre patches to the kernel sources:

        cd /usr/src/redhat/BUILD/kernel-2.6.9/linux-2.6.9
        quilt push -av


        Points to note in output:

        Hunk #1 FAILED at 770.
        1 out of 1 hunk FAILED -- rejects in file fs/nfs/nfs4proc.c

        Patch patches/vfs_intent-2.6-rhel4.patch does not apply (enforce with 
-f)

        Full output:

[EMAIL PROTECTED] linux-2.6.9]# quilt push -av
Applying patch patches/vfs_intent-2.6-rhel4.patch
patching file fs/cifs/dir.c
patching file fs/exec.c
patching file fs/inode.c
patching file fs/namei.c
patching file fs/namespace.c
patching file fs/nfs/dir.c
patching file fs/nfs/nfs4proc.c
Hunk #1 FAILED at 770.
1 out of 1 hunk FAILED -- rejects in file fs/nfs/nfs4proc.c
patching file fs/open.c
patching file fs/stat.c
patching file include/linux/dcache.h
patching file include/linux/fs.h
patching file include/linux/mount.h
patching file include/linux/namei.h
Restoring include/linux/dcache.h
Restoring include/linux/namei.h
Restoring include/linux/mount.h
Restoring include/linux/fs.h
Restoring fs/stat.c
Restoring fs/open.c
Restoring fs/namespace.c
Restoring fs/exec.c
Restoring fs/nfs/nfs4proc.c
Restoring fs/nfs/dir.c
Restoring fs/namei.c
Restoring fs/cifs/dir.c
Restoring fs/inode.c
Patch patches/vfs_intent-2.6-rhel4.patch does not apply (enforce with -f)
Restoring include/linux/dcache.h
Restoring include/linux/namei.h
Restoring include/linux/mount.h
Restoring include/linux/fs.h
Restoring fs/stat.c
Restoring fs/open.c
Restoring fs/namespace.c
Restoring fs/exec.c
Restoring fs/nfs/nfs4proc.c
Restoring fs/nfs/dir.c
Restoring fs/namei.c
Restoring fs/cifs/dir.c
Restoring fs/inode.c

7.0. Edit the Makefile in the Kernel Source and ensure that the EXTRAVERSION
        variable matches the suffix of the original kernel. e.g.:

EXTRAVERSION = -55.ELsmp

        If this is not done, the OFED build will fail later on, as it will
        incorrectly identify the kernel and apply the wrong patches. This in 
turn
        leads to a build failure when compiling the "ofa_kernel" rpm.

7.2. Prepare the kernel source.

cd /usr/src/redhat/BUILD/kernel-2.6.9/linux-2.6.9
cp 
/usr/src/lustre-1.6.5.1/lustre/kernel_patches/kernel_configs/kernel-2.6.9-2.6-rhel4-x86_64-smp.config
 .config
make oldconfig || make menuconfig
make include/asm
make include/linux/version.h
make SUBDIRS=scripts

7.5. Backup the old kernel modules.

mv /lib/modules/2.6.9-55.ELsmp /lib/modules/2.6.9-55.ELsmp.orig

8. Build the kernel and the kernel modules.

make bzImage
make modules && make modules_install

9. Copy the kernel image into the /boot partition and create initial RAM-disk
        for the new kernel.

cp arch/x86_64/boot/bzImage /boot/vmlinuz-2.6.9-55.EL_lustre.1.6.5.1smp
cd /boot
mkinitrd initrd-2.6.9-55.EL_lustre.1.6.5.1smp 2.6.9-55.ELsmp

10. Create a new menu entry in GRUB.

11. Reboot the system, select the new kernel entry to load the OS.

12. Build the RDAC modules. Install the mpp initrc image into the boot
        partition and update GRUB. Reboot.

13. Build and install OFED. Reboot when complete.

        ./install.pl -c ofed.conf.Volt -s 
/usr/src/redhat/BUILD/kernel-2.6.9/linux-2.6.9

        cd /usr/src/ofa_kernel-1.3.1
        /usr/share/doc/ofed-docs-1.3.1/create_Module.symvers.sh
        cat /usr/src/ofa_kernel-1.3.1/Module.symvers >> 
/usr/src/redhat/BUILD/kernel-2.6.9/linux-2.6.9/Module.symvers

14. Build Lustre

cd /usr/src/lustre-1.6.5.1/

# Use the following command line:
./configure --with-linux=/usr/src/redhat/BUILD/kernel-2.6.9/linux-2.6.9 
--with-o2ib=/usr/src/ofa_kernel-1.3.1

# This is the command line in the Lustre spec file:
./configure '--with-linux=/usr/src/redhat/BUILD/kernel-2.6.9/linux-2.6.9' 
'--with-o2ib=/usr/src/ofa_kernel-1.3.1' '--enable-liblustre' 
'--enable-liblustre-tests' '--with-lustre-hack' '--with-sockets' 
'--sysconfdir=/etc' '--mandir=/usr/share/man' '--libdir=/usr/lib64' 
--with-lustre-hack --with-sockets  --sysconfdir=/etc --mandir=/usr/share/man 
--libdir=/usr/lib64

make
make rpms

_______________________________________________
Lustre-discuss mailing list
[email protected]
http://lists.lustre.org/mailman/listinfo/lustre-discuss

Reply via email to