Jason, The prebuilt server-side Lustre packages from Whamcloud are built against RHEL/CentOS kernel sources with kernel-ib active in them. This means that any of the Lustre prebuilt server packages are already tied to RHEL's kernel-ib.
To accomplish your stated goal you'll have to start with a non Whamcloud, stock kernel (plus headers, devel, etc). Then compile/install the OFED version of your choice. Once you have that you can build Lustre from source where it will compile against OFED and the installed kernel. --Jeff --------------------------- Jeff Johnson Co-Founder Aeon Computing [email protected] www.aeoncomputing.com t: 858-412-3810 x101 f: 858-412-3845 4170 Morena Boulevard, Suite D - San Diego, CA 92117 /* Follow us on Twitter - @AeonComputing */ On 12/28/12 3:54 PM, Jason Brooks wrote: > Hello, > > I am having trouble installing the server modules for lustre 2.1.4 and > use mellanox's OFED distribution so we may use infiniband. Would you > folks look at my procedure and results below and let me know what you > think? Thanks very much! > > The mellanox ofed installation builds and installs some kernel modules > too, so I used this method to ensure OFED compiled against the correct > kernel. This is on centos 6.3. > > 1. download all lustre rpms from whamcloud > 2. install kernel, kernel-firmware, kernel-headers, and kernel-devel > 1. in this case, it's the rpm files with > "2.6.32-279.14.1.el6_lustre.x86_64" in their name > 3. reboot into this lustre kernel > 4. install the remaining rpms > 5. download ofed from mellanox > "MLNX_OFED_LINUX-1.5.3-3.1.0-rhel6.3-x86_64.iso" > 1. build mellanox ofed bits using the lustre kernel and > kernel-devel info > 2. install mellanox ofed > 6. reboot > 7. upon reboot, if I do NOT have o2ib3 in my lnet networks > parameters, I can modprobe lnet and lustre. > 8. if I DO have o2ib3 present in the lnet parameters, running > modprobe lustre gets me: > > ib/modules/2.6.32-279.14.1.el6_lustre.x86_64/updates/kernel/fs/lustre/fld.ko): > > Input/output error > WARNING: Error inserting fid > (/lib/modules/2.6.32-279.14.1.el6_lustre.x86_64/updates/kernel/fs/lustre/fid.ko): > > Input/output error > WARNING: Error inserting mdc > (/lib/modules/2.6.32-279.14.1.el6_lustre.x86_64/updates/kernel/fs/lustre/mdc.ko): > > Input/output error > WARNING: Error inserting osc > (/lib/modules/2.6.32-279.14.1.el6_lustre.x86_64/updates/kernel/fs/lustre/osc.ko): > > Input/output error > WARNING: Error inserting lov > (/lib/modules/2.6.32-279.14.1.el6_lustre.x86_64/updates/kernel/fs/lustre/lov.ko): > > Input/output error > FATAL: Error inserting lustre > (/lib/modules/2.6.32-279.14.1.el6_lustre.x86_64/updates/kernel/fs/lustre/lustre.ko): > > Input/output error > > > dmesg shows: > ko2iblnd: disagrees about version of symbol ib_fmr_pool_unmap > ko2iblnd: Unknown symbol ib_fmr_pool_unmap > ko2iblnd: disagrees about version of symbol ib_create_cq > ko2iblnd: Unknown symbol ib_create_cq > … > > > > > > > > _______________________________________________ > Lustre-discuss mailing list > [email protected] > http://lists.lustre.org/mailman/listinfo/lustre-discuss _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
