Chris, Try using /usr/local/ofed/current/src/ofa_kernel/ instead of the version specific one. We were seeing Oops when compiling against the version specific tree. I've been meaning to post to the list about this.
--Shane -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Chris Worley Sent: Friday, February 29, 2008 1:32 PM To: lustre-discuss Subject: Re: [Lustre-discuss] Latest RHEL kernel: won't make mgs module,and ko2iblnd not built for right OFED modules On Fri, Feb 29, 2008 at 8:50 AM, Canon, Richard Shane <[EMAIL PROTECTED]> wrote: > > > Yes. We also had to fix that. Sorry I left that off. In our case, we > created a modified Module.symvers by grep'ing out anything with > drivers/infiniband. Also, OFED will create a Module.symvers for its > symbols. You can append that to the (fixed) Module.symvers so that the > symbols will still be checked. Okay, I found it: /usr/share/doc/ofed-docs-1.2.5.5/create_Module.symvers.sh I've also changed my config line in Lustre to point to the proper modules: ./configure --with-o2ib=/usr/local/ofed/current/src/ofa_kernel-1.2.5.5/ --with-ldiskfs ... that caused a few other Lustre build errors that were easy to fix/hack. To fix the mgs.ko issue, I'm just manually copying the module into /lib/modules. ... now it just panics when I "modprobe mgs" ;) Chris > > > --Shane > > -----Original Message----- > From: [EMAIL PROTECTED] > > [mailto:[EMAIL PROTECTED] On Behalf Of Craig > Prescott > Sent: Friday, February 29, 2008 10:41 AM > To: Chris Worley > Cc: [email protected] > Subject: Re: [Lustre-discuss] Latest RHEL kernel: won't make mgs module, > and ko2iblnd not built for right OFED modules > > > > > Nod - we ran into this issue as well. Your Module.symvers > file for probably has entries in it for the IB modules > included with the RHEL kernel (and not from OFED). > > Here's what we do to avoid this issue (and others) when > building a patched Lustre kernel with OFED: > > 1) Get the source for that kernel (rpmbuild -bp the > src.rpm and copy it somewhere). > 2) Configure that kernel *without* infiniband > (edit the config for that kernel to say > "#CONFIG_INFINIBAND is not set" and get rid > of all the other IB stuff). > 3) patch that kernel for Lustre > 4) build and install that kernel > 5) build ofa-kernel against that kernel and install it > 6) cat <ofa-kernel-dir>/Module.symvers >> > <linux-kernel-dir>/Module.symvers > 7) Build and install Lustre against this kernel and > ofa-kernel. > > Hope that helps, > Craig Prescott > UF HPC Center > > Chris Worley wrote: > > Lustre 1.6.4.2, OFED 1.2.5.5, RHEL kernel 2.6.9.67.0.4. > > > > Configured with: --with-o2ib --with-ldiskfs > > > > mgs.ko is built but not wrapped into modules rpm, with an undefined > > external that should be from the lvfs module (lvfs_check_rdonly). > > > > When ko2iblnd loads, it gets lots of "disagrees about version of > > symbol <some OFED symbol>" messages. I'm guessing this is building > > wrt the ib modules in the kernel rather than the OFED modules I have > > loaded. > > > > There were other patches I needed to make for this kernel (to resolve > > mutex_lock/unlock, __d_rehash, and __d_move... but those were easy to > > fix). > > > > What am I doing wrong? > > > > Thanks, > > > > Chris > > _______________________________________________ > > Lustre-discuss mailing list > > [email protected] > > http://lists.lustre.org/mailman/listinfo/lustre-discuss > > _______________________________________________ > Lustre-discuss mailing list > [email protected] > http://lists.lustre.org/mailman/listinfo/lustre-discuss > _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
