Dear All, I found that my lustre-2.10.5 with ZFS (either 0.7.9 or 0.7.11) cannot load the "lustre" modules because it cannot load the "obdclass.ko" module. The error message is the following:
# modprobe -v -v obdclass insmod /lib/modules/3.12.72/updates/fs/lustre/obdclass.ko libkmod: INFO ../libkmod/libkmod-module.c:829 kmod_module_insert_module: Failed to insert module '/lib/modules/3.12.72/updates/fs/lustre/obdclass.ko': No such device ERROR: could not insert 'obdclass': No such device libkmod: INFO ../libkmod/libkmod.c:319 kmod_unref: context 0x7fb945d321e0 released Could anyone suggest how to debug ? Thanks very much. T.H.Hsieh On Tue, Sep 25, 2018 at 12:14:00AM +0800, Tung-Han Hsieh wrote: > Dear Nathaniel, > > Thank you very much for your kindly reply. Indeed I modified the > lustre-2.10.5 codes: > > lustre/osd-zfs/osd_object.c > lustre/osd-zfs/osd_xattr.c > > for the declaration: > > inode_timespec_t now; > > Similar to what you have done in your patch. So I can compile > lustre-2.10.5 cleanly with zfs-0.7.11. Sorry I forgot to mention. > > But my problem is still there. Actually I just tried: > > 1. Applying your patch to the original lustre-2.10.5 code, and > recompile with spl-0.7.11 and zfs-0.7.11. But loading "lustre" > module still gives "no such device" error. > > 2. I recompile the original lustre-2.10.5 with spl-0.7.9 and > zfs-0.7.9. They can be compiled cleanly. But again I got the > "no such device" error when loading "lustre" module. > > I am wondering that I must overlooked a trivial step, something > like one (or some) of the utilities in /opt/lustre/sbin/* should > be linked to /sbin/ or /usr/sbin/ .... > > Any suggestions are very appreciated. > > Thank you very much. > > > T.H.Hsieh > > > On Mon, Sep 24, 2018 at 01:21:19PM +0000, Nathaniel Clark wrote: > > Hello Tung-Han, > > > > ZFS 0.7.11 doesn’t compile cleanly with Lustre, yet. > > > > There’s a ticket for adding ZFS 0.7.11 support to lustre: > > https://jira.whamcloud.com/browse/LU-11393 > > > > It has patches for master (pre-2.12) and a separate patch for 2.10. > > > > — > > Nathaniel Clark <[email protected]<mailto:[email protected]>> > > Senior Engineer > > Whamcloud / DDN > > > > On Sep 24, 2018, at 2:15 PM, Tung-Han Hsieh > > <[email protected]<mailto:[email protected]>> wrote: > > > > Dear All, > > > > I am trying to install Lustre version 2.10.5 with ZFS-0.7.11 > > from source code. After compilation and installation, I tried > > to load the "lustre" module, but encountered the following > > error: > > > > # modprobe lustre > > could not load module 'lustre': no such device > > > > My procedure of installation is the following: > > > > 1. Compile vanilla kernel 3.12.72 downloaded from: > > https://mirrors.edge.kernel.org/pub/linux/kernel/v3.x/linux-3.12.72.tar.gz > > > > 2. Compile spl-0.7.11 downloaded from: > > > > https://github.com/zfsonlinux/zfs/releases/download/zfs-0.7.11/spl-0.7.11.tar.gz > > > > with the following steps: > > # ./configure --prefix=/opt/lustre --with-linux=/path/to/linux-3.12.72 > > # make > > # make install > > > > 3. Compile zfs-0.7.11 downloaded from: > > > > https://github.com/zfsonlinux/zfs/releases/download/zfs-0.7.11/zfs-0.7.11.tar.gz > > > > with the following steps: > > # ./configure --prefix=/opt/lustre \ > > --with-linux=/path/to/linux-3.12.72 \ > > --with-spl=/path/to/spl-0.7.11 > > # make > > # make install > > > > 4. Compile lustre downloaded from: > > > > https://downloads.whamcloud.com/public/lustre/lustre-2.10.5/sles12sp3/client/SRPMS/lustre-2.10.5-1.src.rpm > > > > Then I unpack the SRPM by the command: > > # rpm2cpio lustre-2.10.5-1.src.rpm | cpio --extract --make-directories > > > > and compile it by the following: > > # ./configure --prefix=/opt/lustre \ > > --with-linux=/path/to/linux-3.12.72 \ > > --with-spl=/path/to/spl-0.7.11 \ > > --with-zfs=/path/to/zfs-0.7.11 \ > > --with-o2ib=no \ > > --disable-ldiskfs > > # make > > # make install > > > > 5. I have made sure the following settings and utilities are correct: > > - PATH contains /opt/lustre/bin and /opt/lustre/sbin > > - /sbin/mount.lustre exists. > > - /sbin/mount.zfs exists. > > - /usr/sbin/l_getidentity exists. > > - /usr/sbin/ko2iblnd-probe exists. > > - /etc/modprobe.d/lustre.conf contains: > > options lnet networks=tcp > > - /etc/modprobe.d/ko2iblnd.conf contains: > > alias ko2iblnd-opa ko2iblnd > > options ko2iblnd-opa peer_credits=128 peer_credits_hiw=64 credits=1024 > > concurrent_sends=256 ntx=2048 map_on_demand=32 fmr_pool_size=2048 > > fmr_flush_trigger=512 fmr_cache=1 > > install ko2iblnd /usr/sbin/ko2iblnd-probe > > > > Then I tried to run "modprobe lustre", it says "no such device" error. > > > > I tried to replace Lustre-2.10.5 by Lustre-2.9 downloaded from: > > > > https://downloads.whamcloud.com/public/lustre/lustre-2.9.0/sles12sp1/client/SRPMS/lustre-2.9.0-1.src.rpm > > > > and proceed exactly the same installation steps. Everything works fine. > > > > Could anyone suggest me what have I missed for lustre-2.10.5 ? Or suggest > > me how to debug. > > > > Thanks very much. > > > > > > T.H.Hsieh > > _______________________________________________ > > lustre-discuss mailing list > > [email protected] > > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org > > > _______________________________________________ > lustre-discuss mailing list > [email protected] > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org _______________________________________________ lustre-discuss mailing list [email protected] http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
