Hello Anjana, I can confirm that this setup works (ZFS-MGS/MDT or LDFISKFS-MGS/MDT and ZFS-OSS/OST)
I used a Cent OS 6.4 build: 2.4.0-RC2-gd3f91c4-PRISTINE-2.6.32-358.6.2.el6_lustre.g230b174.x86_64 and the Lustre Packages from http://downloads.whamcloud.com/public/lustre/latest-feature-release/el6/server/RPMS/x86_64/ ZFS is downloaded from ZOL and compiled/installed. SPL: Loaded module v0.6.2-1 SPL: using hostid 0x00000000 ZFS: Loaded module v0.6.2-1, ZFS pool version 5000, ZFS filesystem version 5 I first run in the same problem: mkfs.lustre --fsname=lustrefs --reformat --ost --backfstype=zfs ..... mkfs.lustre FATAL: unable to prepare backend (22) mkfs.lustre: exiting with 22 (Invalid argument) and saw that ZFS libraries in /usr/local/lib where not known to Cent OS 6.4. A quick: echo "/usr/local/lib" >> /etc/ld.so.conf.d/zfs.conf echo "/usr/local/lib64" >> /etc/ld.so.conf.d/zfs.conf ldconfig solved the problem. (LDISKFS) mkfs.lustre --reformat --mgs /dev/sda16 mkfs.lustre --reformat --fsname=zlust --mgsnode=10.16.0.104@o2ib0 --mdt --index=0 /dev/sda5 (ZFS) mkfs.lustre --reformat --mgs --backfstype=zfs mgs/mgs /dev/sda16 mkfs.lustre --reformat --fsname=zlust --mgsnode=10.16.0.104@o2ib0 --mdt --index=0 --backfstype=zfs mdt0/mdt0 /dev/sda5 is working fine. The OSS/OST is a debian wheezy box with 70 disks JBOD and kernel 3.6.11-lustre-tstibor-build with patch series 3.x-fc18.series and SPL/ZFS v0.6.2-1 Best, Thomas On 10/08/2013 05:40 PM, Anjana Kar wrote: > The git checkout was on Sep. 20. Was the patch before or after? > > The zpool create command successfully creates a raidz2 pool, and mkfs.lustre > does not complain, but > > [root@cajal kar]# zpool list > NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT > lustre-ost0 36.2T 2.24M 36.2T 0% 1.00x ONLINE - > > [root@cajal kar]# /usr/sbin/mkfs.lustre --fsname=cajalfs --ost > --backfstype=zfs --index=0 --mgsnode=10.10.101.171@o2ib lustre-ost0 > > [root@cajal kar]# /sbin/service lustre start lustre-ost0 > lustre-ost0 is not a valid lustre label on this node > > I think we'll be splitting up the MDS and OSTs on 2 nodes as some of you > said > there could be other issues down the road, but thanks for all the good > suggestions. > > -Anjana > > On 10/07/2013 07:24 PM, Ned Bass wrote: >> I'm guessing your git checkout doesn't include this commit: >> >> * 010a78e Revert "LU-3682 tunefs: prevent tunefs running on a mounted device" >> >> It looks like the LU-3682 patch introduced a bug that could cause your issue, >> so its reverted in the latest master. >> >> Ned >> >> On Mon, Oct 07, 2013 at 04:54:13PM -0400, Anjana Kar wrote: >>> On 10/07/2013 04:27 PM, Ned Bass wrote: >>>> On Mon, Oct 07, 2013 at 02:23:32PM -0400, Anjana Kar wrote: >>>>> Here is the exact command used to create a raidz2 pool with 8+2 drives, >>>>> followed by the error messages: >>>>> >>>>> mkfs.lustre --fsname=cajalfs --reformat --ost --backfstype=zfs >>>>> --index=0 --mgsnode=10.10.101.171@o2ib lustre-ost0/ost0 raidz2 >>>>> /dev/sda /dev/sdc /dev/sde /dev/sdg /dev/sdi /dev/sdk /dev/sdm >>>>> /dev/sdo /dev/sdq /dev/sds >>>>> >>>>> mkfs.lustre FATAL: Invalid filesystem name /dev/sds >>>> It seems that either the version of mkfs.lustre you are using has a >>>> parsing bug, or there was some sort of syntax error in the actual >>>> command entered. If you are certain your command line is free from >>>> errors, please post the version of lustre you are using, or report the >>>> bug in the Lustre issue tracker. >>>> >>>> Thanks, >>>> Ned >>> For building this server, I followed steps from the walk-thru-build* >>> for Centos 6.4, >>> and added --with-spl and --with-zfs when configuring lustre.. >>> *https://wiki.hpdd.intel.com/pages/viewpage.action?pageId=8126821 >>> >>> spl and zfs modules were installed from source for the lustre 2.4 kernel >>> 2.6.32.358.18.1.el6_lustre2.4 >>> >>> Device sds appears to be valid, but I will try issuing the command >>> using by-path >>> names.. >>> >>> -Anjana > _______________________________________________ > Lustre-discuss mailing list > [email protected] > http://lists.lustre.org/mailman/listinfo/lustre-discuss
smime.p7s
Description: S/MIME Cryptographic Signature
_______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
