Re: [lustre-discuss] Lustre Installation Question: 158-c: Can't load module 'osd-zfs'

2023-06-30 Thread Jeff Johnson 
Yao,

Glad you have the correct modules working now. Can you explain why you are
employing the virtual disk driver with osd-zfs?

Capacities with Lustre-ZFS are estimated as zfs has functions like
compression that can change "capacity". As ZFS volumes are used and filled
as Lustre volumes their capacities will be reported more accurately as
objects are allocated to storage.

Running a Lustre command from a client `lfs df -h` will give a better view
of Lustre capacities from the client perspective.

Your use of dev/vda may be adding obscurity and I'm not sure why you would
be adding that.

--Jeff



--Jeff


On Thu, Jun 29, 2023 at 10:33 PM Yao Weng  wrote:

> Hi Jeff:
> Thank you very much ! I install lustre-zfs-dkms and lustre client can
> mount lustre filesystem. However, the size does not adds-up.
> I have
> - msg
>
>  sudo mkfs.lustre --mgs  --reformat --backfstype=zfs --fsname=lustre  
> lustre-mgs/mgs
> /dev/vda2
>
>
>Permanent disk data:
>
> Target: MGS
>
> Index:  unassigned
>
> Lustre FS:  lustre
>
> Mount type: zfs
>
> Flags:  0x64
>
>   (MGS first_time update )
>
> Persistent mount opts:
>
> Parameters:
>
> mkfs_cmd = zpool create -f -O canmount=off lustre-mgs /dev/vda2
>
> mkfs_cmd = zfs create -o canmount=off  lustre-mgs/mgs
>
>   xattr=sa
>
>   dnodesize=auto
>
> Writing lustre-mgs/mgs properties
>
>   lustre:version=1
>
>   lustre:flags=100
>
>   lustre:index=65535
>
>   lustre:fsname=lustre
>
>   lustre:svname=MGS
>
>
> - mdt
>
>  sudo mkfs.lustre --mdt --reformat --backfstype=zfs --fsname=lustre
> --index=0  --mgsnode=10.34.0.103@tcp0 lustre-mdt0/mdt0 /dev/vda2
>
>
>Permanent disk data:
>
> Target: lustre:MDT
>
> Index:  0
>
> Lustre FS:  lustre
>
> Mount type: zfs
>
> Flags:  0x61
>
>   (MDT first_time update )
>
> Persistent mount opts:
>
> Parameters: mgsnode=10.34.0.103@tcp
>
> mkfs_cmd = zpool create -f -O canmount=off lustre-mdt0 /dev/vda2
>
> mkfs_cmd = zfs create -o canmount=off  lustre-mdt0/mdt0
>
>   xattr=sa
>
>   dnodesize=auto
>
> Writing lustre-mdt0/mdt0 properties
>
>   lustre:mgsnode=10.34.0.103@tcp
>
>   lustre:version=1
>
>   lustre:flags=97
>
>   lustre:index=0
>
>   lustre:fsname=lustre
>
>   lustre:svname=lustre:MDT
>
>
> - ost
>
> sudo mkfs.lustre --ost --reformat --backfstype=zfs --fsname=lustre
> --index=0  --mgsnode=10.34.0.103@tcp0 lustre-ost0/ost0 /dev/vda2
>
>
>Permanent disk data:
>
> Target: lustre:OST
>
> Index:  0
>
> Lustre FS:  lustre
>
> Mount type: zfs
>
> Flags:  0x62
>
>   (OST first_time update )
>
> Persistent mount opts:
>
> Parameters: mgsnode=10.34.0.103@tcp
>
> mkfs_cmd = zpool create -f -O canmount=off lustre-ost0 /dev/vda2
>
> mkfs_cmd = zfs create -o canmount=off  lustre-ost0/ost0
>
>   xattr=sa
>
>   dnodesize=auto
>
>   recordsize=1M
>
> Writing lustre-ost0/ost0 properties
>
>   lustre:mgsnode=10.34.0.103@tcp
>
>   lustre:version=1
>
>   lustre:flags=98
>
>   lustre:index=0
>
>   lustre:fsname=lustre
>
>   lustre:svname=lustre:OST
>
>
>
> I have 51G for /dev/vda
>
> df -H /dev/vda2
>
> Filesystem  Size  Used Avail Use% Mounted on
>
> devtmpfs 51G 0   51G   0% /dev
>
> On my client node,
>
> sudo mount -t lustre 10.34.0.103@tcp0:/lustre /mnt
>
> However, the size is only 25M, shouldn't it be 51G ?
>
> df -H /mnt
>
> Filesystem   Size  Used Avail Use% Mounted on
>
> 10.34.0.103@tcp:/lustre   25M  3.2M   19M  15% /mnt
>
> Thanks
> Yao
>
> On Wed, Jun 28, 2023 at 12:22 PM Jeff Johnson <
> jeff.john...@aeoncomputing.com> wrote:
>
>> Yao,
>>
>> Either add the required kernel-{devel,debuginfo} so the osd-ldiskfs
>> kernel modules can be built against your kernel or remove lustre-all-dkms
>> package and replace with lustre-zfs-dkms and build ZFS only Lustre modules.
>>
>> --Jeff
>>
>>
>> On Wed, Jun 28, 2023 at 8:45 AM Yao Weng  wrote:
>>
>>> I have error when installing lustre-all-dkms-2.15.3-1.el8.noarch
>>>
>>> Loading new lustre-all-2.15.3 DKMS files...
>>>
>>> Deprecated feature: REMAKE_INITRD (/usr/src/lustre-all-2.15.3/dkms.conf)
>>>
>>> Building for 4.18.0-477.15.1.el8_8.x86_64
>>> 4.18.0-477.10.1.el8_lustre.x86_64
>>>
>>> Building initial module for 4.18.0-477.15.1.el8_8.x86_64
>>>
>>> Deprecated feature: REMAKE_INITRD
>>> (/var/lib/dkms/lustre-all/2.15.3/source/dkms.conf)
>>>
>>> realpath: /var/lib/dkms/spl/2.1.12/source: No such file or directory
>>>
>>> realpath: /var/lib/dkms/spl/kernel-4.18.0-477.15.1.el8_8.x86_64-x86_64:
>>> No such file or directory
>>>
>>> configure: WARNING:
>>>
>>>
>>> Disabling ldiskfs support because complete ext4 source does not exist.
>>>
>>>
>>> If you are building using kernel-devel packages and require ldiskfs
>>>
>>> server support then ensure that the matching kernel-debuginfo-common
>>>
>>> and kernel-debuginfo-common- packages are installed.
>>>
>>>
>>> awk: fatal: cannot open file
>>> `/var/lib/dkms/lustre-all/2.15.3/build/_lpb/Makefile.compile.lustre' for
>>> reading (No

Re: [lustre-discuss] Lustre Installation Question: 158-c: Can't load module 'osd-zfs'

2023-06-28 Thread Jeff Johnson 
Yao,

Either add the required kernel-{devel,debuginfo} so the osd-ldiskfs kernel
modules can be built against your kernel or remove lustre-all-dkms package
and replace with lustre-zfs-dkms and build ZFS only Lustre modules.

--Jeff


On Wed, Jun 28, 2023 at 8:45 AM Yao Weng  wrote:

> I have error when installing lustre-all-dkms-2.15.3-1.el8.noarch
>
> Loading new lustre-all-2.15.3 DKMS files...
>
> Deprecated feature: REMAKE_INITRD (/usr/src/lustre-all-2.15.3/dkms.conf)
>
> Building for 4.18.0-477.15.1.el8_8.x86_64 4.18.0-477.10.1.el8_lustre.x86_64
>
> Building initial module for 4.18.0-477.15.1.el8_8.x86_64
>
> Deprecated feature: REMAKE_INITRD
> (/var/lib/dkms/lustre-all/2.15.3/source/dkms.conf)
>
> realpath: /var/lib/dkms/spl/2.1.12/source: No such file or directory
>
> realpath: /var/lib/dkms/spl/kernel-4.18.0-477.15.1.el8_8.x86_64-x86_64: No
> such file or directory
>
> configure: WARNING:
>
>
> Disabling ldiskfs support because complete ext4 source does not exist.
>
>
> If you are building using kernel-devel packages and require ldiskfs
>
> server support then ensure that the matching kernel-debuginfo-common
>
> and kernel-debuginfo-common- packages are installed.
>
>
> awk: fatal: cannot open file
> `/var/lib/dkms/lustre-all/2.15.3/build/_lpb/Makefile.compile.lustre' for
> reading (No such file or directory)
>
> ./configure: line 53751: test: too many arguments
>
> ./configure: line 53755: test: too many arguments
>
> Error!  Build of osd_ldiskfs.ko failed for: 4.18.0-477.15.1.el8_8.x86_64
> (x86_64)
>
> Make sure the name of the generated module is correct and at the root of
> the
>
> build directory, or consult make.log in the build directory
>
> /var/lib/dkms/lustre-all/2.15.3/build for more information.
>
> warning: %post(lustre-all-dkms-2.15.3-1.el8.noarch) scriptlet failed, exit
> status 7
>
>
> Error in POSTIN scriptlet in rpm package lustre-all-dkms
>
>
>
> On Wed, Jun 28, 2023 at 9:57 AM Yao Weng  wrote:
>
>> Thank Jeff:
>> My installation steps are
>>
>> step1: set local software provision (
>> https://wiki.lustre.org/Installing_the_Lustre_Software)
>> I download all rpm from
>>
>> https://downloads.whamcloud.com/public/lustre/lustre-2.15.3/el8.8/server
>>
>> https://downloads.whamcloud.com/public/lustre/lustre-2.15.3/el8.8/client
>>
>> https://downloads.whamcloud.com/public/e2fsprogs/1.47.0.wc2/el8
>>
>> step 2: Install the Lustre e2fsprogs distribution:
>>
>> sudo yum --nogpgcheck --disablerepo=* --enablerepo=e2fsprogs-wc install
>> e2fsprogs
>>
>>
>> step 3Install EPEL repository support:
>>
>> sudo yum -y install epel-release
>>
>>
>> step 4 Follow the instructions from the ZFS on Linux project
>> 
>>  to
>> install the ZFS YUM repository definition. Use the DKMS package repository
>> (the default)
>>
>> sudo dnf install https://zfsonlinux.org/epel/zfs-release-2-3$(rpm --eval
>> "%{dist}").noarch.rpm
>>
>>
>> step 5 Install the Lustre-patched kernel packages. Ensure that the Lustre
>> repository is picked for the kernel packages, by disabling the OS repos:
>>
>>
>> sudo yum --nogpgcheck --disablerepo=base,extras,updates \
>>
>> --enablerepo=lustre-server install \
>>
>> kernel \
>>
>> kernel-devel \
>>
>> kernel-headers \
>>
>> kernel-tools \
>>
>> kernel-tools-libs \
>>
>> kernel-tools-libs-devel
>>
>>
>> step 6 Generate a persistent hostid on the machine, if one does not
>> already exist. This is needed to help protect ZFS zpools against
>> simultaneous imports on multiple servers. For example:
>>
>> hid=`[ -f /etc/hostid ] && od -An -tx /etc/hostid|sed 's/ //g'`
>>
>> [ "$hid" = `hostid` ] || genhostid
>>
>>
>> step 7 reboot
>>
>> step 8 install lustre and zfs
>> sudo yum --skip-broken --nogpgcheck --enablerepo=lustre-server install \
>>  lustre-dkms \
>>  lustre-osd-zfs-mount \
>>  lustre \
>>  lustre-resource-agents \
>> lustre-dkms \
>>  zfs
>>
>> step 9 Load the Lustre and ZFS kernel modules to verify that the
>> software has installed correctly
>>
>> sudo modprobe -v zfs
>>
>> sudo modprobe -v lustre
>>
>> On Wed, Jun 28, 2023 at 1:04 AM Jeff Johnson <
>> jeff.john...@aeoncomputing.com> wrote:
>>
>>> Did you install the Lustre server RPMs?
>>> Your email lists both server and client repositories.
>>>
>>> Are you using DKMS? Did you install and built lustre-zfs-dkms or
>>> lustre-all-dkms packages?
>>>
>>> It doesn’t appear that you have any Lustre server kernel modules loaded,
>>> which makes me suspect you didn’t install or built the server side RPMs or
>>> DKMS trees
>>>
>>>
>>>
>>> On Tue, Jun 27, 2023 at 21:41 Yao Weng via lustre-discuss <
>>> lustre-discuss@lists.lustre.org> wrote:
>>>
 Hi:
 I follow https://wiki.lustre.org/Installing_the_Lustre_Software to
 install lustre.

 My kernel is

 $ uname -r

 4.18.0-477.13.1.el8_8.x86_64

 I install

Re: [lustre-discuss] Lustre Installation Question: 158-c: Can't load module 'osd-zfs'

2023-06-28 Thread Yao Weng via lustre-discuss 
I have error when installing lustre-all-dkms-2.15.3-1.el8.noarch

Loading new lustre-all-2.15.3 DKMS files...

Deprecated feature: REMAKE_INITRD (/usr/src/lustre-all-2.15.3/dkms.conf)

Building for 4.18.0-477.15.1.el8_8.x86_64 4.18.0-477.10.1.el8_lustre.x86_64

Building initial module for 4.18.0-477.15.1.el8_8.x86_64

Deprecated feature: REMAKE_INITRD
(/var/lib/dkms/lustre-all/2.15.3/source/dkms.conf)

realpath: /var/lib/dkms/spl/2.1.12/source: No such file or directory

realpath: /var/lib/dkms/spl/kernel-4.18.0-477.15.1.el8_8.x86_64-x86_64: No
such file or directory

configure: WARNING:


Disabling ldiskfs support because complete ext4 source does not exist.


If you are building using kernel-devel packages and require ldiskfs

server support then ensure that the matching kernel-debuginfo-common

and kernel-debuginfo-common- packages are installed.


awk: fatal: cannot open file
`/var/lib/dkms/lustre-all/2.15.3/build/_lpb/Makefile.compile.lustre' for
reading (No such file or directory)

./configure: line 53751: test: too many arguments

./configure: line 53755: test: too many arguments

Error!  Build of osd_ldiskfs.ko failed for: 4.18.0-477.15.1.el8_8.x86_64
(x86_64)

Make sure the name of the generated module is correct and at the root of the

build directory, or consult make.log in the build directory

/var/lib/dkms/lustre-all/2.15.3/build for more information.

warning: %post(lustre-all-dkms-2.15.3-1.el8.noarch) scriptlet failed, exit
status 7


Error in POSTIN scriptlet in rpm package lustre-all-dkms



On Wed, Jun 28, 2023 at 9:57 AM Yao Weng  wrote:

> Thank Jeff:
> My installation steps are
>
> step1: set local software provision (
> https://wiki.lustre.org/Installing_the_Lustre_Software)
> I download all rpm from
>
> https://downloads.whamcloud.com/public/lustre/lustre-2.15.3/el8.8/server
>
> https://downloads.whamcloud.com/public/lustre/lustre-2.15.3/el8.8/client
>
> https://downloads.whamcloud.com/public/e2fsprogs/1.47.0.wc2/el8
>
> step 2: Install the Lustre e2fsprogs distribution:
>
> sudo yum --nogpgcheck --disablerepo=* --enablerepo=e2fsprogs-wc install
> e2fsprogs
>
>
> step 3Install EPEL repository support:
>
> sudo yum -y install epel-release
>
>
> step 4 Follow the instructions from the ZFS on Linux project
> 
>  to
> install the ZFS YUM repository definition. Use the DKMS package repository
> (the default)
>
> sudo dnf install https://zfsonlinux.org/epel/zfs-release-2-3$(rpm --eval
> "%{dist}").noarch.rpm
>
>
> step 5 Install the Lustre-patched kernel packages. Ensure that the Lustre
> repository is picked for the kernel packages, by disabling the OS repos:
>
>
> sudo yum --nogpgcheck --disablerepo=base,extras,updates \
>
> --enablerepo=lustre-server install \
>
> kernel \
>
> kernel-devel \
>
> kernel-headers \
>
> kernel-tools \
>
> kernel-tools-libs \
>
> kernel-tools-libs-devel
>
>
> step 6 Generate a persistent hostid on the machine, if one does not
> already exist. This is needed to help protect ZFS zpools against
> simultaneous imports on multiple servers. For example:
>
> hid=`[ -f /etc/hostid ] && od -An -tx /etc/hostid|sed 's/ //g'`
>
> [ "$hid" = `hostid` ] || genhostid
>
>
> step 7 reboot
>
> step 8 install lustre and zfs
> sudo yum --skip-broken --nogpgcheck --enablerepo=lustre-server install \
>  lustre-dkms \
>  lustre-osd-zfs-mount \
>  lustre \
>  lustre-resource-agents \
> lustre-dkms \
>  zfs
>
> step 9 Load the Lustre and ZFS kernel modules to verify that the software
> has installed correctly
>
> sudo modprobe -v zfs
>
> sudo modprobe -v lustre
>
> On Wed, Jun 28, 2023 at 1:04 AM Jeff Johnson <
> jeff.john...@aeoncomputing.com> wrote:
>
>> Did you install the Lustre server RPMs?
>> Your email lists both server and client repositories.
>>
>> Are you using DKMS? Did you install and built lustre-zfs-dkms or
>> lustre-all-dkms packages?
>>
>> It doesn’t appear that you have any Lustre server kernel modules loaded,
>> which makes me suspect you didn’t install or built the server side RPMs or
>> DKMS trees
>>
>>
>>
>> On Tue, Jun 27, 2023 at 21:41 Yao Weng via lustre-discuss <
>> lustre-discuss@lists.lustre.org> wrote:
>>
>>> Hi:
>>> I follow https://wiki.lustre.org/Installing_the_Lustre_Software to
>>> install lustre.
>>>
>>> My kernel is
>>>
>>> $ uname -r
>>>
>>> 4.18.0-477.13.1.el8_8.x86_64
>>>
>>> I install
>>>
>>> https://downloads.whamcloud.com/public/lustre/lustre-2.15.3/el8.8/server
>>>
>>> https://downloads.whamcloud.com/public/lustre/lustre-2.15.3/el8.8/client
>>>
>>> https://downloads.whamcloud.com/public/e2fsprogs/1.47.0.wc2/el8
>>>
>>>
>>> lsmod | grep lustre
>>>
>>> *lustre*   1048576  0
>>>
>>> lmv   204800  1 *lustre*
>>>
>>> mdc   282624  1 *lustre*
>>>
>>> lov   344064  2 mdc,*lustre*
>>>
>>> ptlrpc   2490368  7 fld,osc,fid,lov,mdc,lmv,*lustre*

Re: [lustre-discuss] Lustre Installation Question: 158-c: Can't load module 'osd-zfs'

2023-06-28 Thread Yao Weng via lustre-discuss 
Thank Jeff:
My installation steps are

step1: set local software provision (
https://wiki.lustre.org/Installing_the_Lustre_Software)
I download all rpm from

https://downloads.whamcloud.com/public/lustre/lustre-2.15.3/el8.8/server

https://downloads.whamcloud.com/public/lustre/lustre-2.15.3/el8.8/client

https://downloads.whamcloud.com/public/e2fsprogs/1.47.0.wc2/el8

step 2: Install the Lustre e2fsprogs distribution:

sudo yum --nogpgcheck --disablerepo=* --enablerepo=e2fsprogs-wc install
e2fsprogs


step 3Install EPEL repository support:

sudo yum -y install epel-release


step 4 Follow the instructions from the ZFS on Linux project

to
install the ZFS YUM repository definition. Use the DKMS package repository
(the default)

sudo dnf install https://zfsonlinux.org/epel/zfs-release-2-3$(rpm --eval
"%{dist}").noarch.rpm


step 5 Install the Lustre-patched kernel packages. Ensure that the Lustre
repository is picked for the kernel packages, by disabling the OS repos:


sudo yum --nogpgcheck --disablerepo=base,extras,updates \

--enablerepo=lustre-server install \

kernel \

kernel-devel \

kernel-headers \

kernel-tools \

kernel-tools-libs \

kernel-tools-libs-devel


step 6 Generate a persistent hostid on the machine, if one does not already
exist. This is needed to help protect ZFS zpools against simultaneous
imports on multiple servers. For example:

hid=`[ -f /etc/hostid ] && od -An -tx /etc/hostid|sed 's/ //g'`

[ "$hid" = `hostid` ] || genhostid


step 7 reboot

step 8 install lustre and zfs
sudo yum --skip-broken --nogpgcheck --enablerepo=lustre-server install \
 lustre-dkms \
 lustre-osd-zfs-mount \
 lustre \
 lustre-resource-agents \
lustre-dkms \
 zfs

step 9 Load the Lustre and ZFS kernel modules to verify that the software
has installed correctly

sudo modprobe -v zfs

sudo modprobe -v lustre

On Wed, Jun 28, 2023 at 1:04 AM Jeff Johnson 
wrote:

> Did you install the Lustre server RPMs?
> Your email lists both server and client repositories.
>
> Are you using DKMS? Did you install and built lustre-zfs-dkms or
> lustre-all-dkms packages?
>
> It doesn’t appear that you have any Lustre server kernel modules loaded,
> which makes me suspect you didn’t install or built the server side RPMs or
> DKMS trees
>
>
>
> On Tue, Jun 27, 2023 at 21:41 Yao Weng via lustre-discuss <
> lustre-discuss@lists.lustre.org> wrote:
>
>> Hi:
>> I follow https://wiki.lustre.org/Installing_the_Lustre_Software to
>> install lustre.
>>
>> My kernel is
>>
>> $ uname -r
>>
>> 4.18.0-477.13.1.el8_8.x86_64
>>
>> I install
>>
>> https://downloads.whamcloud.com/public/lustre/lustre-2.15.3/el8.8/server
>>
>> https://downloads.whamcloud.com/public/lustre/lustre-2.15.3/el8.8/client
>>
>> https://downloads.whamcloud.com/public/e2fsprogs/1.47.0.wc2/el8
>>
>>
>> lsmod | grep lustre
>>
>> *lustre*   1048576  0
>>
>> lmv   204800  1 *lustre*
>>
>> mdc   282624  1 *lustre*
>>
>> lov   344064  2 mdc,*lustre*
>>
>> ptlrpc   2490368  7 fld,osc,fid,lov,mdc,lmv,*lustre*
>>
>> obdclass 3633152  8 fld,osc,fid,ptlrpc,lov,mdc,lmv,*lustre*
>>
>> lnet  704512  7 osc,obdclass,ptlrpc,ksocklnd,lmv,*lustre*
>>
>> libcfs266240  11
>> fld,lnet,osc,fid,obdclass,ptlrpc,ksocklnd,lov,mdc,lmv,
>>
>> *lustre*lsmod | grep zfs
>>
>> lsmod | grep zfs
>>
>> *zfs*  3887104  0
>>
>> zunicode  335872  1 *zfs*
>>
>> zzstd 512000  1 *zfs*
>>
>> zlua  176128  1 *zfs*
>>
>> zavl   16384  1 *zfs*
>>
>> icp   319488  1 *zfs*
>>
>> zcommon   102400  2 *zfs*,icp
>>
>> znvpair90112  2 *zfs*,zcommon
>>
>> spl   114688  6 *zfs*,icp,zzstd,znvpair,zcommon,zavl
>>
>>
>> I am able to create mgs/mdt/ost
>>
>> But when I try to mount
>>
>> sudo mount.lustre lustre-mgs/mgs /lustre/mnt
>>
>> mount.lustre: mount lustre-mgs/mgs at /lustre/mnt failed: No such device
>>
>> Are the lustre modules loaded?
>>
>>  Check /etc/modprobe.conf and /proc/filesystems
>>
>> dmesg gives these error
>>
>> [76783.604090] LustreError: 158-c: Can't load module 'osd-zfs'
>>
>> [76783.606174] LustreError: 223535:0:(genops.c:361:class_newdev()) OBD:
>> unknown type: osd-zfs
>>
>> [76783.607856] LustreError: 223535:0:(obd_config.c:620:class_attach())
>> Cannot create device MGS-osd of type osd-zfs : -19
>>
>> [76783.609805] LustreError:
>> 223535:0:(obd_mount.c:195:lustre_start_simple()) MGS-osd attach error -19
>>
>> [76783.611426] LustreError:
>> 223535:0:(obd_mount_server.c:1993:server_fill_super()) Unable to start osd
>> on lustre-mgs/mgs: -19
>>
>> [76783.613457] LustreError: 223535:0:(super25.c:183:lustre_fill_super())
>> llite: Unable to mount : rc = -19
>>
>>
>> ___
>>

Re: [lustre-discuss] Lustre Installation Question: 158-c: Can't load module 'osd-zfs'

2023-06-27 Thread Jeff Johnson 
Did you install the Lustre server RPMs?
Your email lists both server and client repositories.

Are you using DKMS? Did you install and built lustre-zfs-dkms or
lustre-all-dkms packages?

It doesn’t appear that you have any Lustre server kernel modules loaded,
which makes me suspect you didn’t install or built the server side RPMs or
DKMS trees



On Tue, Jun 27, 2023 at 21:41 Yao Weng via lustre-discuss <
lustre-discuss@lists.lustre.org> wrote:

> Hi:
> I follow https://wiki.lustre.org/Installing_the_Lustre_Software to
> install lustre.
>
> My kernel is
>
> $ uname -r
>
> 4.18.0-477.13.1.el8_8.x86_64
>
> I install
>
> https://downloads.whamcloud.com/public/lustre/lustre-2.15.3/el8.8/server
>
> https://downloads.whamcloud.com/public/lustre/lustre-2.15.3/el8.8/client
>
> https://downloads.whamcloud.com/public/e2fsprogs/1.47.0.wc2/el8
>
>
> lsmod | grep lustre
>
> *lustre*   1048576  0
>
> lmv   204800  1 *lustre*
>
> mdc   282624  1 *lustre*
>
> lov   344064  2 mdc,*lustre*
>
> ptlrpc   2490368  7 fld,osc,fid,lov,mdc,lmv,*lustre*
>
> obdclass 3633152  8 fld,osc,fid,ptlrpc,lov,mdc,lmv,*lustre*
>
> lnet  704512  7 osc,obdclass,ptlrpc,ksocklnd,lmv,*lustre*
>
> libcfs266240  11
> fld,lnet,osc,fid,obdclass,ptlrpc,ksocklnd,lov,mdc,lmv,
>
> *lustre*lsmod | grep zfs
>
> lsmod | grep zfs
>
> *zfs*  3887104  0
>
> zunicode  335872  1 *zfs*
>
> zzstd 512000  1 *zfs*
>
> zlua  176128  1 *zfs*
>
> zavl   16384  1 *zfs*
>
> icp   319488  1 *zfs*
>
> zcommon   102400  2 *zfs*,icp
>
> znvpair90112  2 *zfs*,zcommon
>
> spl   114688  6 *zfs*,icp,zzstd,znvpair,zcommon,zavl
>
>
> I am able to create mgs/mdt/ost
>
> But when I try to mount
>
> sudo mount.lustre lustre-mgs/mgs /lustre/mnt
>
> mount.lustre: mount lustre-mgs/mgs at /lustre/mnt failed: No such device
>
> Are the lustre modules loaded?
>
>  Check /etc/modprobe.conf and /proc/filesystems
>
> dmesg gives these error
>
> [76783.604090] LustreError: 158-c: Can't load module 'osd-zfs'
>
> [76783.606174] LustreError: 223535:0:(genops.c:361:class_newdev()) OBD:
> unknown type: osd-zfs
>
> [76783.607856] LustreError: 223535:0:(obd_config.c:620:class_attach())
> Cannot create device MGS-osd of type osd-zfs : -19
>
> [76783.609805] LustreError:
> 223535:0:(obd_mount.c:195:lustre_start_simple()) MGS-osd attach error -19
>
> [76783.611426] LustreError:
> 223535:0:(obd_mount_server.c:1993:server_fill_super()) Unable to start osd
> on lustre-mgs/mgs: -19
>
> [76783.613457] LustreError: 223535:0:(super25.c:183:lustre_fill_super())
> llite: Unable to mount : rc = -19
>
>
> ___
> lustre-discuss mailing list
> lustre-discuss@lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>
-- 
--
Jeff Johnson
Co-Founder
Aeon Computing

jeff.john...@aeoncomputing.com
www.aeoncomputing.com
t: 858-412-3810 x1001   f: 858-412-3845
m: 619-204-9061

4170 Morena Boulevard, Suite C - San Diego, CA 92117

High-Performance Computing / Lustre Filesystems / Scale-out Storage
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

[lustre-discuss] Lustre Installation Question: 158-c: Can't load module 'osd-zfs'

2023-06-27 Thread Yao Weng via lustre-discuss 
Hi:
I follow https://wiki.lustre.org/Installing_the_Lustre_Software to install
lustre.

My kernel is

$ uname -r

4.18.0-477.13.1.el8_8.x86_64

I install

https://downloads.whamcloud.com/public/lustre/lustre-2.15.3/el8.8/server

https://downloads.whamcloud.com/public/lustre/lustre-2.15.3/el8.8/client

https://downloads.whamcloud.com/public/e2fsprogs/1.47.0.wc2/el8


lsmod | grep lustre

*lustre*   1048576  0

lmv   204800  1 *lustre*

mdc   282624  1 *lustre*

lov   344064  2 mdc,*lustre*

ptlrpc   2490368  7 fld,osc,fid,lov,mdc,lmv,*lustre*

obdclass 3633152  8 fld,osc,fid,ptlrpc,lov,mdc,lmv,*lustre*

lnet  704512  7 osc,obdclass,ptlrpc,ksocklnd,lmv,*lustre*

libcfs266240  11
fld,lnet,osc,fid,obdclass,ptlrpc,ksocklnd,lov,mdc,lmv,

*lustre*lsmod | grep zfs

lsmod | grep zfs

*zfs*  3887104  0

zunicode  335872  1 *zfs*

zzstd 512000  1 *zfs*

zlua  176128  1 *zfs*

zavl   16384  1 *zfs*

icp   319488  1 *zfs*

zcommon   102400  2 *zfs*,icp

znvpair90112  2 *zfs*,zcommon

spl   114688  6 *zfs*,icp,zzstd,znvpair,zcommon,zavl


I am able to create mgs/mdt/ost

But when I try to mount

sudo mount.lustre lustre-mgs/mgs /lustre/mnt

mount.lustre: mount lustre-mgs/mgs at /lustre/mnt failed: No such device

Are the lustre modules loaded?

 Check /etc/modprobe.conf and /proc/filesystems

dmesg gives these error

[76783.604090] LustreError: 158-c: Can't load module 'osd-zfs'

[76783.606174] LustreError: 223535:0:(genops.c:361:class_newdev()) OBD:
unknown type: osd-zfs

[76783.607856] LustreError: 223535:0:(obd_config.c:620:class_attach())
Cannot create device MGS-osd of type osd-zfs : -19

[76783.609805] LustreError:
223535:0:(obd_mount.c:195:lustre_start_simple()) MGS-osd attach error -19

[76783.611426] LustreError:
223535:0:(obd_mount_server.c:1993:server_fill_super()) Unable to start osd
on lustre-mgs/mgs: -19

[76783.613457] LustreError: 223535:0:(super25.c:183:lustre_fill_super())
llite: Unable to mount : rc = -19
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org