Package: lvm2
Version: 2.03.16-1
Severity: important
X-Debbugs-Cc: vermee...@vermwa.re

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Dear Maintainer,

Since LVM 2.03.15 or possibly 2.03.16 my system fails to boot during initramfs
because the root LVM LV is not activated. I did some digging and confirmed this
is caused by the new method of using udev rules, which have a big flaw.

In /lib/udev/rules.d/69-lvm.rules a VG is only activated once the VG is fully
complete. However in initramfs my HDD arrays are not yet activated because they
need keyfile stored on root partition for automatic activation. This means the
VG is never complete (missing PVs) so it is never activated, meaning the root LV
is never activated at all, resulting in boot failure with initramfs shell.

I have the following storage stack in use on this system:
Disk partition -> integrity -> mdadm -> luks -> lvm -> ext4
Integrity is not part of Debian, I submitted patches a while ago at:
https://salsa.debian.org/cryptsetup-team/cryptsetup/-/merge_requests/22

My LVM setup is the following.
The fc520 contains root, swap LV and all integritysetup meta LVs.
The nas4 and ex12 PVs are exclusively used by hdd LV.

root@verm-r4e:~# pvs
  PV                           VG       Fmt  Attr PSize  PFree
  /dev/mapper/md_ex12_0_crypt  verm-r4e lvm2 a--  10.91t       0
  /dev/mapper/md_fc520_0_crypt verm-r4e lvm2 a--   1.81t  256.77g
  /dev/mapper/md_nas4_0_crypt  verm-r4e lvm2 a--  <3.64t <565.54g
root@verm-r4e:~# vgs
  VG       #PV #LV #SN Attr   VSize  VFree
  verm-r4e   3   9   0 wz--l- 16.36t 822.31g
root@verm-r4e:~# lvs
  LV             VG       Attr       LSize
  hdd            verm-r4e -wi-ao---- 14.00t
  is_boot_0_meta verm-r4e -wi-ao----  8.00m
  is_boot_1_meta verm-r4e -wi-ao----  8.00m
  is_ex12_0_meta verm-r4e -wi-ao---- 11.41g
  is_ex12_1_meta verm-r4e -wi-ao---- 11.41g
  is_nas4_0_meta verm-r4e -wi-ao----  4.14g
  is_nas4_1_meta verm-r4e -wi-ao----  4.14g
  root           verm-r4e -wi-ao----  1.50t
  swap           verm-r4e -wi-ao---- 32.00g

Although my integrity setup is not common it's very common for servers with
many encrypted disks is to have something like this:

hdd -> mdadm -> luks -> lvm

With current version udev rules simply having all these disks in the same VG
will result in boot failure, because udev rule will only activate the entire VG
once all PVs are there.

The fix is to use behaviour like in old version and similar to cryptsetup hooks.
Actaully parse what is needed from kernel arg, fstab, etc and try to activate
these LVs directly even if the VG is incomplete. Concretely in my case the hook
should activate verm-r4e/swap (resume device) and verm-r4e/root (root).

This is quite a key regression for servers, as I'm not sure you can access
initramfs shell remotely when it drops into it. Maybe if you have
dropbear-initramfs setup? In my case it's on a workstation.

Could you share thoughts and/or investigate this problem?
Perhaps upstream udev rules are not suited for initramfs and only for regular
system operation once fully booted?

Thanks,

Melvin Vermeeren.

- -- System Information:
Debian Release: bookworm/sid
  APT prefers testing
  APT policy: (500, 'testing')
merged-usr: no
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 5.18.0-4-amd64 (SMP w/16 CPU threads; PREEMPT)
Kernel taint flags: TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8), 
LANGUAGE=en_GB:en
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages lvm2 depends on:
ii  dmeventd                    2:1.02.185-1
ii  dmsetup                     2:1.02.185-1
ii  init-system-helpers         1.64
ii  libaio1                     0.3.113-2
ii  libblkid1                   2.38.1-1
ii  libc6                       2.34-4
ii  libdevmapper-event1.02.1    2:1.02.185-1
ii  libedit2                    3.1-20210910-1
ii  libselinux1                 3.4-1+b1
ii  libsystemd0                 251.3-1
ii  libudev1                    251.3-1
ii  lsb-base                    11.2
ii  systemd [systemd-tmpfiles]  251.3-1

Versions of packages lvm2 recommends:
ii  thin-provisioning-tools  0.9.0-2

lvm2 suggests no packages.

- -- Configuration Files:
/etc/lvm/lvm.conf changed [not included]
Edit: I set issue_discards = 1 only.

- -- no debconf information

-----BEGIN PGP SIGNATURE-----

iQJIBAEBCgAyFiEEiu1YAh/qzdXye6Dmpy9idxbqnZYFAmMM7koUHHZlcm1lZXJl
bkB2ZXJtd2EucmUACgkQpy9idxbqnZbDvBAAtOAjlLnoQv6ashtTpuk0b/6MG3UW
BE/we/Z15SC2bsM59zpci9BMsFmfNvS5iVznDC0TRy9P0zaLS+jQ4T4NNd6KPLwe
5mRorEFMI7INwD8vWBkhhCzCXyU6Wp8jCOkngVMND04wU/zq56amKHFnQ9yGaqFl
/jLzTVnUQ2hxJNRvmbghX5x7n4SQw1oSNlBAG8i1Pz2mdMCH2cLWz+ieP2fCYffp
eHkFs+ECTkcnEvDIlRZuIx34+ZZAh2JI/2bPqCtQRZOWOUEujlQDlgEhgp0qORdU
wY2/q1dTsPOwFhgUUvgCtSS0o451E7boZ82EigaGFO/YXWvjiC0eS+rCXokQBJx+
cU89fjgnT/83WlDYN26TiVulX+2peXomZ5c4aCFHQ1jZTbsYePd0ucg2YbwzNAq7
HEQ0hQR6WWVziiHZfrYceyoKgKdcOIpjrnbPra1EHtKGL6o0J5LSzK8ZF45oyD6T
v+gQnzhSEUPL6NPrL+hUSpCwDx70ubLdnNP7n4eiCbaPmTwPoW4p5ohv9Rkeueuz
Ee64BhmVhG3vy8jRHH2gzEmH56oafHfbH7N3tpQtqzuQwdJ6vHs0rniKJaGId6st
9412vuoA3CmmrFFGyha+1gZZif65L6AvXhrqMbcUGUJVuIc20liqAnyDsKEngKJ7
JoJW3tmk89N4uOY=
=8+2U
-----END PGP SIGNATURE-----

Reply via email to