Hi! I have been trying to set up the "perfect" (for me) encrypted server for some time now, and I have been running into a lot of issues. After extensive digging and trying to learn about systemd, I think I have found the root cause, and I would like some input from people with more experience. Please excuse the verbosity of this email, but it is essentially a detailed bug report containing what I hope is every little detail that is required to recreate and debug this issue.
First some background: The problems arise when I want to run a multi-device btrfs filesystem on LUKS encrypted backing devices, and use a keyfile that resides on another encrypted filesystem. Since I am a Debian user I would have previously used the keyscript=decrypt_derived method to solve this, but now I want to use systemd, where this is missing. The problem is that I get a lot of timeouts for no apparent reason, and I have not managed to tell systemd about the dependencies, and they keep timing out even though they are obviously correctly unlocked. After debugging this for a while I think it comes down to that when systemd tries to unlock an encrypted block device and create a corresponding /dev/mapper/foo and a dev-mapper-foo.device, it fails if the unlocked device does not have a valid UUID. It will successfully unlock the device and create the correct mapping, but then it hangs, and reports the device as inactive (dead). I will explain what I expect to work, and what does not work, with these commands. This is on a Fedora Server virtual machine, which I have never tried before but installed just to test this! I have created a partition on a block device (/dev/sdb1) which I have prepared with luksFormat using a keyfile called "/keyfile", located on the root filesystem. Nothing fancy here. I put a simple entry in /etc/crypttab and had systemctl daemon-reload generate the corresponding files in /var/run/systemd/generator, which I copied to /etc/systemd/system/. -------- /etc/systemd/system/[email protected] -------- # Automatically generated by systemd-cryptsetup-generator [Unit] Description=Cryptography Setup for %I Documentation=man:crypttab(5) man:systemd-cryptsetup-generator(8) man:[email protected](8) SourcePath=/etc/crypttab DefaultDependencies=no Conflicts=umount.target BindsTo=dev-mapper-%i.device IgnoreOnIsolate=true After=cryptsetup-pre.target Before=cryptsetup.target RequiresMountsFor=/keyfile BindsTo=dev-sdb1.device After=dev-sdb1.device Before=umount.target [Service] Type=oneshot RemainAfterExit=yes TimeoutSec=0 ExecStart=/usr/lib/systemd/systemd-cryptsetup attach 'crypt' '/dev/sdb1' '/keyfile' 'luks' ExecStop=/usr/lib/systemd/systemd-cryptsetup detach 'crypt' -------- /etc/systemd/system/[email protected] -------- There is also a link from dev-mapper-crypt.device.requires: dev-mapper-crypt.device.requires └── [email protected] -> ../[email protected] With this set up, and the encrypted target block device has a plain old ext4 filesystem, everything works like I expect it to: [root@fedora ~]# systemctl status dev-mapper-crypt.device ● dev-mapper-crypt.device Loaded: loaded Active: inactive (dead) [root@fedora ~]# systemctl start dev-mapper-crypt.device [root@fedora ~]# systemctl status dev-mapper-crypt.device ● dev-mapper-crypt.device - /dev/mapper/crypt Follow: unit currently follows state of sys-devices-virtual-block-dm\x2d2.device Loaded: loaded Active: active (plugged) since Sat 2015-12-12 14:12:44 CET; 6s ago Device: /sys/devices/virtual/block/dm-2 This means that systemd knows about dev-mapper-crypt.device, it can start automatically if wanted, and it can be used as a dependency. Great! Now I make a change. I remove the ext4 filesystem from the encrypted target by filling /dev/mapper/crypt with zeros, reboot for good measure, and try this again: root@fedora ~]# systemctl status dev-mapper-crypt.device ● dev-mapper-crypt.device Loaded: loaded Active: inactive (dead) [root@fedora ~]# systemctl start dev-mapper-crypt.device ... and here it stops. Nothing happens. If I log in again and check what's going on I see this: 1197 pts/0 Ss 0:00 | \_ -bash 1219 pts/0 S+ 0:00 | \_ systemctl start dev-mapper-crypt.device 1220 pts/0 S+ 0:00 | \_ /usr/bin/systemd-tty-ask-password-agent --watch 1221 pts/0 Sl+ 0:00 | \_ /usr/bin/pkttyagent --notify-fd 5 --fallback But there are no password prompts anywhere, not in any SSH sessions nor in the actual console. Eventually the original command times out: Job for dev-mapper-crypt.device timed out. [root@fedora ~]# systemctl status dev-mapper-crypt.device ● dev-mapper-crypt.device Loaded: loaded Active: inactive (dead) Dec 12 14:34:44 fedora.localdomain systemd[1]: dev-mapper-crypt.device: Job dev-mapper-crypt.device/start timed out. Dec 12 14:34:44 fedora.localdomain systemd[1]: Timed out waiting for device dev-mapper-crypt.device. Dec 12 14:34:44 fedora.localdomain systemd[1]: dev-mapper-crypt.device: Job dev-mapper-crypt.device/start failed with result 'timeout'. However, /dev/mapper/crypt is live and well: [root@fedora ~]# ls -l /dev/mapper/crypt lrwxrwxrwx. 1 root root 7 Dec 12 14:33 /dev/mapper/crypt -> ../dm-2 Now, the reason I think that this is a problem with the UUID is that when I have tried this with a multi-volume btrfs filesystem, I have been getting the same type of timeouts on all devices but one, and they have been happening in a "random" order. Btrfs assigns the same UUID to all volumes participating in the filesystem, and this is what leads me to believe that the problem is with that. If anyone has managed to read this far, I thank you for the attention, and I have two questions for you: 1) Am I right in assuming that a valid target UUID should not be in any way necessary or relevant for unlocking an encrypted device? In my eyes it is perfectly valid to have a device filled with for example random data, filled with zeros, or with any other unknown pattern. 2) Is my understanding of systemd devices and services correct, in that it should be possible to treat dev-mapper-crypt.device as a service that I can start and stop using systemctl? // Anders _______________________________________________ systemd-devel mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/systemd-devel
