Bug#924560: cryptsetup luksOpen requires 1GB of RAM in the default configuration
On Tue, 26 Mar 2019 10:42:29 +0100 Ondrej Kozina wrote: On Thu, 14 Mar 2019 19:43:26 +0100 Guilhem Moulin wrote: > Hi Milan, > > On Thu, 14 Mar 2019 at 19:22:42 +0100, Milan Broz wrote: > > (...) > > FYI we know about that parallel unlocking problem already and we are trying > > to find (with systemd people) some solution (perhaps based on cgroups memory limits > > and some locking). > > Cool, do you have a link to refer to? Couldn't find anything from a > quick glance at systemd's issue tracker. FYI, I've opened cryptsetup issue for parallel Argon2 invocation case (https://gitlab.com/cryptsetup/cryptsetup/issues/446). There should be some systemd issue/PR soon as well. I'll link it to the cryptsetup issue when I see it. Not so soon, but here it is: https://github.com/systemd/systemd/pull/14168 Regards O.
Bug#924560: cryptsetup luksOpen requires 1GB of RAM in the default configuration
On Thu, 14 Mar 2019 19:43:26 +0100 Guilhem Moulin wrote: Hi Milan, On Thu, 14 Mar 2019 at 19:22:42 +0100, Milan Broz wrote: > (...) > FYI we know about that parallel unlocking problem already and we are trying > to find (with systemd people) some solution (perhaps based on cgroups memory limits > and some locking). Cool, do you have a link to refer to? Couldn't find anything from a quick glance at systemd's issue tracker. FYI, I've opened cryptsetup issue for parallel Argon2 invocation case (https://gitlab.com/cryptsetup/cryptsetup/issues/446). There should be some systemd issue/PR soon as well. I'll link it to the cryptsetup issue when I see it. Regards Ondra -- Guilhem.
Bug#924560: cryptsetup luksOpen requires 1GB of RAM in the default configuration
Hi Milan, On Thu, 14 Mar 2019 at 19:22:42 +0100, Milan Broz wrote: >>> I think diverging from upstream (and other distros) with respect to >>> default algorithms requires careful consideration. And in that case, >>> compared to PBKDF2 Argon2 has interesting properties (such as resistance >>> to GPU cracking) which would be a shame not to benefit from out of the >>> box. > > For this case you need to specify PBKDF parameters directly and skip benchmark > (these PBKDF options were added exactly for this use case). > > This problem is there even with PBKDF2 for the iterations time - on some > IoT devices with LUKS device (formatted on developer's machine) the unlocking > time increases to many minutes. (With Argon PBKDF it is just worse because > memory > can be unavailable.) Aha, you beat me to it :-) >> I guess dracut with systemd in the initrd might be affected worse, >> than initramfs-tools. I wonder if I should open a bug report in >> systemd, to potentially execute luks2 unlock with some locking / >> sequentially. > > FYI we know about that parallel unlocking problem already and we are trying > to find (with systemd people) some solution (perhaps based on cgroups memory > limits > and some locking). Cool, do you have a link to refer to? Couldn't find anything from a quick glance at systemd's issue tracker. -- Guilhem. signature.asc Description: PGP signature
Bug#924560: cryptsetup luksOpen requires 1GB of RAM in the default configuration
Control: tag -1 + wontfix Control: tag -1 - moreinfo Control: severity -1 normal On Thu, 14 Mar 2019 at 17:31:05 +, Dimitri John Ledkov wrote: > On Thu, 14 Mar 2019 at 16:55, Guilhem Moulin wrote: >> AFAICT it does. What I guess doesn't is if the machine's resources are >> significantly reduced between luksFormat/luksAddKey and luksOpen. > > I guess that is the reason here. Majority of IoT / pi / etc devices > might prepare rootfs / SDcard / golden image / etc on a bigger > developer machine, prior to flashing it onto SDcard to then deploy on > the target device. I see. > So the solution there is to run the benchmark on the device, once, > record parameters and use those when creating the golden image for > memory, threads and possibly iterations too? Yup, you can re-use the parameters from a previous benchmark run on the target system (where the volume will be unlocked) and use it to luksFormat elsewhere. This applies to PBKDF2 too by the way, though skipping this step doesn't have the same severity; for PBKDF2 the benchmark only affect the iteration time, so if the volume is formatted on a fast system, unlocking on a slower one will slower than it should be (but won't OOM). > I wonder if I should open a bug report in systemd, to potentially > execute luks2 unlock with some locking / sequentially. Seems sensible. > I guess this bug should be tagged wontfixing and lowered priority to > normal, or something. Alright :-) -- Guilhem. signature.asc Description: PGP signature
Bug#924560: [pkg-cryptsetup-devel] Bug#924560: cryptsetup luksOpen requires 1GB of RAM in the default configuration
>> I think diverging from upstream (and other distros) with respect to >> default algorithms requires careful consideration. And in that case, >> compared to PBKDF2 Argon2 has interesting properties (such as resistance >> to GPU cracking) which would be a shame not to benefit from out of the >> box. For this case you need to specify PBKDF parameters directly and skip benchmark (these PBKDF options were added exactly for this use case). This problem is there even with PBKDF2 for the iterations time - on some IoT devices with LUKS device (formatted on developer's machine) the unlocking time increases to many minutes. (With Argon PBKDF it is just worse because memory can be unavailable.) So the suggestion above is correct - you need to measure some viable parameters and use them directly. If you run luksFormat on a small IoT system directly, it trims parameters and memory use according to system (it will never use more than half of physical available memory). ... > I guess dracut with systemd in the initrd might be affected worse, > than initramfs-tools. I wonder if I should open a bug report in > systemd, to potentially execute luks2 unlock with some locking / > sequentially. FYI we know about that parallel unlocking problem already and we are trying to find (with systemd people) some solution (perhaps based on cgroups memory limits and some locking). (Parallel unlocking can cause OOM killer to kill even different processes here.) You can change PBKDF parameters for existing device (preserving data) with cryptsetup luksConvertKey command, it takes the same PBKDF options. (So you can "downgrade" to PBKDF2 or decrease limits.) Milan
Bug#924560: [pkg-cryptsetup-devel] Bug#924560: cryptsetup luksOpen requires 1GB of RAM in the default configuration
On Thu, 14 Mar 2019 at 16:55, Guilhem Moulin wrote: > > > For example many IoT and Pi devices have 1GB of ram in total, and thus > > would OOM kill when trying to luksOpen. > > Is that something you experienced? I just deployed a fresh a Debian sid > VM with 2vCPUs, 1GiB RAM a default encryption stack (LVM2 on top of > LUKS/dm-crypt). Works just fine :-) The actual memory cost is about > half of the physical memory. > > Works equally fine on a VM with 512MiB RAM. There the memory cost is > set to ~250MiB. > Nice. > > Please consider reducing the default memory requirement of the argon2i > > in luks2 by default, or switching to pbkdf2 for LUKS2 as well. > > I think diverging from upstream (and other distros) with respect to > default algorithms requires careful consideration. And in that case, > compared to PBKDF2 Argon2 has interesting properties (such as resistance > to GPU cracking) which would be a shame not to benefit from out of the > box. > > > I think it is unfortunate to not support default encryption on 1GB big > > devices and VMs. > > AFAICT it does. What I guess doesn't is if the machine's resources are > significantly reduced between luksFormat/luksAddKey and luksOpen. > I guess that is the reason here. Majority of IoT / pi / etc devices might prepare rootfs / SDcard / golden image / etc on a bigger developer machine, prior to flashing it onto SDcard to then deploy on the target device. So the solution there is to run the benchmark on the device, once, record parameters and use those when creating the golden image for memory, threads and possibly iterations too? I.e. I think I managed to use printf test | sudo cryptsetup luksFormat --debug --key-file=- --pbkdf-memory=505372 --pbkdf-parallel=2 --pbkdf-force-iterations=4 /dev/nbd0 To create an "easy" encrypted filesystem with matching algo parameters for a "small" device, on my "big" desktop. > OTOH parallel unlocking might be an issue indeed, especially if the > devices are unlocked in an unattended fashion (for instance via key > files). That said on my 1GiB VM I could reliably boot with > > $ cat /etc/crypttab > root_crypt /dev/vda4 noneluks > swap_crypt /dev/vda2 /etc/keys/swap.key luks,swap > home_crypt /dev/vda3 /etc/keys/home.key luks > > (root_crypt being unlocked an initramfs stage, the other two later by > systemd) and > > $ cat /etc/fstab > /dev/vda1 /boot ext2 defaults 0 2 > /dev/mapper/swap_crypt none swap sw0 0 > /dev/mapper/root_crypt / ext4 defaults 0 1 > /dev/mapper/home_crypt /home ext4 defaults 0 2 > > However the OOM killer does trigger when I add a third volume. I guess > the boot process is racy and might, or might not, abort depending on > what's being started at the same time. But I'd argue that setup is far > from a typical installation, and the same goes for low-memory devices > with plenty of encrypted disks attached. (Moreover Debian's default > initramfs implementation comes from initramfs-tools, where unlocking is > sequential. So root-device encryption is out of the picture.) > > Having a separate encrypted /home partition (unlocked by systemd) > doesn't seem to cause any problem when formatted with the default > parameters, which I understood is what this bug report is about. > I guess dracut with systemd in the initrd might be affected worse, than initramfs-tools. I wonder if I should open a bug report in systemd, to potentially execute luks2 unlock with some locking / sequentially. Similarly, I shall probably back out hardcoding of --pbkdf pbkdf out of s390-tools zkey. I guess this bug should be tagged wontfixing and lowered priority to normal, or something. -- Regards, Dimitri.
Bug#924560: [pkg-cryptsetup-devel] Bug#924560: cryptsetup luksOpen requires 1GB of RAM in the default configuration
Control: tag -1 moreinfo Hi Dimitri, On Thu, 14 Mar 2019 at 12:36:13 +, Dimitri John Ledkov wrote: > Currently the new cryptsetup defaults to LUKS2 format with the > following parameters: > > Default PBKDF for LUKS2: argon2i > Iteration time: 2000, Memory required: 1048576kB, Parallel threads: 4 The actual parameters used for the KDF are determined by benchmark upon luksFormat (or luksAddKey). See cryptsetup(8): --pbkdf […] Note that increasing memory cost also increases time, so the final parameter values are measured by a benchmark. The benchmark tries to find iteration time (--iter-time) with required memory cost --pbkdf-memory. If it is not possible, the memory cost is decreased as well. The parallel cost --pbkdf-parallel is constant, is is checked against available CPU cores (if not available, it is decreased) and the maximum parallel cost is 4. > For example many IoT and Pi devices have 1GB of ram in total, and thus > would OOM kill when trying to luksOpen. Is that something you experienced? I just deployed a fresh a Debian sid VM with 2vCPUs, 1GiB RAM a default encryption stack (LVM2 on top of LUKS/dm-crypt). Works just fine :-) The actual memory cost is about half of the physical memory. $ cryptsetup luksDump /dev/vda5 | grep -Fe "Time cost:" -e "Memory:" -e "Threads:" Time cost: 4 Memory: 505372 Threads:2 $ dd if=/dev/zero of=/tmp/disk.img bs=1M count=64 $ printf test | cryptsetup luksFormat --debug --key-file=- /tmp/disk.img […] # Running argon2i() benchmark. # PBKDF benchmark: memory cost = 32, iterations = 4, threads = 2 (took 2 ms) # PBKDF benchmark: memory cost = 512, iterations = 4, threads = 2 (took 1 ms) # PBKDF benchmark: memory cost = 8192, iterations = 4, threads = 2 (took 18 ms) # PBKDF benchmark: memory cost = 113777, iterations = 4, threads = 2 (took 233 ms) # PBKDF benchmark: memory cost = 122078, iterations = 4, threads = 2 (took 234 ms) # PBKDF benchmark: memory cost = 130425, iterations = 4, threads = 2 (took 249 ms) # PBKDF benchmark: memory cost = 130948, iterations = 4, threads = 2 (took 251 ms) # PBKDF benchmark: memory cost = 505422, iterations = 8, threads = 2 (took 2015 ms) # Benchmark returns argon2i() 8 iterations, 505422 memory, 2 threads (for 512-bits key). […] Command successful. $ printf test | cryptsetup luksOpen --debug --key-file=- /tmp/disk.img crypt_disk […] # Only 2 active CPUs detected, PBKDF threads decreased from 4 to 2. # Not enough physical memory detected, PBKDF max memory decreased from 1048576kB to 505422kB. # PBKDF argon2i, hash sha256, time_ms 2000 (iterations 0), max_memory_kb 505422, parallel_threads 2. […] Command successful. Works equally fine on a VM with 512MiB RAM. There the memory cost is set to ~250MiB. > Please consider reducing the default memory requirement of the argon2i > in luks2 by default, or switching to pbkdf2 for LUKS2 as well. I think diverging from upstream (and other distros) with respect to default algorithms requires careful consideration. And in that case, compared to PBKDF2 Argon2 has interesting properties (such as resistance to GPU cracking) which would be a shame not to benefit from out of the box. > I think it is unfortunate to not support default encryption on 1GB big > devices and VMs. AFAICT it does. What I guess doesn't is if the machine's resources are significantly reduced between luksFormat/luksAddKey and luksOpen. OTOH parallel unlocking might be an issue indeed, especially if the devices are unlocked in an unattended fashion (for instance via key files). That said on my 1GiB VM I could reliably boot with $ cat /etc/crypttab root_crypt /dev/vda4 noneluks swap_crypt /dev/vda2 /etc/keys/swap.key luks,swap home_crypt /dev/vda3 /etc/keys/home.key luks (root_crypt being unlocked an initramfs stage, the other two later by systemd) and $ cat /etc/fstab /dev/vda1 /boot ext2 defaults 0 2 /dev/mapper/swap_crypt none swap sw0 0 /dev/mapper/root_crypt / ext4 defaults 0 1 /dev/mapper/home_crypt /home ext4 defaults 0 2 However the OOM killer does trigger when I add a third volume. I guess the boot process is racy and might, or might not, abort depending on what's being started at the same time. But I'd argue that setup is far from a typical installation, and the same goes for low-memory devices with plenty of encrypted disks attached. (Moreover Debian's default initramfs implementation comes from initramfs-tools, where unlocking is sequential. So root-device encryption is out of the picture.) Having a separate encrypted /home partition (unlocked by systemd) doesn't seem to cause any problem when formatted with the default parameters, which I understood is what this bug report is about. Cheers, -- Guilhem
Bug#924560: [pkg-cryptsetup-devel] Bug#924560: cryptsetup luksOpen requires 1GB of RAM in the default configuration
On Thu, 2019-03-14 at 12:36 +, Dimitri John Ledkov wrote: > Meaning that 1GB of RAM is required at luksOpen. This is a > significant > RAM increase compared to the previous defaults used in LUKS1. Well that's by design of Argon2, to make brute force hashing much harder for an attacker. > For example many IoT and Pi devices have 1GB of ram in total, and > thus > would OOM kill when trying to luksOpen. IoT devices typicall give a sh** about security (at least commercial ones),... so why should the majority of users suffer greatly in their security just to fulfill the anyway questionable needs of a small minority? > Please consider reducing the default memory requirement of the > argon2i > in luks2 by default, or switching to pbkdf2 for LUKS2 as well. Or one could just disable the iterations altogether,... or switch do faster MD5, or replace AES by even faster Caesar cipher... If someone really thinks he needs the just "encrypted-label" attached to their thing but doesn't care about actual security, the defaults can always manually be lowered... but why should everyone suffer from this? Debian should not further weaken the sane (and actually quite conservative) defaults of upstream. There's apparently already the decision made (against the explicit recommendation from upstream) to overwrite the no-discard default. Let's please try to not create even yet another debacle like Debian- OpenSSL-RNG in dm-crypt/cryptsetup. Cheers, Chris.
Bug#924560: cryptsetup luksOpen requires 1GB of RAM in the default configuration
Package: cryptsetup Version: 2:2.1.0-1 Severity: important Dear Maintainer, Currently the new cryptsetup defaults to LUKS2 format with the following parameters: Default PBKDF for LUKS2: argon2i Iteration time: 2000, Memory required: 1048576kB, Parallel threads: 4 Meaning that 1GB of RAM is required at luksOpen. This is a significant RAM increase compared to the previous defaults used in LUKS1. Meaning that many devices will no longer be able to installs afresh, using full-disk encryption. For example many IoT and Pi devices have 1GB of ram in total, and thus would OOM kill when trying to luksOpen. Please consider reducing the default memory requirement of the argon2i in luks2 by default, or switching to pbkdf2 for LUKS2 as well. If there are multiple encrypted datavolumes, unlocked automatically with crypttab, under systemd, they would be unlocked in parallel, meaning peak memory requirement would be 1GB*N on boot for those systems. I think it is unfortunate to not support default encryption on 1GB big devices and VMs. I have filed a similar bug report in Ubuntu as well just now: https://bugs.launchpad.net/ubuntu/+source/cryptsetup/+bug/1820049 Regards, Dimitri.
