With no_write_workqueue, kcryptd_crypt_write_io_submit() submits the
bio directly, even when called from kcryptd_async_done(). An async
cipher may run this callback in softirq or with BH disabled, where
submit_bio_noacct() must not be called because it can sleep.
Submit inline only when async is false. Async completions are sent to
the write thread, as is already done for submit_from_crypt_cpus. Use
the async flag rather than in_interrupt(), since callback context
depends on the cipher driver. Sync ciphers still submit inline.
Tested with cryptd in QEMU (x86_64 and arm64) with
CONFIG_DEBUG_ATOMIC_SLEEP, and on a P2020 running 6.18.55 without it.
In QEMU:
- 4k writes logging "sleeping function called from invalid context":
100/100 unpatched, 0/155 patched.
- nr_requests=4 on a loop device: 0/6 unpatched runs completed,
4/4 patched runs completed.
On the P2020, a warning in blk_mq_run_hw_queue() appeared in the
6th unpatched run and did not appear in 30 patched runs.
Fixes: 39d42fa96ba1 ("dm crypt: add flags to optionally bypass kcryptd
workqueues")
Cc: [email protected]
Signed-off-by: Josef Schlehofer <[email protected]>
---
Based on dm-7.4 (dab527d83462).
To reproduce, with CONFIG_DEBUG_ATOMIC_SLEEP, DM_CRYPT, BLK_DEV_LOOP,
CRYPTO_CRYPTD, CRYPTO_XTS and CRYPTO_AES:
modprobe -a cryptd dm_crypt loop
truncate -s 256M /dev/shm/nowq.img
L=$(losetup -f)
losetup "$L" /dev/shm/nowq.img
KEY=$(od -An -v -tx1 -N64 /dev/urandom | tr -d ' \n')
dmsetup create nowq --table "0 $((256 * 2048)) crypt
capi:cryptd(xts-aes-lib)-plain64 $KEY 0 $L 0 2 no_read_workqueue
no_write_workqueue"
D=/dev/dm-$(dmsetup info -c --noheadings -o minor nowq)
dd if=/dev/urandom of="$D" bs=4k count=1 conv=fsync
dmesg | grep 'sleeping function called'
For the nr_requests=4 case, run this instead of the dd. The writes did
not finish without this patch:
echo 4 > /sys/block/${L##*/}/queue/nr_requests
for i in 1 2 3 4; do dd if=/dev/urandom of="$D" bs=64k count=512 seek=$((i *
512)) conv=fsync & done; wait
drivers/md/dm-crypt.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c
index 8853e9af1c3d..c0552a300e62 100644
--- a/drivers/md/dm-crypt.c
+++ b/drivers/md/dm-crypt.c
@@ -1982,8 +1982,9 @@ static void kcryptd_crypt_write_io_submit(struct
dm_crypt_io *io, int async)
/* crypt_convert should have filled the clone bio */
BUG_ON(io->ctx.iter_out.bi_size);
- if ((likely(!async) && test_bit(DM_CRYPT_NO_OFFLOAD, &cc->flags)) ||
- test_bit(DM_CRYPT_NO_WRITE_WORKQUEUE, &cc->flags)) {
+ if (likely(!async) &&
+ (test_bit(DM_CRYPT_NO_OFFLOAD, &cc->flags) ||
+ test_bit(DM_CRYPT_NO_WRITE_WORKQUEUE, &cc->flags))) {
dm_submit_bio_remap(io->base_bio, clone);
return;
}
--
2.54.0 (Apple Git-157)
