On Wed, 16 Sep 2026, Léo Terziman wrote:
> NULL pointer dereference in mempool_free_bulk via crypt_endio during > dm-crypt device teardown > > SUMMARY > ======= > Since upgrading from Ubuntu 24.04 (Linux 6.8) to Ubuntu 26.04 (Linux 7.0), > this machine panics roughly once a week. The panic is always a NULL pointer > dereference in mempool_free_bulk(), reached from crypt_endio() in dm_crypt > via bio_put()/bio_free(), running in softirq context from a SCSI completion. > > It reproduces during teardown of a dm-crypt device: a late bio completion > arrives after the dm-crypt target's bio_set has already been destroyed, so > bio_free() returns the bio to a mempool whose ->elements is NULL. > > The system was stable on 6.8 for approximately two years with the identical > storage configuration and the identical backup script. Nothing in userspace > changed at the time the crashes began. > > > ENVIRONMENT > =========== > Distribution: Ubuntu 26.04 LTS > Kernel: 7.0.0-31-generic #31-Ubuntu PREEMPT(lazy) x86_64 > Hardware: Supermicro X13SAE-F, BIOS 5.3 06/04/2026 > Memory: 64 GB ECC (ie31200_edac; 0 CE, 0 UE reported) > DKMS modules: none > Taint: G W -- the W is an unrelated boot-time WARN in i915 > (print_ddi_port, intel_bios.c VBT parsing, headless system). > No out-of-tree modules are loaded. > > > STORAGE STACK > ============= > Four backup volumes, each: > > iSCSI (iscsi_tcp, Synology target over TCP/IP) > -> sd (SCSI disk) > -> dm-crypt > -> btrfs (zstd:3, blake2b csums, async discard enabled) > > Mounted together nightly by a backup script, then unmounted and torn down > (umount -> cryptsetup close -> iscsiadm logout) when the backup finishes. > > Separate from this, the system also uses hardware RAID -> bcache -> btrfs > for primary storage and md RAID1 for SSD storage. Neither appears in any > crash trace. > > > TRIGGER > ======= > Every crash occurs during teardown of the backup volumes, within seconds of > unmount. The preceding log is consistent across occurrences: > > [256109.45] BTRFS info (device dm-15): last unmount of filesystem ... > [256109.53] sd 9:0:0:1: [sdb] Synchronizing SCSI cache > [256109.75] BTRFS info (device dm-16): last unmount of filesystem ... > [256109.96] sd 10:0:0:1: [sdc] Synchronizing SCSI cache > [256110.06] BTRFS info (device dm-17): last unmount of filesystem ... > [256110.17] sd 11:0:0:1: [sdd] Synchronizing SCSI cache > [256111.50] BTRFS info (device dm-18): last unmount of filesystem ... > [256115.15] BTRFS warning (device dm-18): folio private not zero on folio > 906100736 > [~100 more identical warnings for consecutive folios] > [256115.18] BUG: kernel NULL pointer dereference, address: 0000000000000000 > > The "folio private not zero" flood immediately before the oops indicates > btrfs is releasing folios that still carry I/O state, i.e. the unmount is > completing while work is still outstanding beneath it. I suspect that this is btrfs bug. It seems that btrfs closes a block device without waiting for bios to finish. The mempool_free function was reworked between 6.8 and 7.0. In 6.8 it didn't crash if we attempted to free an entry into a free mempool. In 7.0 it crashes in this case. So, the btrfs bug could have been there forever, it was just latent and didn't result in a crash on old kernels. You can try to apply the patch 83f7e52b7ed1c3e03b79123e20b6f6adf8d886bb - maybe it helps. Mikulas
