Package: ceph Version: 18.2.7+ds-1 Severity: grave Justification: renders package unusable
Dear Maintainer, I am running ceph-fs across at least five different machines. After a whole-house power outage, two of the machines on debian stable can no longer access the ceph file system, they get kernel null pointer derefs. Other machines (on different kernel versions) work. The working kernels include: $ cat /proc/version Linux version 6.12.57+deb13-amd64 ([email protected]) (x86_64-linux-gnu-gcc-14 (Debian 14.2.0-19) 14.2.0, GNU ld (GNU Binutils for Debian) 2.44) #1 SMP PREEMPT_DYNAMIC Debian 6.12.57-1 (2025-11-05) $ cat /proc/version Linux version 6.18.5+deb14-amd64 ([email protected]) (x86_64-linux-gnu-gcc-15 (Debian 15.2.0-12) 15.2.0, GNU ld (GNU Binutils for Debian) 2.45.50.20251209) #1 SMP PREEMPT_DYNAMIC Debian 6.18.5-1 (2026-01-16) The non-working kernels are both: # cat /proc/version Linux version 6.12.63+deb13-amd64 ([email protected]) (x86_64-linux-gnu-gcc-14 (Debian 14.2.0-19) 14.2.0, GNU ld (GNU Binutils for Debian) 2.44) #1 SMP PREEMPT_DYNAMIC Debian 6.12.63-1 (2025-12-30) I include several stack traces below, one for each machine. The system information here is for one of the machines; it should be the same on the other non-working machine as well. (The config has been working great for over a year; the issue occurred only after reboot to the latest kernels.) -- System Information: Debian Release: 13.3 APT prefers stable-updates APT policy: (500, 'stable-updates'), (500, 'stable-security'), (500, 'stable') Architecture: amd64 (x86_64) Kernel: Linux 6.12.63+deb13-amd64 (SMP w/32 CPU threads; PREEMPT) Kernel taint flags: TAINT_DIE, TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set Shell: /bin/sh linked to /usr/bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enabled Versions of packages ceph depends on: ii ceph-mgr 18.2.7+ds-1 ii ceph-mon 18.2.7+ds-1 ii ceph-osd 18.2.7+ds-1 ceph recommends no packages. Versions of packages ceph suggests: ii ceph-mds 18.2.7+ds-1 -- no debconf information [257335.136735] BUG: kernel NULL pointer dereference, address: 0000000000000000 [257335.137170] #PF: supervisor read access in kernel mode [257335.137474] #PF: error_code(0x0000) - not-present page [257335.137758] PGD 0 P4D 0 [257335.138029] Oops: Oops: 0000 [#1] PREEMPT SMP NOPTI [257335.138303] CPU: 9 UID: 1000 PID: 1299922 Comm: ls Tainted: G S 6.12.63+deb13-amd64 #1 Debian 6.12.63-1 [257335.138599] Tainted: [S]=CPU_OUT_OF_SPEC [257335.138877] Hardware name: ASUS System Product Name/Pro WS WRX80E-SAGE SE WIFI, BIOS 1003 02/18/2022 [257335.139157] RIP: 0010:strcmp+0x1c/0x40 [257335.139422] Code: 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 31 c0 eb 10 0f 1f 84 00 00 00 00 00 48 83 c0 01 84 d2 74 13 0f b6 14 07 <3a> 14 06 74 ef 19 c0 83 c8 01 e9 90 4e 27 00 31 c0 e9 89 4e 27 00 [257335.140017] RSP: 0018:ffffbba6631879e0 EFLAGS: 00010246 [257335.140330] RAX: 0000000000000000 RBX: ffff969844ba4980 RCX: ffffffff8b64c704 [257335.140646] RDX: 0000000000000068 RSI: 0000000000000000 RDI: ffff96984211bda0 [257335.140960] RBP: 00000000000003e8 R08: 0000000000000000 R09: 0000000000000000 [257335.141264] R10: 0000000000000001 R11: 0000000000000002 R12: 0000000000000000 [257335.141570] R13: ffff969ac2d40000 R14: ffff96984a025000 R15: ffff96992046a780 [257335.141882] FS: 00007f492196c240(0000) GS:ffff97157d480000(0000) knlGS:0000000000000000 [257335.142201] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [257335.142507] CR2: 0000000000000000 CR3: 000000089843a003 CR4: 0000000000f70ef0 [257335.142816] PKRU: 55555554 [257335.143135] Call Trace: [257335.143454] <TASK> [257335.143775] ceph_mds_check_access+0x10c/0x790 [ceph] [257335.144184] ceph_open+0x171/0x5c0 [ceph] [257335.144559] ? __pfx_ceph_open+0x10/0x10 [ceph] [257335.144915] do_dentry_open+0x14f/0x440 [257335.145254] vfs_open+0x2e/0xe0 [257335.145570] path_openat+0x82e/0x12d0 [257335.145900] ? srso_alias_return_thunk+0x5/0xfbef5 [257335.146230] ? srso_alias_return_thunk+0x5/0xfbef5 [257335.146552] ? arch_exit_to_user_mode_prepare.isra.0+0x16/0xa0 [257335.146884] ? srso_alias_return_thunk+0x5/0xfbef5 [257335.147212] ? syscall_exit_to_user_mode+0x37/0x1b0 [257335.147543] do_filp_open+0xc4/0x170 [257335.147880] do_sys_openat2+0xae/0xe0 [257335.148216] ? srso_alias_return_thunk+0x5/0xfbef5 [257335.148562] __x64_sys_openat+0x55/0xa0 [257335.148910] do_syscall_64+0x82/0x190 [257335.149259] ? srso_alias_return_thunk+0x5/0xfbef5 [257335.149579] ? dput+0x32/0x1b0 [257335.149906] ? srso_alias_return_thunk+0x5/0xfbef5 [257335.150231] ? __fsnotify_parent+0x153/0x3a0 [257335.150561] ? srso_alias_return_thunk+0x5/0xfbef5 [257335.150895] ? __memcg_slab_free_hook+0xf7/0x140 [257335.151221] ? srso_alias_return_thunk+0x5/0xfbef5 [257335.151538] ? kmem_cache_free+0x3ee/0x440 [257335.151867] ? srso_alias_return_thunk+0x5/0xfbef5 [257335.152187] ? srso_alias_return_thunk+0x5/0xfbef5 [257335.152502] ? arch_exit_to_user_mode_prepare.isra.0+0x16/0xa0 [257335.152816] ? srso_alias_return_thunk+0x5/0xfbef5 [257335.153127] ? syscall_exit_to_user_mode+0x37/0x1b0 [257335.153427] ? srso_alias_return_thunk+0x5/0xfbef5 [257335.153721] ? do_syscall_64+0x8e/0x190 [257335.154015] ? srso_alias_return_thunk+0x5/0xfbef5 [257335.154299] ? arch_exit_to_user_mode_prepare.isra.0+0x16/0xa0 [257335.154582] ? srso_alias_return_thunk+0x5/0xfbef5 [257335.154855] entry_SYSCALL_64_after_hwframe+0x76/0x7e [257335.155116] RIP: 0033:0x7f4921b20b7c [257335.155399] Code: 4c 89 54 24 18 41 89 f2 41 83 e2 40 75 40 89 f0 f7 d0 a9 00 00 41 00 74 35 89 f2 b8 01 01 00 00 48 89 fe bf 9c ff ff ff 0f 05 <48> 3d 00 f0 ff ff 77 44 48 8b 54 24 18 64 48 2b 14 25 28 00 00 00 [257335.155949] RSP: 002b:00007ffe14f5a220 EFLAGS: 00000206 ORIG_RAX: 0000000000000101 [257335.156237] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f4921b20b7c [257335.156522] RDX: 0000000000090800 RSI: 0000562444659010 RDI: 00000000ffffff9c [257335.156817] RBP: 0000000000000000 R08: 0000000000000020 R09: 00007f4921c02b20 [257335.157104] R10: 0000000000000000 R11: 0000000000000206 R12: 00007f492196c0e0 [257335.157389] R13: 00007ffe14f5a660 R14: 0000562444659010 R15: 000056244464ff40 [257335.157674] </TASK> [257335.157941] Modules linked in: veth nf_conntrack_netlink xfrm_user br_netfilter msr ceph netfs nft_masq rbd libceph bridge stp llc overlay nft_chain_nat xt_nat xt_MASQUERADE nf_nat xt_addrtype xt_tcpudp xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nft_compat nf_tables binfmt_misc ipmi_ssif nls_ascii nls_cp437 vfat fat amd_atl intel_rapl_msr intel_rapl_common amd64_edac edac_mce_amd iwlmvm kvm_amd mac80211 kvm libarc4 snd_hda_intel snd_intel_dspcfg snd_usb_audio snd_intel_sdw_acpi irqbypass btusb snd_hda_codec crct10dif_pclmul iwlwifi snd_usbmidi_lib ghash_clmulni_intel btrtl sha512_ssse3 snd_rawmidi btintel snd_hda_core sha256_ssse3 snd_seq_device btbcm mc sha1_ssse3 btmtk snd_hwdep snd_pcm ast aesni_intel eeepc_wmi asus_wmi gf128mul bluetooth cfg80211 sparse_keymap drm_shmem_helper acpi_ipmi snd_timer crypto_simd platform_profile cryptd evdev jc42 ecdh_generic battery drm_kms_helper ipmi_si snd i2c_algo_bit mxm_wmi pcspkr video ipmi_devintf rapl ccp wmi_bmof soundcore rfkill k10temp ee1004 ipmi_msghandler [257335.158067] button sg nvme_fabrics nvme_keyring drm efi_pstore configfs nfnetlink efivarfs ip_tables x_tables autofs4 ext4 crc16 mbcache jbd2 crc32c_generic btrfs xor raid6_pq libcrc32c hid_generic usbhid hid sd_mod dm_mod ahci libahci ixgbe libata sp5100_tco xhci_pci xfrm_algo xhci_hcd watchdog mdio_devres nvme libphy scsi_mod crc32_pclmul usbcore nvme_core mdio crc32c_intel i2c_piix4 scsi_common dca nvme_auth usb_common i2c_smbus wmi [257335.162707] CR2: 0000000000000000 [257335.163118] ---[ end trace 0000000000000000 ]--- [257335.213175] RIP: 0010:strcmp+0x1c/0x40 [257335.213982] Code: 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 31 c0 eb 10 0f 1f 84 00 00 00 00 00 48 83 c0 01 84 d2 74 13 0f b6 14 07 <3a> 14 06 74 ef 19 c0 83 c8 01 e9 90 4e 27 00 31 c0 e9 89 4e 27 00 [257335.215155] RSP: 0018:ffffbba6631879e0 EFLAGS: 00010246 [257335.215677] RAX: 0000000000000000 RBX: ffff969844ba4980 RCX: ffffffff8b64c704 [257335.216176] RDX: 0000000000000068 RSI: 0000000000000000 RDI: ffff96984211bda0 [257335.216678] RBP: 00000000000003e8 R08: 0000000000000000 R09: 0000000000000000 [257335.217182] R10: 0000000000000001 R11: 0000000000000002 R12: 0000000000000000 [257335.217670] R13: ffff969ac2d40000 R14: ffff96984a025000 R15: ffff96992046a780 [257335.218144] FS: 00007f492196c240(0000) GS:ffff97157d480000(0000) knlGS:0000000000000000 [257335.218632] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [257335.219121] CR2: 0000000000000000 CR3: 000000089843a003 CR4: 0000000000f70ef0 [257335.219603] PKRU: 55555554 [257335.220080] note: ls[1299922] exited with irqs disabled ============================================================= and on a different machine: [40126.841779] BUG: kernel NULL pointer dereference, address: 0000000000000000 [40126.841786] #PF: supervisor read access in kernel mode [40126.841787] #PF: error_code(0x0000) - not-present page [40126.841788] PGD 0 P4D 0 [40126.841790] Oops: Oops: 0000 [#1] PREEMPT SMP NOPTI [40126.841792] CPU: 21 UID: 0 PID: 61260 Comm: updatedb.plocat Tainted: G OE 6.12.63+deb13-amd64 #1 Debian 6.12.63-1 [40126.841795] Tainted: [O]=OOT_MODULE, [E]=UNSIGNED_MODULE [40126.841796] Hardware name: ASRock X870E Taichi/X870E Taichi, BIOS 3.20 02/21/2025 [40126.841797] RIP: 0010:strcmp+0x1c/0x40 [40126.841801] Code: 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 31 c0 eb 10 0f 1f 84 00 00 00 00 00 48 83 c0 01 84 d2 74 13 0f b6 14 07 <3a> 14 06 74 ef 19 c0 83 c8 01 c3 cc cc cc cc 31 c0 c3 cc cc cc cc [40126.841802] RSP: 0018:ffffc20189e43940 EFLAGS: 00010246 [40126.841803] RAX: 0000000000000000 RBX: ffff9d7551d50540 RCX: ffffffffaf24c704 [40126.841803] RDX: 0000000000000068 RSI: 0000000000000000 RDI: ffff9d7467462f10 [40126.841804] RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000 [40126.841804] R10: 0000000000000001 R11: 0000000000000002 R12: 0000000000000000 [40126.841805] R13: ffff9d740f5a2000 R14: ffff9d73185ed800 R15: ffff9d73026e1ec0 [40126.841806] FS: 00007fab706c6780(0000) GS:ffff9d915f880000(0000) knlGS:0000000000000000 [40126.841807] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [40126.841807] CR2: 0000000000000000 CR3: 000000073b1b2000 CR4: 0000000000f50ef0 [40126.841808] PKRU: 55555554 [40126.841809] Call Trace: [40126.841811] <TASK> [40126.841815] ceph_mds_check_access+0x10c/0x790 [ceph] [40126.841827] ceph_open+0x171/0x5c0 [ceph] [40126.841834] ? __pfx_ceph_open+0x10/0x10 [ceph] [40126.841838] do_dentry_open+0x14c/0x440 [40126.841840] vfs_open+0x2e/0xe0 [40126.841842] path_openat+0x82e/0x12d0 [40126.841844] do_filp_open+0xc4/0x170 [40126.841846] do_sys_openat2+0xae/0xe0 [40126.841848] __x64_sys_openat+0x55/0xa0 [40126.841849] do_syscall_64+0x82/0x190 [40126.841851] ? from_kgid_munged+0x12/0x30 [40126.841853] ? cp_new_stat+0x131/0x170 [40126.841855] ? __do_sys_newfstat+0x68/0x70 [40126.841856] ? syscall_exit_to_user_mode_prepare+0x149/0x170 [40126.841858] ? arch_exit_to_user_mode_prepare.isra.0+0x16/0xa0 [40126.841859] ? syscall_exit_to_user_mode+0x37/0x1b0 [40126.841860] ? do_syscall_64+0x8e/0x190 [40126.841861] ? arch_exit_to_user_mode_prepare.isra.0+0x16/0xa0 [40126.841862] entry_SYSCALL_64_after_hwframe+0x76/0x7e [40126.841864] RIP: 0033:0x7fab701b59ee [40126.841884] Code: 08 0f 85 f5 4b ff ff 49 89 fb 48 89 f0 48 89 d7 48 89 ce 4c 89 c2 4d 89 ca 4c 8b 44 24 08 4c 8b 4c 24 10 4c 89 5c 24 08 0f 05 <c3> 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 80 00 00 00 00 48 83 ec 08 [40126.841884] RSP: 002b:00007fff8ca8a4f8 EFLAGS: 00000246 ORIG_RAX: 0000000000000101 [40126.841886] RAX: ffffffffffffffda RBX: 00007fab706c6780 RCX: 00007fab701b59ee [40126.841886] RDX: 0000000000050000 RSI: 000055b403783490 RDI: 0000000000000006 [40126.841887] RBP: 000055b403783490 R08: 0000000000000000 R09: 0000000000000000 [40126.841887] R10: 0000000000000000 R11: 0000000000000246 R12: 000055b40377b110 [40126.841887] R13: 0000000000000000 R14: 00007fff8ca8a700 R15: 000055b403783480 [40126.841888] </TASK> [40126.841889] Modules linked in: nvidia_drm(OE) nvidia_modeset(OE) nvidia(OE) drm_ttm_helper ttm drm_kms_helper snd_seq_dummy snd_hrtimer snd_seq nft_masq nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ceph libceph bridge netfs stp llc nf_tables qrtr binfmt_misc nls_ascii nls_cp437 vfat fat amd_atl intel_rapl_msr intel_rapl_common mt7925e mt7925_common edac_mce_amd mt792x_lib mt76_connac_lib kvm_amd mt76 mac80211 kvm btusb snd_hda_codec_hdmi btrtl btintel snd_hda_intel btbcm libarc4 snd_intel_dspcfg btmtk irqbypass snd_intel_sdw_acpi snd_usb_audio bluetooth crct10dif_pclmul snd_hda_codec snd_usbmidi_lib ghash_clmulni_intel cfg80211 sha512_ssse3 snd_rawmidi sha256_ssse3 sha1_ssse3 snd_hda_core snd_seq_device aesni_intel mc snd_hwdep gf128mul ecdh_generic crypto_simd snd_pcm corsair_psu cryptd ccp wmi_bmof rapl snd_timer pcspkr k10temp snd rfkill soundcore joydev sg evdev drm parport_pc ppdev lp parport nvme_fabrics nvme_keyring configfs efi_pstore nfnetlink ip_tables x_tables autofs4 ext4 crc16 mbcache [40126.841922] jbd2 btrfs blake2b_generic raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c crc32c_generic raid0 raid1 hid_generic usbhid hid md_mod sd_mod xhci_pci dm_mod xhci_hcd ahci libahci libata r8169 sp5100_tco thunderbolt usbcore nvme realtek watchdog scsi_mod mdio_devres nvme_core libphy crc32_pclmul i2c_piix4 crc32c_intel usb_common i2c_smbus scsi_common video nvme_auth wmi gpio_amdpt gpio_generic button efivarfs [last unloaded: vfio] [40126.841941] CR2: 0000000000000000 [40126.841943] ---[ end trace 0000000000000000 ]--- [40127.152257] RIP: 0010:strcmp+0x1c/0x40 [40127.152265] Code: 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 31 c0 eb 10 0f 1f 84 00 00 00 00 00 48 83 c0 01 84 d2 74 13 0f b6 14 07 <3a> 14 06 74 ef 19 c0 83 c8 01 c3 cc cc cc cc 31 c0 c3 cc cc cc cc [40127.152266] RSP: 0018:ffffc20189e43940 EFLAGS: 00010246 [40127.152269] RAX: 0000000000000000 RBX: ffff9d7551d50540 RCX: ffffffffaf24c704 [40127.152269] RDX: 0000000000000068 RSI: 0000000000000000 RDI: ffff9d7467462f10 [40127.152270] RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000 [40127.152270] R10: 0000000000000001 R11: 0000000000000002 R12: 0000000000000000 [40127.152271] R13: ffff9d740f5a2000 R14: ffff9d73185ed800 R15: ffff9d73026e1ec0 [40127.152271] FS: 00007fab706c6780(0000) GS:ffff9d915f880000(0000) knlGS:0000000000000000 [40127.152272] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [40127.152273] CR2: 0000000000000000 CR3: 000000073b1b2000 CR4: 0000000000f50ef0 [40127.152273] PKRU: 55555554 [40127.152274] note: updatedb.plocat[61260] exited with irqs disabled [40127.152257] RIP: 0010:strcmp+0x1c/0x40 [40127.152265] Code: 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 31 c0 eb 10 0f 1f 84 00 00 00 00 00 48 83 c0 01 84 d2 74 13 0f b6 14 07 <3a> 14 06 74 ef 19 c0 83 c8 01 c3 cc cc cc cc 31 c0 c3 cc cc cc cc [40127.152266] RSP: 0018:ffffc20189e43940 EFLAGS: 00010246 [40127.152269] RAX: 0000000000000000 RBX: ffff9d7551d50540 RCX: ffffffffaf24c704 [40127.152269] RDX: 0000000000000068 RSI: 0000000000000000 RDI: ffff9d7467462f10 [40127.152270] RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000 [40127.152270] R10: 0000000000000001 R11: 0000000000000002 R12: 0000000000000000 [40127.152271] R13: ffff9d740f5a2000 R14: ffff9d73185ed800 R15: ffff9d73026e1ec0 [40127.152271] FS: 00007fab706c6780(0000) GS:ffff9d915f880000(0000) knlGS:0000000000000000 [40127.152272] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [40127.152273] CR2: 0000000000000000 CR3: 000000073b1b2000 CR4: 0000000000f50ef0 [40127.152273] PKRU: 55555554 [40127.152274] note: updatedb.plocat[61260] exited with irqs disabled
