Public bug reported:
Hi,
I have a fairly simple NFS server setup, running on top of a somewhat large XFS
filesystem (102TB).
This morning the nfs service was interrupted and was not operational until it
crashed hard ~2 hours later and was powercycled via iDRAC.
There was practically no load, especially nfs load on the system during or
before that period. It does get much more loaded during the day.
The server is running:
Linux myserver 7.0.0-38-generic #38~24.04.4-Ubuntu SMP PREEMPT_DYNAMIC
Mon Sep 14 16:37:11 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
The server crashed around 2 hours later while dumping traces from various nfsd
code paths. This is the relevant bits from dmesg, I omitted all but the first
two for brevity:
2026-10-21T06:28:37.136775+02:00 myserver kernel: UBSAN:
array-index-out-of-bounds in
/build/linux-hwe-7.0-gwDnwj/linux-hwe-7.0-7.0.0/kernel/locking/qspinlock.h:68:9
2026-10-21T06:28:37.136781+02:00 myserver kernel: index 16382 is out of range
for type 'long unsigned int [8192]'
2026-10-21T06:28:37.136786+02:00 myserver kernel: CPU: 5 UID: 0 PID: 3022700
Comm: kworker/u258:7 Not tainted 7.0.0-31-generic #31~24.04.1-Ubuntu
PREEMPT(lazy)
2026-10-21T06:28:37.136802+02:00 myserver kernel: Hardware name: Dell Inc.
PowerEdge R740/0RR8YK, BIOS 2.26.1 01/28/2026
2026-10-21T06:28:37.136829+02:00 myserver kernel: Workqueue: rpciod
rpc_async_schedule [sunrpc]
2026-10-21T06:28:37.136834+02:00 myserver kernel: Call Trace:
2026-10-21T06:28:37.136845+02:00 myserver kernel: <TASK>
2026-10-21T06:28:37.136846+02:00 myserver kernel: dump_stack_lvl+0x76/0xa0
2026-10-21T06:28:37.136848+02:00 myserver kernel: dump_stack+0x10/0x20
2026-10-21T06:28:37.136850+02:00 myserver kernel:
__ubsan_handle_out_of_bounds+0xdd/0x120
2026-10-21T06:28:37.136859+02:00 myserver kernel:
native_queued_spin_lock_slowpath+0x318/0x320
2026-10-21T06:28:37.136861+02:00 myserver kernel: _raw_spin_lock+0x3f/0x60
2026-10-21T06:28:37.137542+02:00 myserver kernel:
nfsd41_cb_release_slot+0x44/0xa0 [nfsd]
2026-10-21T06:28:37.137550+02:00 myserver kernel: ?
__pfx_rpc_exit_task+0x10/0x10 [sunrpc]
2026-10-21T06:28:37.137552+02:00 myserver kernel:
nfsd4_cb_sequence_done+0xa9/0x230 [nfsd]
2026-10-21T06:28:37.137554+02:00 myserver kernel: ?
__pfx_rpc_exit_task+0x10/0x10 [sunrpc]
2026-10-21T06:28:37.137555+02:00 myserver kernel: nfsd4_cb_done+0x118/0x1b0
[nfsd]
2026-10-21T06:28:37.137556+02:00 myserver kernel: ?
__pfx_rpc_exit_task+0x10/0x10 [sunrpc]
2026-10-21T06:28:37.137558+02:00 myserver kernel: rpc_exit_task+0x64/0x1a0
[sunrpc]
2026-10-21T06:28:37.137559+02:00 myserver kernel: __rpc_execute+0xaf/0x370
[sunrpc]
2026-10-21T06:28:37.137560+02:00 myserver kernel: rpc_async_schedule+0x33/0x50
[sunrpc]
2026-10-21T06:28:37.137562+02:00 myserver kernel: process_one_work+0x1af/0x430
2026-10-21T06:28:37.137563+02:00 myserver kernel: worker_thread+0x1bf/0x350
2026-10-21T06:28:37.137564+02:00 myserver kernel: ?
__pfx_worker_thread+0x10/0x10
2026-10-21T06:28:37.137566+02:00 myserver kernel: kthread+0xfb/0x140
2026-10-21T06:28:37.137574+02:00 myserver kernel: ? __pfx_kthread+0x10/0x10
2026-10-21T06:28:37.137580+02:00 myserver kernel: ret_from_fork+0x1f8/0x2b0
2026-10-21T06:28:37.137582+02:00 myserver kernel: ? __pfx_kthread+0x10/0x10
2026-10-21T06:28:37.137583+02:00 myserver kernel: ret_from_fork_asm+0x1a/0x30
2026-10-21T06:28:37.137585+02:00 myserver kernel: </TASK>
2026-10-21T06:28:37.137586+02:00 myserver kernel: ---[ end trace ]---
2026-10-21T06:28:37.137588+02:00 myserver kernel: BUG: unable to handle page
fault for address: ffffffffac451720
2026-10-21T06:28:43.455500+02:00 myserver kernel: #PF: supervisor write access
in kernel mode
2026-10-21T06:28:43.455526+02:00 myserver kernel: #PF: error_code(0x0002) -
not-present page
2026-10-21T06:28:43.455528+02:00 myserver kernel: PGD 1053c847067 P4D
1053c847067 PUD 1053c848063 PMD 100959063 PTE 800ffefac2dae062
2026-10-21T06:28:43.455530+02:00 myserver kernel: Oops: Oops: 0002 [#1] SMP PTI
2026-10-21T06:28:43.455532+02:00 myserver kernel: CPU: 5 UID: 0 PID: 3022700
Comm: kworker/u258:7 Not tainted 7.0.0-31-generic #31~24.04.1-Ubuntu
PREEMPT(lazy)
2026-10-21T06:28:43.455534+02:00 myserver kernel: Hardware name: Dell Inc.
PowerEdge R740/0RR8YK, BIOS 2.26.1 01/28/2026
2026-10-21T06:28:43.455536+02:00 myserver kernel: Workqueue: rpciod
rpc_async_schedule [sunrpc]
2026-10-21T06:28:43.455537+02:00 myserver kernel: RIP:
0010:native_queued_spin_lock_slowpath+0x2c8/0x320
2026-10-21T06:28:43.455539+02:00 myserver kernel: Code: f6 c1 ea 12 83 e0 03 44
8d 7a ff 48 c1 e0 05 4d 63 ff 4c 8d a8 c0 16 45 ac 49 81 ff 00 20 00 00 73 49
4e 03 2c fd e0 10 05 ab <4d> 89 65 00 41 8b 44 24 08 85 c0 75 0b f3 90 41 8b 44
24 08 85 c0
2026-10-21T06:28:43.455541+02:00 myserver kernel: RSP: 0018:ffffcf1900b7bce0
EFLAGS: 00010282
2026-10-21T06:28:43.455542+02:00 myserver kernel: RAX: 0000000000000000 RBX:
ffff8b8cd9939c04 RCX: 0000000000000000
2026-10-21T06:28:43.455544+02:00 myserver kernel: RDX: 0000000000000000 RSI:
0000000000000000 RDI: 0000000000000000
2026-10-21T06:28:43.455545+02:00 myserver kernel: RBP: ffffcf1900b7bd08 R08:
0000000000000000 R09: 0000000000000000
2026-10-21T06:28:43.455546+02:00 myserver kernel: R10: 0000000000000000 R11:
0000000000000000 R12: ffff8c46bef346c0
2026-10-21T06:28:43.455548+02:00 myserver kernel: R13: ffffffffac451720 R14:
0000000000180000 R15: 0000000000003ffe
2026-10-21T06:28:43.455553+02:00 myserver kernel: FS: 0000000000000000(0000)
GS:ffff8c4712ae3000(0000) knlGS:0000000000000000
2026-10-21T06:28:43.455576+02:00 myserver kernel: CS: 0010 DS: 0000 ES: 0000
CR0: 0000000080050033
2026-10-21T06:28:43.455577+02:00 myserver kernel: CR2: ffffffffac451720 CR3:
000001053c844002 CR4: 00000000007726f0
2026-10-21T06:28:43.455579+02:00 myserver kernel: PKRU: 55555554
2026-10-21T06:28:43.455581+02:00 myserver kernel: Call Trace:
2026-10-21T06:28:43.455589+02:00 myserver kernel: <TASK>
2026-10-21T06:28:43.455591+02:00 myserver kernel: _raw_spin_lock+0x3f/0x60
2026-10-21T06:28:43.455592+02:00 myserver kernel:
nfsd41_cb_release_slot+0x44/0xa0 [nfsd]
2026-10-21T06:28:43.455594+02:00 myserver kernel: ?
__pfx_rpc_exit_task+0x10/0x10 [sunrpc]
2026-10-21T06:28:43.455595+02:00 myserver kernel:
nfsd4_cb_sequence_done+0xa9/0x230 [nfsd]
2026-10-21T06:28:43.455596+02:00 myserver kernel: ?
__pfx_rpc_exit_task+0x10/0x10 [sunrpc]
2026-10-21T06:28:43.455602+02:00 myserver kernel: nfsd4_cb_done+0x118/0x1b0
[nfsd]
2026-10-21T06:28:43.455604+02:00 myserver kernel: ?
__pfx_rpc_exit_task+0x10/0x10 [sunrpc]
2026-10-21T06:28:43.455605+02:00 myserver kernel: rpc_exit_task+0x64/0x1a0
[sunrpc]
2026-10-21T06:28:43.455607+02:00 myserver kernel: __rpc_execute+0xaf/0x370
[sunrpc]
2026-10-21T06:28:43.455608+02:00 myserver kernel: rpc_async_schedule+0x33/0x50
[sunrpc]
2026-10-21T06:28:43.455609+02:00 myserver kernel: process_one_work+0x1af/0x430
2026-10-21T06:28:43.455610+02:00 myserver kernel: worker_thread+0x1bf/0x350
2026-10-21T06:28:43.455616+02:00 myserver kernel: ?
__pfx_worker_thread+0x10/0x10
2026-10-21T06:28:43.455618+02:00 myserver kernel: kthread+0xfb/0x140
2026-10-21T06:28:43.455620+02:00 myserver kernel: ? __pfx_kthread+0x10/0x10
2026-10-21T06:28:43.455621+02:00 myserver kernel: ret_from_fork+0x1f8/0x2b0
2026-10-21T06:28:43.455622+02:00 myserver kernel: ? __pfx_kthread+0x10/0x10
2026-10-21T06:28:43.455624+02:00 myserver kernel: ret_from_fork_asm+0x1a/0x30
2026-10-21T06:28:43.455630+02:00 myserver kernel: </TASK>
2026-10-21T06:28:43.455632+02:00 myserver kernel: Modules linked in: mptcp_diag
xsk_diag tcp_diag udp_diag raw_diag inet_diag unix_diag af_packet_diag
netlink_diag nfsv3 cpuid st tls ib_core bluetooth rpcsec_gss_krb5 nfsv4 nfs
ceph libceph krb5enc authenc camellia_generic camellia_aesni_avx2
camellia_aesni_avx_x86_64 camellia_x86_64 cmac krb5 netfs dell_rbu binfmt_misc
xt_tcpudp xt_comment xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4
nft_compat nf_tables intel_rapl_msr intel_rapl_common intel_uncore_frequency
intel_uncore_frequency_common skx_edac skx_edac_common xfs nfit
x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm cmdlinepart
irqbypass dell_smbios platform_profile ipmi_ssif spi_nor ghash_clmulni_intel
aesni_intel i2c_i801 dcdbas mtd mei_me dell_wmi_descriptor rapl mei
intel_cstate mgag200 i2c_smbus wmi_bmof joydev input_leds acpi_power_meter
intel_pch_thermal spi_intel_pci ipmi_si i2c_mux spi_intel lpc_ich acpi_ipmi
ipmi_devintf ipmi_msghandler mac_hid sch_fq_codel lp parport nvme_fabrics
nvme_core nvme_keyring nvme_auth hkdf
2026-10-21T06:28:43.455635+02:00 myserver kernel: efi_pstore nfsd auth_rpcgss
nfs_acl lockd grace sunrpc nfnetlink dmi_sysfs ip_tables x_tables autofs4
hid_generic ixgbe igb usbhid libie_fwlog vga16fb ahci i2c_algo_bit xfrm_algo
hid megaraid_sas dca libahci vgastate wmi mdio
2026-10-21T06:28:43.455636+02:00 myserver kernel: CR2: ffffffffac451720
2026-10-21T06:28:43.455638+02:00 myserver kernel: ---[ end trace
0000000000000000 ]---
2026-10-21T06:28:43.455644+02:00 myserver kernel: pstore: backend (erst)
writing error (-22)
2026-10-21T06:28:43.455645+02:00 myserver kernel: RIP:
0010:native_queued_spin_lock_slowpath+0x2c8/0x320
2026-10-21T06:28:43.455647+02:00 myserver kernel: Code: f6 c1 ea 12 83 e0 03 44
8d 7a ff 48 c1 e0 05 4d 63 ff 4c 8d a8 c0 16 45 ac 49 81 ff 00 20 00 00 73 49
4e 03 2c fd e0 10 05 ab <4d> 89 65 00 41 8b 44 24 08 85 c0 75 0b f3 90 41 8b 44
24 08 85 c0
2026-10-21T06:28:43.455648+02:00 myserver kernel: RSP: 0018:ffffcf1900b7bce0
EFLAGS: 00010282
2026-10-21T06:28:43.455650+02:00 myserver kernel: RAX: 0000000000000000 RBX:
ffff8b8cd9939c04 RCX: 0000000000000000
2026-10-21T06:28:43.455651+02:00 myserver kernel: RDX: 0000000000000000 RSI:
0000000000000000 RDI: 0000000000000000
2026-10-21T06:28:43.455657+02:00 myserver kernel: RBP: ffffcf1900b7bd08 R08:
0000000000000000 R09: 0000000000000000
2026-10-21T06:28:43.455659+02:00 myserver kernel: R10: 0000000000000000 R11:
0000000000000000 R12: ffff8c46bef346c0
2026-10-21T06:28:43.455661+02:00 myserver kernel: R13: ffffffffac451720 R14:
0000000000180000 R15: 0000000000003ffe
2026-10-21T06:28:43.455662+02:00 myserver kernel: FS: 0000000000000000(0000)
GS:ffff8c4712ae3000(0000) knlGS:0000000000000000
2026-10-21T06:28:43.455663+02:00 myserver kernel: CS: 0010 DS: 0000 ES: 0000
CR0: 0000000080050033
2026-10-21T06:28:43.455665+02:00 myserver kernel: CR2: ffffffffac451720 CR3:
000001053c844002 CR4: 00000000007726f0
2026-10-21T06:28:43.455671+02:00 myserver kernel: PKRU: 55555554
2026-10-21T06:28:43.455673+02:00 myserver kernel: note: kworker/u258:7[3022700]
exited with irqs disabled
2026-10-21T06:28:43.455674+02:00 myserver kernel: note: kworker/u258:7[3022700]
exited with preempt_count 1
2026-10-21T06:28:54.011458+02:00 myserver kernel: Oops: general protection
fault, probably for non-canonical address 0x288b89f1b57e00: 0000 [#2] SMP PTI
2026-10-21T06:28:54.011481+02:00 myserver kernel: CPU: 5 UID: 0 PID: 2805 Comm:
nfsd Tainted: G D 7.0.0-31-generic #31~24.04.1-Ubuntu
PREEMPT(lazy)
2026-10-21T06:28:54.011484+02:00 myserver kernel: Tainted: [D]=DIE
2026-10-21T06:28:54.011487+02:00 myserver kernel: Hardware name: Dell Inc.
PowerEdge R740/0RR8YK, BIOS 2.26.1 01/28/2026
2026-10-21T06:28:54.011489+02:00 myserver kernel: RIP:
0010:flush_workqueue_prep_pwqs+0x5a/0x180
2026-10-21T06:28:54.011492+02:00 myserver kernel: Code: 00 00 c7 43 48 01 00 00
00 48 8b 03 4c 8d b0 78 ff ff ff 48 39 c3 0f 84 11 01 00 00 49 63 c5 c6 45 cf
00 45 31 ff 48 89 45 d0 <49> 8b 06 4c 39 f8 74 1c 4d 85 ff 0f 84 b7 00 00 00 4c
89 ff e8 bd
2026-10-21T06:28:54.011494+02:00 myserver kernel: RSP: 0018:ffffcf18f5967b58
EFLAGS: 00010246
2026-10-21T06:28:54.011496+02:00 myserver kernel: RAX: 0000000000000000 RBX:
ffff8b8cd9939c00 RCX: 0000000000000000
2026-10-21T06:28:54.011498+02:00 myserver kernel: RDX: 0000000000000001 RSI:
0000000000000000 RDI: ffff8b8cd9939c00
2026-10-21T06:28:54.011500+02:00 myserver kernel: RBP: ffffcf18f5967b90 R08:
0000000000000000 R09: 0000000000000000
2026-10-21T06:28:54.011502+02:00 myserver kernel: R10: 0000000000000000 R11:
0000000000000000 R12: 0000000000000001
2026-10-21T06:28:54.011504+02:00 myserver kernel: R13: 0000000000000000 R14:
00288b89f1b57e00 R15: 0000000000000000
2026-10-21T06:28:54.011505+02:00 myserver kernel: FS: 0000000000000000(0000)
GS:ffff8c4712ae3000(0000) knlGS:0000000000000000
2026-10-21T06:28:54.011506+02:00 myserver kernel: CS: 0010 DS: 0000 ES: 0000
CR0: 0000000080050033
2026-10-21T06:28:54.011508+02:00 myserver kernel: CR2: ffffffffac451720 CR3:
000001053c844002 CR4: 00000000007726f0
2026-10-21T06:28:54.011509+02:00 myserver kernel: PKRU: 55555554
2026-10-21T06:28:54.011511+02:00 myserver kernel: Call Trace:
2026-10-21T06:28:54.011512+02:00 myserver kernel: <TASK>
2026-10-21T06:28:54.011514+02:00 myserver kernel: __flush_workqueue+0x1c9/0x460
2026-10-21T06:28:54.011515+02:00 myserver kernel:
nfsd4_probe_callback_sync+0x1e/0x30 [nfsd]
2026-10-21T06:28:54.011517+02:00 myserver kernel:
nfsd4_init_conn.isra.0+0xdd/0x120 [nfsd]
2026-10-21T06:28:54.011518+02:00 myserver kernel:
nfsd4_create_session+0x827/0xb10 [nfsd]
2026-10-21T06:28:54.011520+02:00 myserver kernel:
nfsd4_proc_compound+0x3b2/0x810 [nfsd]
2026-10-21T06:28:54.011521+02:00 myserver kernel: nfsd_dispatch+0xd6/0x230
[nfsd]
2026-10-21T06:28:54.011522+02:00 myserver kernel:
svc_process_common+0x4e4/0x710 [sunrpc]
2026-10-21T06:28:54.011524+02:00 myserver kernel: ?
__pfx_nfsd_dispatch+0x10/0x10 [nfsd]
2026-10-21T06:28:54.011525+02:00 myserver kernel: svc_process+0x15b/0x280
[sunrpc]
2026-10-21T06:28:54.011526+02:00 myserver kernel: svc_handle_xprt+0x486/0x5b0
[sunrpc]
2026-10-21T06:28:54.011528+02:00 myserver kernel: svc_recv+0x122/0x2c0 [sunrpc]
2026-10-21T06:28:54.011529+02:00 myserver kernel: nfsd+0xf8/0x3b0 [nfsd]
2026-10-21T06:28:54.011530+02:00 myserver kernel: ? __pfx_nfsd+0x10/0x10 [nfsd]
2026-10-21T06:28:54.011531+02:00 myserver kernel: kthread+0xfb/0x140
2026-10-21T06:28:54.011533+02:00 myserver kernel: ? __pfx_kthread+0x10/0x10
2026-10-21T06:28:54.011534+02:00 myserver kernel: ret_from_fork+0x1f8/0x2b0
2026-10-21T06:28:54.011536+02:00 myserver kernel: ? __pfx_kthread+0x10/0x10
2026-10-21T06:28:54.011537+02:00 myserver kernel: ret_from_fork_asm+0x1a/0x30
2026-10-21T06:28:54.011538+02:00 myserver kernel: </TASK>
2026-10-21T06:28:54.011540+02:00 myserver kernel: Modules linked in: mptcp_diag
xsk_diag tcp_diag udp_diag raw_diag inet_diag unix_diag af_packet_diag
netlink_diag nfsv3 cpuid st tls ib_core bluetooth rpcsec_gss_krb5 nfsv4 nfs
ceph libceph krb5enc authenc camellia_generic camellia_aesni_avx2
camellia_aesni_avx_x86_64 camellia_x86_64 cmac krb5 netfs dell_rbu binfmt_misc
xt_tcpudp xt_co
mment xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nft_compat
nf_tables intel_rapl_msr intel_rapl_common intel_uncore_frequency
intel_uncore_frequency_common skx_edac skx_edac_common xfs nfit
x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm cmdlinepart
irqbypass dell_smbios platform_profile ipmi_ssif spi_nor ghash_clmulni_intel
aesni_intel i2c_i801 dcdbas m
td mei_me dell_wmi_descriptor rapl mei intel_cstate mgag200 i2c_smbus wmi_bmof
joydev input_leds acpi_power_meter intel_pch_thermal spi_intel_pci ipmi_si
i2c_mux spi_intel lpc_ich acpi_ipmi ipmi_devintf ipmi_msghandler mac_hid
sch_fq_codel lp parport nvme_fabrics nvme_core nvme_keyring nvme_auth hkdf
2026-10-21T06:28:54.011543+02:00 myserver kernel: efi_pstore nfsd auth_rpcgss
nfs_acl lockd grace sunrpc nfnetlink dmi_sysfs ip_tables x_tables autofs4
hid_generic ixgbe igb usbhid libie_fwlog vga16fb ahci i2c_algo_bit xfrm_algo
hid megaraid_sas dca libahci vgastate wmi mdio
2026-10-21T06:28:54.011545+02:00 myserver kernel: ---[ end trace
0000000000000000 ]---
To my knowledge this is the first time this particular error occurs, but it
looks like it might happen again.
Let me know if more details are needed.
** Affects: linux (Ubuntu)
Importance: Undecided
Status: New
** Tags: nfs
** Attachment added: "lspci-vnvn.log"
https://bugs.launchpad.net/bugs/2171589/+attachment/6011231/+files/lspci-vnvn.log
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2171589
Title:
array-index-out-of-bounds somewhere in nfsd
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2171589/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
