On Tue, Jun 09, 2026 at 03:56:54AM -0700, Breno Leitao wrote:
> A multi-bit ECC error on a kernel-owned page that the memory failure
> handler cannot recover is currently swallowed: PG_hwpoison is set, the
> event is logged, and the kernel keeps running. The corrupted memory
> remains accessible to the kernel and either drives silent data
> corruption or surfaces seconds-to-minutes later as an apparently
> unrelated crash. In a large fleet that delayed, unattributable crash
> turns into significant engineering effort to root-cause; in a kdump
> configuration, by the time the crash happens the original error
> context (faulting PFN, MCE/GHES record, page state) is long gone.
>
> This series adds an opt-in sysctl,
> vm.panic_on_unrecoverable_memory_failure, that converts an
> unrecoverable kernel-page hwpoison event into an immediate panic with
> a clean dmesg/vmcore that still contains the original failure
> context. The default is disabled so existing workloads see no
> change.
>
> There is a selftest that test different cases, and I tested it using
> the following variants:
>
>
> ┌─────────┬──────────┬───────────────────────────────────────────────────────────┐
> │ Variant │ PFN │ Result
> │
>
> ├─────────┼──────────┼───────────────────────────────────────────────────────────┤
> │ rodata │ 0x2600 │ Panic with "Memory failure: 0x2600: unrecoverable
> page" │
>
> ├─────────┼──────────┼───────────────────────────────────────────────────────────┤
> │ slab │ 0x100032 │ Panic with "Memory failure: 0x100032: unrecoverable
> page" │
>
> ├─────────┼──────────┼───────────────────────────────────────────────────────────┤
> │ pgtable │ 0x100000 │ Panic with "Memory failure: 0x100000: unrecoverable
> page" │
>
> └─────────┴──────────┴───────────────────────────────────────────────────────────┘
>
> Each one shows the same call trace, exactly the path the series builds:
>
> hard_offline_page_store
> → memory_failure
> → action_result
> → panic("Memory failure: %#lx: unrecoverable page")
Debugging another issue earlier today, just found a kernel crash that is
hitting a ignored page later in the day, and randomly misbehaving/crashing.
Memory failure: 0x140ae: unhandlable page.
Memory failure: 0x140ae: recovery action for get hwpoison page: Ignored
<-- Ignored
loop0: detected capacity change from 0 to 15241056
EDAC MC0: 1 UE multi-bit ECC on LP5x_0 LP5x_0 (node:0 card:0 module:0 rank:0
bank:2 device:28 row:42700 column:96
{3}[Hardware Error]: Hardware error from APEI Generic Hardware Error Source:
308
{3}[Hardware Error]: event severity: recoverable
{3}[Hardware Error]: imprecise tstamp: 2026-06-16 02:50:03
{3}[Hardware Error]: Error 0, type: recoverable
{3}[Hardware Error]: section_type: memory error
{3}[Hardware Error]: physical_address: 0x0000000aeccde180
{3}[Hardware Error]: physical_address_mask: 0xfffffffffffff000
{3}[Hardware Error]: node:0 card:0 module:0 rank:0 bank:2 device:28
row:42700 column:960 requestor_id:0x0000000
{3}[Hardware Error]: error_type: 3, multi-bit ECC
{3}[Hardware Error]: DIMM location: LP5x_0 LP5x_0
Memory failure: 0xaeccd: recovery action for dirty LRU page: Recovered
Internal error: synchronous external abort: 0000000096000410 [#1] SMP
Modules linked in: ghes_edac(E) squashfs(E) act_gact(E) sch_fq(E) tcp_diag(E)
inet_diag(E) cls_bpf(E) evdev(E) sm
CPU: 51 UID: 0 PID: 1 Comm: systemd Kdump: loaded Tainted: G M OE K
6.16.1-0_fbk2_0_gf40efc324cc8 #1
Tainted: [M]=MACHINE_CHECK, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE, [K]=LIVEPATCH
pstate: 834010c9 (Nzcv daIF +PAN -UAO +TCO +DIT +SSBS BTYPE=--)
pc : clear_inode+0x34/0x108
lr : proc_evict_inode.llvm.1771226604092943895+0x28/0x68
sp : ffff800083f6f8d0
x29: ffff800083f6f8e0 x28: 0000000000000011 x27: ffff0000c1378788
x26: ffffffffffffffff x25: ffff800082747de0 x24: ffff0000c0ae9898
x23: ffff8000819155f8 x22: ffff0000c0ae9888 x21: ffff0000c0ae9808
x20: ffff0000c0ae9818 x19: ffff0000c0ae9788 x18: 000000000000001c
x17: 0000000000000018 x16: 0000000000000040 x15: 0000000000000000
x14: 0000000000000001 x13: 0000000000000000 x12: 0000000000002710
x11: ffff0000c0ae9898 x10: ffff0000c1299b58 x9 : 0000000000000001
x8 : ffff0000c0ae9900 x7 : ffff8000828db000 x6 : 0000000000005040
x5 : ffffffffffffffff x4 : ffffffdfc05c8aa0 x3 : ffff000126470000
x2 : ffffffffffffffff x1 : 0000000000000000 x0 : ffff0000c0ae9788
Call trace:
clear_inode+0x34/0x108 (P)
proc_evict_inode.llvm.1771226604092943895+0x28/0x68
evict+0xec/0x328
iput+0xa8/0x310
dentry_unlink_inode+0xa4/0x188
__dentry_kill+0x74/0x358
shrink_dentry_list+0xc8/0x198
....
