Hi Christian,
while testing this series (applied on top of 7.2.6) we hit a regression
in amdgpu_vm_update_leaves() that deterministically produces MMHUB
page faults under KFD SVM load on gfx906 (MI50 32GB, Vega 20, PCIe via
Thunderbolt 3 eGPU).
> + pt = cursor.parent->bo;
> + shift = amdgpu_vm_pt_level_shift(adev, cursor.level - 1);
> + mask = amdgpu_vm_pt_entries_mask(adev, cursor.level - 1);
> +
> + /* Looks good so far, calculate parameters for the update */
> + pe_start = ((cursor.pfn >> shift) & mask) * 8;
> +
> + entry_end = ((uint64_t)mask + 1) << shift;
> + entry_end += cursor.pfn & ~(entry_end - 1);
> + entry_end = min(entry_end, end);
> +
> + nptes = (entry_end - cursor.pfn) >> shift;
> + /*
> + * This can happen when we set higher level PDEs to unmap and/or
> + * silent to stop fault floods.
> + */
> + nptes = max(nptes, 1u);
> +
> + trace_amdgpu_vm_update_ptes(params, cursor.pfn, entry_end,
> + min(nptes, 32u), dst, 0, flags,
> + tgid,
> + vm->immediate.fence_context);
> + amdgpu_vm_pte_update_flags(params, to_amdgpu_bo_vm(pt),
> + cursor.level - 1, pe_start, dst,
> + nptes, 0, flags);
When amdgpu_vm_pt_descendant() cannot descend (a gap — the position has
no leaf table in the tree), the write targets the parent's slot at
level-1, i.e. a PDE slot. nptes as computed above covers *every*
remaining slot up to entry_end (the end of the parent-table span,
clamped to the op end) — not just the slot the cursor is at.
The old code had an explicit clamp for this exact case, in
amdgpu_vm_ptes_update() under params->unlocked:
if (cursor.level < AMDGPU_VM_PTB && params->unlocked)
/* only clear one entry. Next entry search again for PDE or
* PTE leave.
*/
entry_end = 1ULL << shift;
That clamp was lost in the rewrite. Without it, an unlocked unmap
(PATCH 08/13 wires amdgpu_vm_unmap_range() to update_leaves()) that
hits a gap *before* still-linked sibling subtrees will clear their
parent slots too:
- the gap's unmap write covers N parent slots (we observed runs like
pe 0x58 n15, pe 0x60 n14, pe 0x68 n13 — each gap position
re-clearing the whole remaining tail of the parent page),
- sibling PDE slots inside that span still hold live child pointers,
- the write overwrites them with the PDE_PTE-only marker
(0x40000000000000 in RAM — bit 54 set, no address),
- the in-memory PT tree still considers those subtrees linked, so
their needs_update is never set and a later amdgpu_vm_update_pdes()
pass does not re-link them,
- the next GPU page walk through that slot reads a non-pointer and
faults: "memory access fault ... reason: PTE not valid" (MMHUB
no-retry).
Captured write ordering (instrumented amdgpu_vm_pte_update_flags /
amdgpu_vm_pde_update, job ids tagged):
MAP_RANGE PTEs pe 0x368..0x800 (big map)
UPDATE_PDES pde 0x0..0x1a8 links, incl. [0x80] -> child 0x5d4dd0000
MAP_RANGE PTEs pe 0x58..0x1e0 (re-map)
UNMAP_RANGE clear pe 0x58 n15 -> covers 0x58..0xc8, incl. 0x80
UPDATE_PDES relinks 0x58..0x78, 0xf0..0x1f0 -- but not 0x80..0xe8
FAULT slot 0x80 reads 0x40000000000000,
tree still expects 0x5d4dd0001
The unmap's clear overshoots slot 0x80; the relink pass skips it
because the subtree was never dropped from the tree. This is fully
deterministic — our reproducer (vLLM/ROCm loading a 10-shard model,
i.e. heavy SVM map/unmap/re-add churn) faulted at the same stage on
every run.
Locally we fixed it by restoring the single-entry semantics for
non-leaf slot writes:
/* only clear one entry, next entry search again for PDE/PTE leaf */
if (cursor.level < AMDGPU_VM_PTB)
nptes = 1;
(nptes = 1 rather than re-clamping entry_end, since entry_end is only
used for the tracepoint afterward; either form works.)
With the clamp restored the full workload completes cleanly — all
checkpoint shards load, engine serves inference, zero page faults
across repeated runs and reboots.
Two questions:
1. Is this already fixed in a later revision of the series? I tested
the patches as posted on amd-gfx (2026-05-29), not a git branch.
2. If not — since 08/13 makes the unlocked unmap path use
update_leaves(), everyone driving SVM eviction/restore traffic will
hit this. Would you like a formal patch, or do you prefer to fold
the clamp into a v2?
Full debugging trail (STALE PDE dumps, write histories, vanilla diff)
is available if useful.
Regards,
Olivier
== Environment ==
GPU: AMD Radeon Instinct MI50 32GB, gfx906, 0000:05:00.0
(Vega 20, PCIe via Thunderbolt 3 eGPU enclosure)
Host: AMD Strix Halo (Ryzen AI Max+ 395), 28GB RAM, Proxmox VE 9
Kernel: 7.2.6 + amd-gfx VM critical-section series (vmcrit 1-14)
+ TLB invalidation V3 series
Workload: vLLM 0.28.0 (rocm, gfx906 build) loading GLM-4-9B 1M
via KFD SVM; deterministic fault during weight staging
