== Summary == Restarting a KFD application reliably wedges the next instance during initial GPU memory allocations whenever the system is under memory pressure. The root cause is an invalidation-storm livelock in the SVM eviction/restore path:
kswapd / kcompactd reclaim or migrate a page inside the process's
SVM-registered address space
-> mmu interval notifier -> svm_range_cpu_invalidate_pagetables
-> svm_range_evict: prange->invalid++, svms->evicted_ranges++,
quiesce_mm(), schedule svm_range_restore_work
-> svm_range_restore_work takes mmap write lock, walks the range
list, re-faults pages via hmm_vma_fault and remaps them to the
GPU
-> those pages are reclaim candidates again immediately; the next
invalidation lands mid-pass -> per-range cmpxchg on
prange->invalid fails -> goto out_reschedule aborts the whole
pass
-> evicted_ranges never drains to 0 -> kgd2kfd_resume_mm() never
runs -> KFD queues stay quiesced forever
Because svm_range_evict() increments both counters on EVERY invalidate
callback (even for ranges already marked invalid) and restore_work
aborts the ENTIRE pass on the first re-dirtied range, convergence
requires a perfectly quiet window that does not exist under sustained
reclaim. Observed on the wedged system: ~900 invalidations/sec
(svm_range_cpu_invalidate_pagetables, counted via ftrace), per-range
"inv" counters in the tens of thousands (max seen: 32769), and the
userspace process spinning in the HSA runtime waiting for a queue
signal that never arrives — or blocked in D state in
svm_range_list_lock_and_flush_work.
Two secondary bugs compound this:
(a) svm_range_restore_work holds mmap_write_lock for the whole pass;
concurrent mm walkers (incl. reclaim-side access_remote_vm) stall
behind it — Emily Deng's Feb-2025 mmap_write_downgrade fix
addresses the lock-ordering part and is confirmed missing in the
tested kernels.
(b) A !vma range returns -EFAULT and reschedules forever instead of
unmapping the stale GPU mapping — Philip Yang's Oct-2025 fix;
also missing in the tested kernels. On teardown/restart (dead
VMAs pending restore) this alone can pin the worker.
== Environment ==
GPU: AMD Radeon Instinct MI50 32GB, gfx906, 0000:05:00.0
(Vega 20, PCIe, attached via Thunderbolt 3 eGPU enclosure)
Host: AMD Strix Halo (Ryzen AI Max+ 395), 28GB RAM, Proxmox VE 9
Kernels: 6.17.13-13-pve, 7.0.12-1-pve, 7.2.6-k726 — all affected
Cmdline: amdgpu.dc=0 modprobe.blacklist=amdxdna apparmor=0
Userspace: llama.cpp llama-server (HIP/ROCm), hsakmt; ~19GB model
mmap'd into the SVM-registered address space
Taint: 512 (unsigned module; none in the KFD path)
== Trigger: memory pressure ==
The wedge is deterministically controlled by reclaim pressure on the
KFD process's SVM-registered pages:
* Docker swarm (~95 services) up, ~8-12GB available:
restart -> invalidation storm, never healthy (health endpoint
stays 503; process spins in userspace or D-state in the alloc
ioctl; storm continues even after SIGSTOPping the process —
17,488 invalidates in 20s with userspace frozen; it is
kernel-driven).
* Same machine, swarm scaled to 0, ~24GB available:
restart -> health=200, clean load, real inference OK.
* Same machine, swarm up, llama-server started with --mlock
(model pages unevictable):
restart under pressure -> health=200, ftrace count of
svm_range_cpu_invalidate_pagetables = 0. The storm cannot start
when reclaim cannot touch the pages.
* ftrace stack traces of svm_range_cpu_invalidate_pagetables during
the storm: kcompactd -> compact_zone -> migrate_pages ->
try_to_migrate -> __mmu_notifier_invalidate_range_start, and
kswapd -> shrink_folio_list -> try_to_unmap -> same path
(via __rmap_walk_file — the mmap'd GGUF file pages).
== Reproduction ==
1. Boot with the MI50 bound to amdgpu/KFD.
2. Start a KFD app that mmap's ~19GB of file-backed model data into
its SVM-registered address space (llama-server -ngl 99).
3. Create sustained memory pressure (busy service stack, or
anonymous churn so kswapd keeps scanning).
4. systemctl restart. The new instance wedges during VRAM/SVM
setup; dmesg (with dyndbg) shows svm_range_restore_work
rescheduling indefinitely.
== Observed stacks (kernel 7.2.6) ==
Userspace process (llama-server, D state):
__flush_work+0x2c4/0x3c0
flush_work+0x1c/0x30
svm_range_list_lock_and_flush_work+0x2f/0x100 [amdgpu]
kfd_ioctl_alloc_memory_of_gpu+0x7c/0x3a0 [amdgpu]
Concurrent kworker (D state, same boot):
__flush_work+0x2c4/0x3c0
flush_work+0x1c/0x30
svm_range_list_lock_and_flush_work+0x2f/0x100 [amdgpu]
svm_range_restore_work+0x75/0x3a0 [amdgpu]
process_one_work+0x1a9/0x390
With dyndbg enabled the log fills with:
kfd kfd: [pid N] restore svm ranges
... restoring svms ... inv 74 / inv 52 / ... / inv 32769
kfd kfd: [pid N] reschedule to restore svm ranges
(~900 lines/sec; journald flooding itself degrades the machine.)
== Tested fix ==
A patch against 7.2.6 combining the two missing upstream fixes plus a
restore-pass convergence fix (attached:
0001-amdkfd-svm-restore-convergence.patch):
1. Backport Emily Deng's mmap_write_downgrade() in
svm_range_restore_work (Feb 2025).
2. Backport Philip Yang's !vma -> svm_range_unmap_from_gpus handling
(Oct 2025), adapted to this tree's amdgpu_hmm_range_* API; this
also fixes a latent units bug: the old !VM_READ path compared
byte-address start/end against page-index prange->start/last, so
the unmap never fired.
3. Convergence fix (new): in svm_range_restore_work, do not abort the
pass on the first range that fails or is re-dirtied mid-restore.
Restore each dirty range independently; if a range is
re-invalidated during its own restore or fails to map, mark
need_retry and continue with the remaining ranges. Reschedule one
more pass only if anything is still dirty, and still take the
evicted_ranges snapshot consistency check before resuming queues.
Each pass now makes monotonic per-range progress instead of
requiring an all-clean sweep.
== Results ==
Patched amdgpu.ko on 7.2.6-k726, swarm fully up (original failing
conditions, swap enabled, default vm tunables), llama WITHOUT mlock:
restart round 1: invalidations = 668 -> health=200 @ 176s
restart round 2: invalidations = 898 -> health=200 @ 102s
restart round 3: invalidations = 2146 -> health=200 @ 125s
vs unpatched: identical conditions produced a non-converging storm
(inv counts into the tens of thousands, max 32769 observed on one
range) and the process never became healthy.
Inference verified working after each healthy restart.
Sustained-pressure follow-up (patch 1 installed, cgroup
MemoryHigh=6G on the service during model reload — continuous
direct reclaim of the process's own pages):
invalidations > 156,000 and still climbing when killed
health=503 persisted >15 min, RSS pinned ~13GB
storm CONTINUED after releasing the cgroup squeeze —
restore_work's own refaults kept feeding it
Conclusion: patch 1 fixes convergence under bursty invalidation
but not the reclaim->invalidate->restore->refault feedback loop
under sustained reclaim. Restored pages remain reclaim candidates,
so a clean pass can never complete while pressure continues.
== Second fix: make registered ranges unevictable (patch 2) ==
On non-xnack hardware the GPU cannot fault CPU pages back in, so
every resident page inside a registered SVM interval must stay
resident while the interval exists; reclaiming one is always futile
— restore_work just faults it straight back in. The bug is not that
restore retries; it is that the VM may take pages the GPU mapping
fundamentally requires.
Two failed mechanisms before the working one:
- Pinning folios at dma_map time (refcount + PG_mlocked mark):
insufficient — a folio only becomes unreclaimable once it is
moved to the unevictable LRU list, and the primitives that move
an already-LRU folio (mlock_folio, __mlock_folio, lruvec helpers)
are not exported to modules. Flagged-but-evictable folios were
still scanned, still unmapped, still stormed (253k+ observed).
- VM_LOCKED alone on covering VMAs: try_to_unmap_one() does bail
on VM_LOCKED before the PTE clear and lazily mlocks the folio —
but mmu_notifier_invalidate_range_start() fires at the TOP of
try_to_unmap, before the rmap walk discovers the flag. Every
reclaim attempt on a not-yet-locked folio still costs one
invalidation. Bounded per page, but under sustained pressure the
load only crawled (RSS growth ~1GB/5min, inv into six figures).
Working approach (attached: 0002-amdkfd-nonxnack-lock.patch):
In svm_range_add_notifier_locked() — the single funnel where every
registered interval is inserted — when !p->xnack_enabled:
1. Set VM_LOCKED on all VMAs covering the registered range
(vma_start_write + vm_flags_set; callers already hold
mmap_write_lock for mmu_interval_notifier_insert_locked).
2. Prefault the range with batched get_user_pages_remote(FOLL_GET
then put_page): pages faulted into a VM_LOCKED vma are placed
on the unevictable LRU by the normal fault path
(lru_cache_add_inactive_or_unevictable), so reclaim never
selects them and the notifier never fires.
This is exactly what userspace mlock() does (VM_LOCKED + prefault),
applied at registration to the ranges the GPU may touch — matching
the --mlock measurement (0 invalidations, immune restart).
== Results with patch 2 ==
Same environment (full swarm, swap enabled, no --mlock):
cold boot load under swarm pressure: inv = 4 -> health=200
restart under live swarm churn, round 1: +694 -> health=200 @102s
restart under live swarm churn, round 2: +83 -> health=200 @102s
restart after repeated cycles: ~3.3k -> health=200 @195s
Residual invalidations are one-time lazy-lock transitions on pages
resident before their VMA was locked plus non-reclaim notifier
events — bounded, not self-sustaining. The storm no longer feeds
on itself: releasing memory pressure once stopped the counter
dead instead of the storm continuing.
Note: with a hard cgroup cap below the model footprint
(MemoryHigh=6G vs ~19G needed), locked+prefaulted pages are
correctly unevictable but the workload cannot fit — expected, and
identical to userspace --mlock under the same cap.
Deployment gotcha worth noting: on this host amdgpu.ko ships in
the initramfs and loads at ~3s, so /lib/modules replacement alone
did nothing until update-initramfs -u was run.
== What was ruled out ==
- HSA_XNACK=0 does not avoid it (gfx906 has no xnack anyway).
- Not alloc-size specific (8GB context reproduces too).
- Not the custom kernel: identical on stock 6.17.13-13-pve,
7.0.12-1-pve.
- Not the model file specifically: --no-mmap still storms (any
file-backed pages inside the SVM-registered mm are enough).
- Not a swap-only issue: storm persists with swap disabled while
other reclaim pressure exists; kcompactd page migration alone
suffices.
- kfdtest memory/queue tests pass while a healthy client holds the
device.
== Logs ==
SysRq-w dumps, blocked stacks, dyndbg excerpts, ftrace stack samples
and counters available on request.
== Notes for maintainers ==
- MI50 is behind Thunderbolt (hotplug PCI topology). If restore
scheduling interacts with hotplug eviction paths this box may
expose a wider timing window than fixed-PCIe systems. Happy to
test patches — reproduction takes ~5 minutes.
- The convergence change (3) is a proposal; it preserves the
"queues resume only when clean" guarantee via the existing
evicted_ranges consistency check, but review of the
retry-vs-resume ordering is welcome.
0001-amdkfd-svm-restore-convergence.patch
Description: Binary data
0002-amdkfd-nonxnack-lock.patch
Description: Binary data
