On 2026-10-13 05:20, Jesse Zhang wrote:
When an SVM range is split, svm_range_split_pages() gives the new range
the number of VRAM pages counted in its part of the dma_addr arrays and
subtracts it from the old range. Pages migrated to VRAM by a prefetch
have no dma_addr entries until the range is mapped on a GPU, e.g. with
no ACCESS attribute or before the first fault, so the new range gets 0
and the old one keeps the count for both.
Migrating the parts back to system memory then gets the accounting
wrong: the old part keeps a VRAM page count after all its pages are
gone, so its svm_bo is never freed, and the new part underflows. On
navi31, as an unprivileged process: prefetch 4 MiB to the GPU, prefetch
the upper 2 MiB and then the lower 2 MiB back to system memory:
Recorded vram pages(0x0) should not be less than migration pages(0x200).
WARNING: ../amdkfd/kfd_migrate.c:922 at svm_migrate_vram_to_ram+0xbdc/0xcd0
[amdgpu]
and/sys/class/kfd/kfd/proc/<pid>/vram_<gpu_id> stays at 4194304 until
the process exits.
Without dma_addr arrays, the split is still exact when the old range is
completely in VRAM, which is what a prefetch of the whole range leaves.
Give the new range all its pages in that case. The warning is gone and
the VRAM usage drops back to 0.
Fixes: 006ad514a50c ("drm/amdkfd: Use partial hmm page walk during buffer validation
in SVM")
Signed-off-by: Jesse Zhang<[email protected]>
---
drivers/gpu/drm/amd/amdkfd/kfd_svm.c | 12 +++++++++++-
1 file changed, 11 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
index 1f5d73f0ebce..7680fd59ecff 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
@@ -1020,9 +1020,12 @@ svm_range_split_pages(struct svm_range *new, struct
svm_range *old,
uint64_t start, uint64_t last)
{
uint64_t npages = last - start + 1;
+ bool counted = false;
int i, r;
for (i = 0; i < MAX_GPU_INSTANCE; i++) {
+ if (old->dma_addr[i])
+ counted = true;
r = svm_range_split_array(&new->dma_addr[i], &old->dma_addr[i],
sizeof(*old->dma_addr[i]), old->start,
npages, new->start, new->npages,
@@ -1030,8 +1033,15 @@ svm_range_split_pages(struct svm_range *new, struct
svm_range *old,
if (r)
return r;
}
- if (old->actual_loc)
+ if (old->actual_loc) {
+ /*
+ * Pages migrated by prefetch have no dma_addr until they are
+ * mapped on a GPU. A fully resident range still splits exactly.
+ */
+ if (!counted && old->vram_pages == npages + new->npages)
+ new->vram_pages = new->npages;
With XNACK off, svm_range_set_attr() leaves the GPU mapping update to
the delayed restore worker rather than doing it inline, so the dma_addr
arrays are not populated yet. The bug is a timing window: the prefetch
back to system memory happens before the scheduled worker gets to run.
The fix, though, assumes prange->vram_pages == npages. That does not
hold for a partial migration — svm_migrate_ram_to_vram() only adds the
pages it actually migrated, so the range can be partly resident in VRAM
with no dma_addr array, and the accounting is still wrong in that case.
Fixing it completely means redesigning the per-range VRAM page
accounting so residency is tracked per page independently of dma_addr.
That is a much bigger change, and this simple solution is good enough to
fix the bug in front of us, so let's go with it. Two suggestions:
Rename bool counted to has_dma_addr
if (!has_dma_addr && old->vram_pages == old->npages)
new->vram_pages = new->npages;
Please also mention the partial-migration limitation in the commit message.
Regards,
Philip
old->vram_pages -= new->vram_pages;
+ }
return 0;
}
-- 2.50.1
