On Wed, Oct 21, 2026 at 09:28:08PM -0500, [email protected] wrote: > Hey Matthew! > > Sounds good to me, the fallback of trying to bind the vma one at a time > sounds good to me. And it is actually a bit broader than my patch since it > covers the -ENOBUFS case as well. It should also be easy to get through > drm-xe-fixes and then backported the normal way. > > I can write it as a v2 & test it on my Panther Lake device, I have a small > repro that does about 20,000 mappings and fails after suspend (every time > without the fix). If you'd prefer to write it, I can test your patch as well. >
Since you have a repro, I'd say go ahead and code it up. This is on my radar, so I can review and pull it quickly. It should be a relatively simple function, and something like Claude can help you code it with a concise query if needed. One follow-up we'd probably want to add is error injection and an IGT test to ensure long-term functionality, but we (Intel) can take care of that part as an AR. Matt > Thanks, > Karim > > On Wed, 21 Oct 2026 11:57:48 -0700, Matthew Brost wrote: > > On Tue, Oct 20, 2026 at 07:08:27PM -0500, [email protected] wrote: > >> From: Karim Sabar <[email protected]> > >> > > > > Thanks for the patch. I do see the problem in the old code. > > Unfortunately, we will not be able to pull this into drm-xe-next or > > drm-xe-fixes, as 7.3 is already out and 0e33330f1238 is included in our > > 7.4 PR. As a result, this would need to go directly to stable. > > > > There is also a risk that predates the series containing 0e33330f1238. > > If the bind has to run on the GPU, you can still get -ENOBUFS later in > > the bind pipeline if the bind is too large to fit into the > > migration-layer BB. If that occurs, the proper fix may actually be in > > xe_vm_validate_rebind(). If xe_vm_rebind() fails with -ENOBUFS or > > -ENOMEM, we could call a new function, such as xe_vm_rebind_iterative(), > > which would run each VMA requiring a rebind in its own job. This is > > effectively what Mesa does in userspace when a bind operation returns > > -ENOBUFS. > > > > We could probably still pull that approach into drm-xe-fixes + have > > naturally backport to stable, and it may ultimately be the better > > solution. > > > > Let me know what you think. I can give this an RB, but I'd like to hold > > off until we've settled on the solution we want to pursue here. > > > > Matt > > > >> Big games die after every s2idle resume on flat-CCS iGPUs. The exec > >> ioctl keeps returning -ENOMEM even with 11-16 GB of RAM free. > >> > >> On suspend xe_bo_evict_all_user() evicts every user BO in XE_PL_TT on > >> these parts. On resume the first exec rebinds every VMA in one go and > >> xe_vma_ops_alloc() kmallocs one 384-byte op per VMA as a single array. > >> Final Fantasy VII Rebirth has about 33,500 VMAs which means a 12.9 MB > >> array. It's over the 4 MB KMALLOC_MAX_SIZE and can never succeed. > >> __GFP_NOWARN keeps it out of dmesg. > >> > >> Data from a Panther Lake handheld (SteamOS kernel 7.2.7), kretprobes > >> after resume: > >> xe_vma_ops_alloc -ENOMEM on 63 of 63 attempts > >> xe_vm_rebind -ENOMEM on 63 of 63 attempts > >> BO validate, TTM populate, PT prepare never failed > >> > >> The array is only used by the CPU and doesn't need to be contiguous. > >> This switches it to kvmalloc and kvfree. > >> > >> With the same change applied as a module on that kernel, the game > >> survives every sleep I tried with zero ENOMEM. The biggest array was > >> 30,753 ops (11.8 MB). > >> > >> drm-xe-next already does this through commit 0e33330f1238 ("drm/xe: Add > >> struct xe_pt_job_ops"), but that's part of a big series that won't go > >> to stable. > >> > >> I tracked this down and tested it on my own device. I used an AI > >> assistant to help with the debugging, i.e the kprobe tracing and > >> reading through the xe code. I ran every test, picked the fix and > >> reviewed the change myself. > >> > >> What I couldn't do: this is build-tested on drm-xe-fixes but only > >> run-tested on the SteamOS 7.2.7 kernel through an equivalent module, > >> not on a mainline kernel. > >> > >> Fixes: 2e524668c440 ("drm/xe: Add xe_vm_pgtable_update_op to xe_vma_ops") > >> Cc: [email protected] # v6.11+ > >> Assisted-by: Claude:claude-opus-5-5 > >> Signed-off-by: Karim Sabar <[email protected]> > >> --- > >> Downstream SteamOS report with the same data: > >> https://github.com/ValveSoftware/SteamOS/issues/2962 > >> --- > >> drivers/gpu/drm/xe/xe_vm.c | 8 ++++---- > >> 1 file changed, 4 insertions(+), 4 deletions(-) > >> > >> diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c > >> index 99e42bd29..5caef443b 100644 > >> --- a/drivers/gpu/drm/xe/xe_vm.c > >> +++ b/drivers/gpu/drm/xe/xe_vm.c > >> @@ -703,9 +703,9 @@ static int xe_vma_ops_alloc(struct xe_vma_ops *vops, > >> bool array_of_binds) > >> continue; > >> > >> vops->pt_update_ops[i].ops = > >> - kmalloc_objs(*vops->pt_update_ops[i].ops, > >> - vops->pt_update_ops[i].num_ops, > >> - GFP_KERNEL | __GFP_RETRY_MAYFAIL | > >> __GFP_NOWARN); > >> + kvmalloc_objs(*vops->pt_update_ops[i].ops, > >> + vops->pt_update_ops[i].num_ops, > >> + GFP_KERNEL | __GFP_RETRY_MAYFAIL | > >> __GFP_NOWARN); > >> if (!vops->pt_update_ops[i].ops) > >> return array_of_binds ? -ENOBUFS : -ENOMEM; > >> } > >> @@ -742,7 +742,7 @@ static void xe_vma_ops_fini(struct xe_vma_ops *vops) > >> xe_vma_svm_prefetch_ops_fini(vops); > >> > >> for (i = 0; i < XE_MAX_TILES_PER_DEVICE; ++i) > >> - kfree(vops->pt_update_ops[i].ops); > >> + kvfree(vops->pt_update_ops[i].ops); > >> } > >> > >> static void xe_vma_ops_incr_pt_update_ops(struct xe_vma_ops *vops, u8 > >> tile_mask, int inc_val) > >> > >> --- > >> base-commit: d2ce8fcd778ee6d92748c92d23a56d4416c29d86 > >> change-id: 20261020-xe-kvmalloc-vma-ops-2637ebf81cfd > >> > >> Best regards, > >> -- > >> Karim Sabar <[email protected]> > >>
