AMD General
Hi Christian,
So adding dma_resv_wait_timeout(vm->root.bo->tbo.base.resv,
DMA_RESV_USAGE_READ, false,MAX_SCHEDULE_TIMEOUT); before the amdgpu_vm_wait_idle()
fixes that gfx reset. So it seems like amdgpu_vm_wait_idle() alone is not enough.
Attached the patch again. Please let me know if it's good.
Thanks,
Tharshan
-----Original Message-----
From: Thothiralingam, Logatharshan
Sent: Tuesday, November 17, 2026 12:52 PM
To: Koenig, Christian <[email protected]>; Limonciello, Mario
<[email protected]>
Cc: [email protected]
Subject: RE: [PATCH] drm/amdgpu: drain queued scheduler jobs before
suspend
Hi Chistian,
Just having the amdgpu_vm_wait_idle() doesn't solve the issue. I
tested it and I do see the reset. I have attached a patch with the
dma_resv_wait_timeout(vm->root.bo->tbo.base.resv, DMA_RESV_USAGE_READ,
false,MAX_SCHEDULE_TIMEOUT); I am building it now to test. Please let
me know your thoughts
Thanks,
Tharshan
-----Original Message-----
From: Thothiralingam, Logatharshan
Sent: Tuesday, November 17, 2026 8:39 AM
To: Koenig, Christian <[email protected]>; Limonciello, Mario
<[email protected]>
Cc: [email protected]
Subject: RE: [PATCH] drm/amdgpu: drain queued scheduler jobs before
suspend
Hi Chistian,
I looked a the amdgpu_vm_wait_idle() function and noticed that starting commit
1f02f20
(https://github.com/eseiker/kernel-asahi/commit/1f02f2044bda1db1fd995bc35961ab075fa7b5a2)
the wait on the VM's root page table fence list is removed. So correct me if
I am wrong, this alone won't wait on the gfx jobs right?
So should I add this before the idle call?
dma_resv_wait_timeout(vm->root.bo->tbo.base.resv, DMA_RESV_USAGE_READ,
false,MAX_SCHEDULE_TIMEOUT);
Thanks,
Tharshan
-----Original Message-----
From: Koenig, Christian <[email protected]>
Sent: Tuesday, November 17, 2026 4:37 AM
To: Limonciello, Mario <[email protected]>
Cc: [email protected]; Thothiralingam, Logatharshan
<[email protected]>
Subject: Re: [PATCH] drm/amdgpu: drain queued scheduler jobs before
suspend
On 11/16/26 21:40, Mario Limonciello wrote:
At suspend, amdgpu_fence_driver_hw_fini() only waits for the last
fence emitted to each ring. Jobs still queued in a scheduler entity
have not reached run_job() yet and so have no hardware fence.
Userspace is frozen at this point, but such jobs can become runnable
later, for example once a dependency signals, and the scheduler then
pushes them into the ring after the hardware wait has finished and
before the IP blocks are torn down. On resume the ring write pointer
is reset, those jobs never signal and the GFX ring times out and hangs.
This was observed as a rare (about 2 in 400 cycles) GFX hang on S3
resume on Picasso.
Flush the context and VM scheduler entities of every open file before
evicting resources, mirroring what amdgpu_flush() does on file close.
Like for any other dma_fence there is no timeout, the wait is not
bounded.
Reported-by: Logatharshan Thothiralingam
<[email protected]>
Closes:
https://lore.kernel.org/amd-gfx/SA1SPRMB0073292C113C6B98D9282092F87F2
@ SA1SPRMB0073.namprd12.prod.outlook.com/
Signed-off-by: Mario Limonciello <[email protected]>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 26
++++++++++++++++++++++++++
1 file changed, 26 insertions(+)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 19a566187dfb5..d18348116d3f1 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -4538,6 +4538,30 @@ void amdgpu_device_complete(struct drm_device *dev)
}
}
+/**
+ * amdgpu_device_flush_file_entities - drain scheduler entities of
+open files
+ * @adev: amdgpu device object
+ *
+ * Wait for the scheduler entities owned by all open files (the
+context
+ * entities used for command submission and the VM entities used for
+page
+ * table updates) to run out of queued jobs, so that nothing can
+slip into
+ * the rings after the hardware fences were waited on at suspend time.
+ */
+static void amdgpu_device_flush_file_entities(struct amdgpu_device
+*adev) {
+ struct drm_device *ddev = adev_to_drm(adev);
+ struct drm_file *file;
+
+ guard(mutex)(&ddev->filelist_mutex);
+ list_for_each_entry(file, &ddev->filelist, lhead) {
+ struct amdgpu_fpriv *fpriv = file->driver_priv;
+
+ amdgpu_ctx_mgr_entity_flush(&fpriv->ctx_mgr,
+ MAX_SCHEDULE_TIMEOUT);
That is actually superfluous and even a little bit dangerous.
amdgpu_ctx_mgr_entity_flush() is only supposed to be called from the process
which has the file descriptor open and is closing it.
We use that to detect if that process using the file descriptor was killed and
we should discard queued up work instead of waiting for it to finish executing.
+ amdgpu_vm_wait_idle(&fpriv->vm, MAX_SCHEDULE_TIMEOUT);
That here should be enough to wait for all work to complete.
Regards,
Christian.
+ }
+}
+
/**
* amdgpu_device_suspend - initiate device suspend
*
@@ -4587,6 +4611,8 @@ int amdgpu_device_suspend(struct drm_device *dev, bool
notify_clients)
if (r)
goto unwind_ip_phase1;
+ amdgpu_device_flush_file_entities(adev);
+
r = amdgpu_device_evict_resources(adev);
if (r)
goto unwind_userq;
---
base-commit: 7475ea8b58616b33c04afb976cf0fbaad9dd63fb
change-id: 20261116-superm1-drain-submissions-17d4414ab3b9
Best regards,
--
Mario Limonciello <[email protected]>