On 11/16/26 09:04, Christian König wrote:
Hi Tharshan,
the idea looks good in general, but could be simplified a bit by waiting for
the VM to be idle.
Question is why don't we already have that? Or do we already have that
somewhere else? I'm pretty sure that waiting for all submissions before S3
should already been implemented somewhere.
Adding Mario to answer that question, he has been working on suspend/resume
quite a lot recently.
I thought we had something like this, but looking through the code base
it seems not (or I missed it).
I think it's the wrong kind of patch though. I believe we should use
some of the existing flush functions and do it at the same time that we
evict resources. Something like this (attached).
Can you see if this also helps?
Adding Vitaly to double check if we have already a test case for that.
Thanks,
Christian.
On 11/13/26 17:44, Thothiralingam, Logatharshan wrote:
AMD General
Hi,
Please help me review the attached patch.
A gfx hang that occurs 2 out of 400 times when performing S3 cycles. The issue
is seen with the kernel version 6.18.8 and on AMD hardware; Picasso gfx9 APU.
When going to S3 suspend, the driver checks whether GPU has completed its
tasks, but it doesn’t wait for jobs that are submitted to the scheduler. As a
result, queued up jobs slip into the ring and are lost when WPTR is set to 0.
On wake up, we hit a timeout followed by a hang.
The fix is to wait for all scheduled task to finish before going to S3. I
tested the patch and the issue is not seen again.
Thanks,
Tharshan
From aecba5949832af50e62a77825734670acdab0bf4 Mon Sep 17 00:00:00 2001
From: Mario Limonciello <[email protected]>
Date: Mon, 16 Nov 2026 09:18:06 -0600
Subject: [PATCH] drm/amdgpu: drain queued user submissions before suspend
At suspend, amdgpu_fence_driver_hw_fini() only waits for the last fence
emitted to each ring. Jobs still queued in a scheduler entity have not
reached run_job() yet and so have no hardware fence. Userspace is frozen
at this point, but such jobs can become runnable later, for example once
a dependency signals, and the scheduler then pushes them into the ring
after the hardware wait has finished and before the IP blocks are torn
down. On resume the ring write pointer is reset, those jobs never
signal and the GFX ring times out and hangs.
This was observed as a rare (about 2 in 400 cycles) GFX hang on S3 resume
on Picasso.
Flush the context entities and the VM update entities of every open file
before evicting resources, mirroring what amdgpu_flush() does on file
close. The wait is bounded; failing to drain in time only logs a warning
and suspend continues, leaving the existing hardware fence wait to deal
with the rest.
Reported-by: Logatharshan Thothiralingam<[email protected]>
Closes: https://lore.kernel.org/amd-gfx/sa1sprmb0073292c113c6b98d9282092f8...@sa1sprmb0073.namprd12.prod.outlook.com/
Signed-off-by: Mario Limonciello <[email protected]>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 34 ++++++++++++++++++++++
1 file changed, 34 insertions(+)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 19a566187dfb5..e336c19640faa 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -4538,6 +4538,38 @@ void amdgpu_device_complete(struct drm_device *dev)
}
}
+/**
+ * amdgpu_device_flush_user_submissions - drain queued user submissions
+ * @adev: amdgpu device object
+ *
+ * Wait for the scheduler entities of all open files (context entities and
+ * VM update entities) to run out of queued jobs, so that nothing can slip
+ * into the rings after the hardware fences were waited on at suspend time.
+ * Failing to drain in time is not fatal, the hardware fence wait and the
+ * IP suspend handle whatever is left.
+ */
+static void amdgpu_device_flush_user_submissions(struct amdgpu_device *adev)
+{
+ struct drm_device *ddev = adev_to_drm(adev);
+ long timeout = MAX_WAIT_SCHED_ENTITY_Q_EMPTY;
+ struct drm_file *file;
+
+ guard(mutex)(&ddev->filelist_mutex);
+ list_for_each_entry(file, &ddev->filelist, lhead) {
+ struct amdgpu_fpriv *fpriv = file->driver_priv;
+
+ timeout = amdgpu_ctx_mgr_entity_flush(&fpriv->ctx_mgr, timeout);
+ if (timeout > 0)
+ timeout = amdgpu_vm_wait_idle(&fpriv->vm, timeout);
+ if (timeout <= 0) {
+ dev_warn(adev->dev,
+ "failed to drain user submissions (%ld)\n",
+ timeout);
+ break;
+ }
+ }
+}
+
/**
* amdgpu_device_suspend - initiate device suspend
*
@@ -4587,6 +4619,8 @@ int amdgpu_device_suspend(struct drm_device *dev, bool notify_clients)
if (r)
goto unwind_ip_phase1;
+ amdgpu_device_flush_user_submissions(adev);
+
r = amdgpu_device_evict_resources(adev);
if (r)
goto unwind_userq;
--
2.43.0