AMD General

HI Mario,

Agreed. However, the customer is on the 6.18.8 kernel. Does the review also 
apply to that older version? They are pushing for a fix ASAP because this 
affects their field product, and they have already waited a few weeks. I am 
considering giving them the patch for testing with a caveat: it is a 
developer-level patch that has not been fully validated, and I am in the 
process of upstreaming it.

Where should I post the patch for review? Also, is the review for upstream code 
only? Please provide guidance.

Thanks,
Tharshan

-----Original Message-----
From: Limonciello, Mario <[email protected]>
Sent: Wednesday, November 18, 2026 2:44 PM
To: Thothiralingam, Logatharshan <[email protected]>; Koenig, 
Christian <[email protected]>
Cc: [email protected]
Subject: Re: [PATCH] drm/amdgpu: drain queued scheduler jobs before suspend



On 11/18/26 13:01, Thothiralingam, Logatharshan wrote:
> AMD General
>
> Hi Christian,
>
> You are right, amdgpu_vm_wait_idle() is not needed. I was thinking of that 
> before, but I added it in the previous patch per your request. But now, I 
> have removed it.
>
> Please see the attached patch. I have tested it, and the hang is not seen. 
> Please let me know if the patch is good to go so that I can share the 
> solution with our customer.
>
> Then, I can create another patch against upstream and work on getting it 
> pushed.

IMV - I don't think you should share the patch to anyone to claim to fix a 
problem until we've successfully merged it.

I would post it for review like any other patch.  Let Sashiko bang on it.  Get 
review tags, merge it internally and let CI bang on it.

When all looks good there you can be confident in sharing it.

>
> Thanks,
> Tharshan
> -----Original Message-----
> From: Koenig, Christian <[email protected]>
> Sent: Wednesday, November 18, 2026 5:15 AM
> To: Thothiralingam, Logatharshan
> <[email protected]>; Limonciello, Mario
> <[email protected]>
> Cc: [email protected]
> Subject: Re: [PATCH] drm/amdgpu: drain queued scheduler jobs before
> suspend
>
> Hi Tharshan,
>
> oh that is a really good point, using amdgpu_vm_wait_idle() here is not 
> correct either because of the same problems as with the context flush.
>
>> dma_resv_wait_timeout(vm->root.bo->tbo.base.resv,
>> DMA_RESV_USAGE_READ, false,MAX_SCHEDULE_TIMEOUT);
>
> That looks like the right thing to do, yes.
>
> It only might be necessary to replace DMA_RESV_USAGE_READ with 
> DMA_RESV_USAGE_BOOKKEEP, but I'm not sure if KFD/userqueue suspend resume is 
> actually fully implemented and handling this.
>
> Maybe we should stick with DMA_RESV_USAGE_READ for now and I will let my team 
> double check the KFD/userqueu thing later on.
>
> Thanks,
> Christian.
>
> On 11/17/26 21:08, Thothiralingam, Logatharshan wrote:
>> AMD General
>>
>> Hi Christian,
>>
>> So adding dma_resv_wait_timeout(vm->root.bo->tbo.base.resv, 
>> DMA_RESV_USAGE_READ, false,MAX_SCHEDULE_TIMEOUT); before the 
>> amdgpu_vm_wait_idle() fixes that gfx reset. So it seems like 
>> amdgpu_vm_wait_idle() alone is not enough.
>>
>> Attached the patch again. Please let me know if it's good.
>>
>> Thanks,
>> Tharshan
>>
>> -----Original Message-----
>> From: Thothiralingam, Logatharshan
>> Sent: Tuesday, November 17, 2026 12:52 PM
>> To: Koenig, Christian <[email protected]>; Limonciello, Mario
>> <[email protected]>
>> Cc: [email protected]
>> Subject: RE: [PATCH] drm/amdgpu: drain queued scheduler jobs before
>> suspend
>>
>> Hi Chistian,
>>
>> Just having the amdgpu_vm_wait_idle() doesn't solve the issue. I
>> tested it and I do see the reset. I have attached a patch with the
>> dma_resv_wait_timeout(vm->root.bo->tbo.base.resv,
>> DMA_RESV_USAGE_READ, false,MAX_SCHEDULE_TIMEOUT); I am building it
>> now to test.  Please let me know your thoughts
>>
>> Thanks,
>> Tharshan
>>
>> -----Original Message-----
>> From: Thothiralingam, Logatharshan
>> Sent: Tuesday, November 17, 2026 8:39 AM
>> To: Koenig, Christian <[email protected]>; Limonciello, Mario
>> <[email protected]>
>> Cc: [email protected]
>> Subject: RE: [PATCH] drm/amdgpu: drain queued scheduler jobs before
>> suspend
>>
>> Hi Chistian,
>>
>> I looked a the amdgpu_vm_wait_idle() function and noticed that starting 
>> commit 1f02f20 
>> (https://github.com/eseiker/kernel-asahi/commit/1f02f2044bda1db1fd995bc35961ab075fa7b5a2)
>>   the wait on the VM's root page table fence list is removed. So correct me 
>> if I am wrong, this alone won't wait on the gfx jobs right?
>>
>> So should I add this before the idle call?
>>
>> dma_resv_wait_timeout(vm->root.bo->tbo.base.resv,
>> DMA_RESV_USAGE_READ, false,MAX_SCHEDULE_TIMEOUT);
>>
>>
>> Thanks,
>> Tharshan
>>
>> -----Original Message-----
>> From: Koenig, Christian <[email protected]>
>> Sent: Tuesday, November 17, 2026 4:37 AM
>> To: Limonciello, Mario <[email protected]>
>> Cc: [email protected]; Thothiralingam, Logatharshan
>> <[email protected]>
>> Subject: Re: [PATCH] drm/amdgpu: drain queued scheduler jobs before
>> suspend
>>
>> On 11/16/26 21:40, Mario Limonciello wrote:
>>> At suspend, amdgpu_fence_driver_hw_fini() only waits for the last
>>> fence emitted to each ring. Jobs still queued in a scheduler entity
>>> have not reached run_job() yet and so have no hardware fence.
>>> Userspace is frozen at this point, but such jobs can become runnable
>>> later, for example once a dependency signals, and the scheduler then
>>> pushes them into the ring after the hardware wait has finished and
>>> before the IP blocks are torn down. On resume the ring write pointer
>>> is reset, those jobs never signal and the GFX ring times out and hangs.
>>>
>>> This was observed as a rare (about 2 in 400 cycles) GFX hang on S3
>>> resume on Picasso.
>>>
>>> Flush the context and VM scheduler entities of every open file
>>> before evicting resources, mirroring what amdgpu_flush() does on file close.
>>> Like for any other dma_fence there is no timeout, the wait is not
>>> bounded.
>>>
>>> Reported-by: Logatharshan Thothiralingam
>>> <[email protected]>
>>> Closes:
>>> https://lore.kernel.org/amd-gfx/SA1SPRMB0073292C113C6B98D9282092F87F
>>> 2 @ SA1SPRMB0073.namprd12.prod.outlook.com/
>>> Signed-off-by: Mario Limonciello <[email protected]>
>>> ---
>>>   drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 26
>>> ++++++++++++++++++++++++++
>>>   1 file changed, 26 insertions(+)
>>>
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>> index 19a566187dfb5..d18348116d3f1 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>> @@ -4538,6 +4538,30 @@ void amdgpu_device_complete(struct drm_device *dev)
>>>        }
>>>   }
>>>
>>> +/**
>>> + * amdgpu_device_flush_file_entities - drain scheduler entities of
>>> +open files
>>> + * @adev: amdgpu device object
>>> + *
>>> + * Wait for the scheduler entities owned by all open files (the
>>> +context
>>> + * entities used for command submission and the VM entities used
>>> +for page
>>> + * table updates) to run out of queued jobs, so that nothing can
>>> +slip into
>>> + * the rings after the hardware fences were waited on at suspend time.
>>> + */
>>> +static void amdgpu_device_flush_file_entities(struct amdgpu_device
>>> +*adev) {
>>> +     struct drm_device *ddev = adev_to_drm(adev);
>>> +     struct drm_file *file;
>>> +
>>> +     guard(mutex)(&ddev->filelist_mutex);
>>> +     list_for_each_entry(file, &ddev->filelist, lhead) {
>>> +             struct amdgpu_fpriv *fpriv = file->driver_priv;
>>> +
>>> +             amdgpu_ctx_mgr_entity_flush(&fpriv->ctx_mgr,
>>> +                                         MAX_SCHEDULE_TIMEOUT);
>>
>> That is actually superfluous and even a little bit dangerous.
>>
>> amdgpu_ctx_mgr_entity_flush() is only supposed to be called from the process 
>> which has the file descriptor open and is closing it.
>>
>> We use that to detect if that process using the file descriptor was killed 
>> and we should discard queued up work instead of waiting for it to finish 
>> executing.
>>
>>> +             amdgpu_vm_wait_idle(&fpriv->vm, MAX_SCHEDULE_TIMEOUT);
>>
>> That here should be enough to wait for all work to complete.
>>
>> Regards,
>> Christian.
>>
>>> +     }
>>> +}
>>> +
>>>   /**
>>>    * amdgpu_device_suspend - initiate device suspend
>>>    *
>>> @@ -4587,6 +4611,8 @@ int amdgpu_device_suspend(struct drm_device *dev, 
>>> bool notify_clients)
>>>        if (r)
>>>                goto unwind_ip_phase1;
>>>
>>> +     amdgpu_device_flush_file_entities(adev);
>>> +
>>>        r = amdgpu_device_evict_resources(adev);
>>>        if (r)
>>>                goto unwind_userq;
>>>
>>> ---
>>> base-commit: 7475ea8b58616b33c04afb976cf0fbaad9dd63fb
>>> change-id: 20261116-superm1-drain-submissions-17d4414ab3b9
>>>
>>> Best regards,
>>> --
>>> Mario Limonciello <[email protected]>
>>>
>>
>

Reply via email to