Thank you for your contribution! Sashiko AI review found 2 potential issues to consider: - [High] drm/amdgpu: deadlock in amdgpu_device_flush_file_entities() on suspend - [Medium] drm/amdgpu: race in amdgpu_device_flush_file_entities() before run_job() --
commit 2d3632cb05b8d04352d9590da3fabb5b096f1b19 Author: Mario Limonciello <[email protected]> drm/amdgpu: drain queued scheduler jobs before suspend This commit adds amdgpu_device_flush_file_entities() to flush the context and VM scheduler entities of all open DRM files during amdgpu_device_suspend() before evicting resources. > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > index 19a566187dfb5..d18348116d3f1 100644 > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > @@ -4538,6 +4538,30 @@ void amdgpu_device_complete(struct drm_device *dev) > } > } > > +/** > + * amdgpu_device_flush_file_entities - drain scheduler entities of open files > + * @adev: amdgpu device object > + * > + * Wait for the scheduler entities owned by all open files (the context > + * entities used for command submission and the VM entities used for page > + * table updates) to run out of queued jobs, so that nothing can slip into > + * the rings after the hardware fences were waited on at suspend time. > + */ > +static void amdgpu_device_flush_file_entities(struct amdgpu_device *adev) > +{ > + struct drm_device *ddev = adev_to_drm(adev); > + struct drm_file *file; > + > + guard(mutex)(&ddev->filelist_mutex); > + list_for_each_entry(file, &ddev->filelist, lhead) { > + struct amdgpu_fpriv *fpriv = file->driver_priv; > + > + amdgpu_ctx_mgr_entity_flush(&fpriv->ctx_mgr, > + MAX_SCHEDULE_TIMEOUT); > + amdgpu_vm_wait_idle(&fpriv->vm, MAX_SCHEDULE_TIMEOUT); > + } > +} [Severity: Medium] Can drm_sched_entity_flush() return while the scheduler worker is still in the middle of popping a job and before sched->ops->run_job() has emitted its hardware fence? CPU0 (submit_wq) CPU1 (suspend) drm_sched_run_job_work() drm_sched_entity_pop_job() spsc_queue_pop(&entity->job_queue) drm_sched_rq_pop_entity(entity) amdgpu_device_suspend() amdgpu_device_flush_file_entities() ... drm_sched_entity_flush() drm_sched_entity_is_idle() == true amdgpu_device_evict_resources() amdgpu_fence_driver_hw_fini() amdgpu_fence_wait_empty() sched->ops->run_job(sched_job) amdgpu_job_run() amdgpu_ib_schedule() amdgpu_fence_emit() amdgpu_ring_commit() <-- ring written after fence wait complete_all(&entity->entity_idle) drm_sched_entity_is_idle() reports the entity idle as soon as spsc_queue_pop() empties entity->job_queue, before drm_sched_run_job_work() calls sched->ops->run_job() and complete_all(&entity->entity_idle). On APUs such as Picasso where amdgpu_device_evict_resources() returns immediately during S3 suspend, amdgpu_fence_driver_hw_fini() can finish waiting on the ring before the popped job emits its fence and commits the ring write pointer, leaving the ring hang on resume window open. > + > /** > * amdgpu_device_suspend - initiate device suspend > * > @@ -4587,6 +4611,8 @@ int amdgpu_device_suspend(struct drm_device *dev, bool > notify_clients) > if (r) > goto unwind_ip_phase1; > > + amdgpu_device_flush_file_entities(adev); [Severity: High] Can calling amdgpu_device_flush_file_entities() here deadlock runtime or system suspend when an entity has a job waiting on an unsignaled dependency? drivers/gpu/drm/scheduler/sched_entity.c:drm_sched_entity_flush() { ... if (current->flags & PF_EXITING) { if (timeout) ret = wait_event_timeout( sched->job_scheduled, drm_sched_entity_is_idle(entity), timeout); } else { wait_event_killable(sched->job_scheduled, drm_sched_entity_is_idle(entity)); } ... } Because the suspend task or runtime PM worker does not have PF_EXITING set, drm_sched_entity_flush() ignores the timeout and sleeps indefinitely in wait_event_killable() while holding ddev->filelist_mutex. In amdgpu_cs_ioctl(), a runtime PM reference is only held during the ioctl itself, and per-job runtime PM references are only taken in amdgpu_fence_emit() when a job reaches run_job(). If a client submits a job with an unsignaled syncobj dependency, runtime autosuspend can enter amdgpu_pmops_runtime_suspend() -> amdgpu_device_suspend() and block in amdgpu_device_flush_file_entities() while the device status is RPM_SUSPENDING. When another thread or process then calls amdgpu_cs_ioctl() to submit the work that would signal that syncobj, its runtime PM resume waits for RPM_SUSPENDING to complete, deadlocking both tasks. Killing the process owning the blocked entity does not break the wait either: amdgpu_flush() times out after 2 seconds and calls drm_sched_entity_kill(), which drains the entity without waking sched->job_scheduled, and the exiting task then blocks on ddev->filelist_mutex in drm_close_helper(). During system suspend, amdgpu_userq_suspend() also runs right before amdgpu_device_flush_file_entities() and unmaps usermode queues while userspace is already frozen, so any entity job waiting on a usermode queue or userspace fence can never make progress. -- Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
