Hi Christian,
> Yeah and you need to fix the fence orphaning and *NOT* the wait!
Agreed — found the exact orphan site. drm_sched_entity_push_job()
pushes the job onto entity->job_queue first, then
drm_sched_rq_add_entity() sees entity->stopped (under entity->lock)
and returns NULL: the scheduler is never woken, the job strands on
the spsc queue, and its fence never signals. In our repro the dying
process's fault path submits the migration job on its own killed
entity — it's the process-kill race, not a device-unplug race at
all, which is why drm_dev_enter() doesn't cover it.
This is the same failure Tong Liu reported in August 2025 (task
hang after job submission during process kill); that fix was
rejected for checking entity->stopped without the lock. The clean
fix is not to pre-check but to handle the NULL return while still
under entity->lock: drain the queue / schedule the kill-jobs work
so each stranded job's finished fence completes with -ESRCH.
> Blocking for all fences is correct shutdown order.
Agreed. v7 drops the timeout entirely — unbounded wait restored.
With fences guaranteed to signal, the copy_done error path becomes
genuinely reachable via the fence's error status, so the error
propagation and migrate->dst rollback Felix asked for stay — now
on a real error path instead of a timeout.
v7 is now implemented and tested on this machine (7.3.0-rc5 +
p1/p2 teardown+restore-stop, sched orphan fix, KFD fence-error
rework). Hot-unplug test: svmhog2 mid-migration plus a second KFD
process, then Thunderbolt deauthorize. With only the sched fix
the host still wedges — the kill-jobs drain prevents the orphan,
but the unplug path additionally needs the KFD teardown ordering
(processes torn down before device early-fini) and the restore
worker stopping on -ENODEV. With the full stack, across two
unplug/replug cycles and a 60-round kill-under-memory-pressure
soak: fences complete with error instead of orphaning, the KFD
processes get SIGBUS and reap cleanly, device tears down, host
stays alive, and re-authorizing gives a clean re-enumeration.
So your direction was right — the wait was never the bug — but
the orphan fix is necessary rather than sufficient for unplug.
On layering: I'm posting the sched fix standalone to dri-devel
("drm/scheduler: signal fences of jobs pushed to a stopped
entity") since it is drm-misc territory and also fixes the same
hang for any other driver pushing to a killed entity. The KFD
series then just needs the error propagation. Happy to fold it
into this series instead if you prefer.
Regards,
Olivier
