On Thu, 2026-11-12 at 12:59 +0000, Tvrtko Ursulin wrote:
> Since this is a lock inversion site we cannot lock

Which deadlock-circumvention strategies have you tried?

Pseudocode:

bool drm_sched_can_queue(struct drm_gpu_scheduler *sched,
                         struct drm_sched_entity *entity)
{
        struct drm_sched_job *s_job;

+       lockdep_assert_held(&entity->lock)

        s_job = drm_sched_entity_queue_peek(entity);


drm_sched_rq_select_entity(…)
{
retry:
        spin_lock(&rq->lock);
        
        walk_tree(…) {
                if (!spin_trylock(&entity->lock) {
                        spin_unlock(&rq-lock);
                        udelay(20);
                        goto retry;
                }
                ready = !list_empty(&entity->job_list);
        }
}


It appears very undesirable for someone to re-walk that tree, I get
that. But do we even know whether that would be a measurable problem in
practice? How often would it occur?


Alternatively, all the other paths that do entity_lock -> rq_lock could
be tested with trylock-patterns. They might be less impacted.

In drm_sched_entity_push_job(), for example, only occasionally are both
locks actually needed, when a job is the first job.

>  but truly have to have an unlocked solution.

How do you know that?


> This is effectively replacing one unlocked read (spsc_queue_peek) with a 
> different one (READ_ONCE(entity->job_credits)). Something unlocked is
> unavoidable here and I think this is more elegant and simple.

Code full of ACCESS_ONCE and RCU that is not RCU is most certainly not
elegant.

This is elegant:

spin_lock()
list_del()
spin_unlock()


With ACCESS_ONCE, manual barriers etc. pp. no one ever knows what's
accessed where with what ordering and why. It would then be better to
have the problem remain centralized in spsc_queue.h

As you say yourself, this here replaces the half-locked hard to reason
about spsc queue (we don't even know whether that thing is formally
correct) with a half-locked "locked" linked list.

It's no improvement in any way regarding our shared goal, which is a 
consistently locked codebase that finally rids us of the race
conditions which are plagueing this system for >10 years


There is really no other way than either redesigning drm_sched so that
the lock inversion disappears altogether, or to address the inversion
with strategies well established in CS.

> 
> And because it is an unlocked read it has to be annotated with 
> READ_ONCE(). Which means the write side, even though it is done under
> the lock, is paired up and marked with WRITE_ONCE().
> 
> 2. drm_sched_entity_pop_job()
> 
> This is not a lock inversion site but needs to call sleepable things 
> while considering the head of the job queue (->prepare_job()).
> 
> This one is the only remaining reason I used list_add_tail_rcu and 
> list_del_rcu, just so from here we can safely touch the first entry 
> while elsewhere more are being added.
> 
> It is possible to do it, as you suggest, by dropping and re-acquiring
> the lock. Note however that too would need a big fat comment why it is 
> safe.

Sure, one can write code comments. No big deal. The job is not in the
list anymore. The list is lock-protected. No one can race it.

There's only one worker popping. This must always be that way, the
entire drm_sched design is founded on that principle.

>  At which point, since both cases have a big fat comment explaining 
> the same thing apart from the implementation details, I do not see the 
> advantage of pointless lock cycling. READ_ONCE and WRITE_ONCE you 
> dislike would still remain.
> 
> As a side note it would be really good to eliminate the prepare_job 
> callback

Yes, the user should prepare everything needed before even submitting a
job. Maybe we can begin by marking it as deprecated.




P.

Reply via email to