Just taking a closer look now since I assumed this is some sort of RFC
for AMD to look first.
On Fri, 2026-11-06 at 14:57 +0000, Tvrtko Ursulin wrote:
> Migrate the entity job queue from the single user SPSC queue (single
> producer, single consumer) to the standard list_head plus a spinlock in
> order to make it easier to argue about correctnes and bring the code base
> close to the usual kernel patterns.
>
> The main thing which needs to be mentioned is the seeming usage of RCU,
> which isn't really so, but only for two trivial reasons:
If that were really the only way then we would, unfortunately, not at
all end up with "usual kernel patterns" :(
>
> 1.
> We need to use the list operations which are safe for unlocked access to
> the list head, while parallel (locked) threads can be appending other
> jobs.
>
> This so the run job worker is able to process the job dependencies which
> includes it calling a potentially sleeping prepare_job() callback. Hence
> it is not possible to hold the entity->lock during that stage.
Why can't it pull a job from the list, release the lock and then do
that? There is only ever 1 worker at a time.
> 2.
> We need to work around the infamous entity<->rq locking inversion which
> would now trigger slightly earlier in the run job worker, from the entity
> selection stage, where there was previosuly the other "spsc queue peek"
> call site.
Is spsc_queue_peek() the only place where an inversion would occur?
> All the list_*_rcu usage stem from those two requirements and there are no
> actual RCU requirements.
IDK, having RCU functions (and all those ACCESS_ONCEs!) in the code
base that is now supposed to be a simple lock + list does not sound
like a good idea for maintainability at all, especially if it's RCU
that is no actual RCU.
[…]
> static struct drm_sched_job *
> drm_sched_entity_queue_pop(struct drm_sched_entity *entity)
> {
> - struct spsc_node *node;
> + struct drm_sched_job *next_job, *sched_job;
>
> - node = spsc_queue_pop(&entity->job_queue);
> - if (!node)
> + lockdep_assert_held(&entity->lock);
> +
> + sched_job = list_first_entry_or_null(&entity->job_queue,
> + typeof(*sched_job), queue_link);
> + if (!sched_job)
> return NULL;
>
> - return container_of(node, struct drm_sched_job, queue_node);
> + list_del_rcu(&sched_job->queue_link);
> +
> +#if IS_ENABLED(CONFIG_TRACEPOINTS) || IS_ENABLED(CONFIG_DRM_SCHED_KUNIT_TEST)
> + WRITE_ONCE(entity->job_count, entity->job_count - 1);
You're literally holding the entity->lock, what do you need read once
for?
> +#endif
> +
> + next_job = list_first_entry_or_null(&entity->job_queue,
> + typeof(*next_job), queue_link);
> + if (next_job)
> + WRITE_ONCE(entity->job_credits, next_job->credits);
> + else
> + WRITE_ONCE(entity->job_credits, 0);
Same.
> +
> + return sched_job;
> }
>
>
[…]
> EXPORT_SYMBOL(drm_sched_entity_kill);
> @@ -557,10 +556,20 @@ struct drm_sched_job *drm_sched_entity_pop_job(struct
> drm_sched_entity *entity)
> /* Helper to avoid dropping the reference while the entity lock is held,
> * just to have some more robustness.
> */
> + struct drm_sched_job *sched_job, *next_job;
> struct dma_fence *prev_last_scheduled;
> - struct drm_sched_job *sched_job;
>
> - sched_job = drm_sched_entity_queue_peek(entity);
> + /*
> + * We cannot lock the entity at this point since processing dependencies
> + * needs to be able to call a sleeping ops->prepare_job(). Note that RCU
> + * lock is not stricly required since single workqueue is the only
> + * caller and that guarantees a single consumer. Ie. the job cannot
> + * disappear on us.
> + */
> + rcu_read_lock();
> + sched_job = list_first_or_null_rcu(&entity->job_queue,
> + typeof(*sched_job), queue_link);
> + rcu_read_unlock();
You can take the lock, take the job, release the lock, do your sleeping
actions, and then take the lock again, can't you?
P.