On Fri, Oct 2, 2026 at 1:41 PM Saleel Kudchadker <[email protected]> wrote: > > On GC 9.4.3, 9.4.4 and 9.5.0, doorbell writes issued from shaders are > routed to an SDMA engine at 32-byte (4 doorbell index) granularity. > SDMA engine doorbell ranges are 10 indices apart, so the range of every > odd engine starts 2 indices into a 4-index block that begins in the > previous engine's range. A shader ringing the doorbell of a queue on an > odd engine therefore rings the previous engine instead. The intended > queue never sees the doorbell, so its work stalls and the application > hangs. > > Align the per-engine doorbell base up to a 4-index boundary on these > ASICs. Each engine only uses 4 of its 10 indices, so the aligned base > plus the queue offset stays within the engine's range. Even engines are > already aligned, so only the doorbell IDs of odd-engine queues change. > > Only doorbells rung by the device itself are routed this way. Doorbell > writes from the host, including the kernel's own SDMA rings, are not > affected. > > The doorbell ID of an SDMA queue is saved in CRIU checkpoints and > checked again on restore. A checkpoint taken before this change that > contains an SDMA queue on an odd engine will fail to restore with > -EINVAL. This is intentional: the restored process still holds the old > doorbell address, so accepting the stale ID would bring back the same > stall. > > Tested on MI300X with a shader-rung SDMA doorbell test: queues on odd > engines hung before this change and complete with it. > > v2: use u32 instead of uint32_t for the new locals (checkpatch) > v3: name the routing granularity, use dev->node_id, document the > layout invariants, describe impact > v4: express the granularity in doorbell indices, drop unrelated > cleanups, explain the CRIU restore behavior, add test note > > Fixes: 6df442a03d1a ("drm/amdgpu: add new doorbell assignment table for > aqua_vanjaram") > Cc: [email protected] > Signed-off-by: Saleel Kudchadker <[email protected]>
Acked-by: Alex Deucher <[email protected]> > --- > .../drm/amd/amdkfd/kfd_device_queue_manager.c | 32 +++++++++++++++---- > drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 7 ++++ > 2 files changed, 33 insertions(+), 6 deletions(-) > > diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c > b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c > index 6d12f9dc7f04..5e57576fba21 100644 > --- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c > +++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c > @@ -599,12 +599,32 @@ static int allocate_doorbell(struct qcm_process_device > *qpd, > * we need the physical sdma engine id in order to get the > * correct doorbell offset. > */ > - uint32_t valid_id = idx_offset[qpd->dqm->dev->node_id * > - > get_num_all_sdma_engines(qpd->dqm) + > - q->properties.sdma_engine_id] > - + > (q->properties.sdma_queue_id & 1) > - * > KFD_QUEUE_DOORBELL_MIRROR_OFFSET > - + > (q->properties.sdma_queue_id >> 1); > + u32 engine_base = idx_offset[dev->node_id * > + > get_num_all_sdma_engines(qpd->dqm) + > + q->properties.sdma_engine_id]; > + u32 valid_id; > + > + /* > + * With the aqua_vanjaram doorbell layout, a shader doorbell > + * write whose 32-byte block starts in the previous engine's > + * range rings that engine instead. Engine ranges are 10 > + * indices apart (sdma_doorbell_range = 20 dwords, halved for > + * 8-byte doorbells, in aqua_vanjaram_doorbell_index_init()), > + * which puts odd engines 2 indices into a shared block. > + * Aligning costs at most 2 indices, and 8 queues per engine > use > + * only 4 of the 10, so the aligned base stays in range. The > + * 512-index mirror offset keeps the alignment. > + */ > + if (KFD_GC_VERSION(dev) == IP_VERSION(9, 4, 3) || > + KFD_GC_VERSION(dev) == IP_VERSION(9, 4, 4) || > + KFD_GC_VERSION(dev) == IP_VERSION(9, 5, 0)) > + engine_base = ALIGN(engine_base, > + > KFD_SDMA_SHADER_DOORBELL_GRANULARITY); > + > + valid_id = engine_base + > + (q->properties.sdma_queue_id & 1) * > + KFD_QUEUE_DOORBELL_MIRROR_OFFSET + > + (q->properties.sdma_queue_id >> 1); > > if (restore_id && *restore_id != valid_id) > return -EINVAL; > diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h > b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h > index 2847a5ec5ede..b9a31489b1c8 100644 > --- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h > +++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h > @@ -133,6 +133,13 @@ > */ > #define KFD_QUEUE_DOORBELL_MIRROR_OFFSET 512 > > +/* > + * On GC 9.4.3, 9.4.4 and 9.5.0, doorbell writes issued from shaders are > + * routed to an SDMA engine at 32-byte granularity, i.e. 4 indices with the > + * 8-byte SDMA doorbells used on SOC15. > + */ > +#define KFD_SDMA_SHADER_DOORBELL_GRANULARITY 4 > + > /** > * enum kfd_ioctl_flags - KFD ioctl flags > * Various flags that can be set in &amdkfd_ioctl_desc.flags to control how > -- > 2.34.1 >
