Re: [PATCH v2 3/4] drm/amdkfd: Move gfx9 MQD to VRAM domain

[Felix Kuehling]

On 2025-12-04 17:51, Philip Yang wrote:


On 2025-12-03 12:55, Kuehling, Felix wrote:
On 2025-12-01 09:28, Philip Yang wrote:
To reduce queue switch latency further, move MQD to VRAM domain, add
AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS flag to allocate contiguous pages
using one buddy block.

Why does it need to be contiguous? In the next patch you're mapping 
it in the GART anyway.
Without AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS flag, amdgpu_bo_gpu_offset 
trigger this warning

    WARN_ON_ONCE(bo->tbo.resource->mem_type == TTM_PL_VRAM &&
             !(bo->flags & AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS));

This makes senses because we pass the FB aperture address to CP, this 
should be contiguous pages.

That's right, if you query the VRAM offset of the BO, you're assuming 
that it's contiguous. If you're mapping it in the GART, you wouldn't use 
that VRAM offset, you'd use the GART address instead. Since you don't 
want to use the mapping of the BO in the FB aperture (because that gives 
you the wrong MTYPE), you should not need to use amdgpu_bo_gpu_offset at 
all.

Regards,
  Felix



Regards,
Philip

Regards,
  Felix



Signed-off-by: Philip Yang <philip.y...@amd.com>
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c      | 3 ++-
  drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c | 2 +-
  2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c
index 9cd1660b8f60..c11e37915365 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c
@@ -329,7 +329,8 @@ int amdgpu_amdkfd_alloc_gtt_mem(struct 
amdgpu_device *adev, size_t size,
      bp.size = size;
      bp.byte_align = PAGE_SIZE;
      bp.domain = domain;
-    bp.flags = AMDGPU_GEM_CREATE_CPU_GTT_USWC;
+    bp.flags = AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS |
+           AMDGPU_GEM_CREATE_CPU_GTT_USWC;
      bp.type = ttm_bo_type_kernel;
      bp.resv = NULL;
      bp.bo_ptr_size = sizeof(struct amdgpu_bo);
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c 
b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c
index a489d43d5f64..c6945c842267 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c
@@ -139,7 +139,7 @@ static struct kfd_mem_obj *allocate_mqd(struct 
kfd_node *node,
              (ALIGN(q->ctl_stack_size, PAGE_SIZE) +
              ALIGN(sizeof(struct v9_mqd), PAGE_SIZE)) *
              NUM_XCC(node->xcc_mask),
-            AMDGPU_GEM_DOMAIN_GTT,
+            AMDGPU_GEM_DOMAIN_VRAM,
              &(mqd_mem_obj->gtt_mem),
              &(mqd_mem_obj->gpu_addr),
              (void *)&(mqd_mem_obj->cpu_ptr), true);