On 2026-10-06 08:51, Dhruv B wrote:
In heterogeneous or non-uniform NUMA systems, a NUMA node may be
CPU-less, memory-less, or both CPU-less and memory-less.
Previously, kfd_create_vcrat_image_cpu() assumed that every online NUMA
node contained both compute units and memory, attempting to create both
CU and memory CRAT entries unconditionally for every online node.
Add explicit guards to:
1. Only create Compute Unit CRAT entries if the NUMA node has valid
CPU cores (kfd_numa_node_to_apic_id() != -1).
2. Only create Memory CRAT entries if the NUMA node actually contains
memory (node_state(numa_node_id, N_MEMORY)).
This avoids emitting invalid or empty CRAT subtype entries for CPU-less,
memory-less, or both CPU-less and memory-less NUMA nodes in virtual CRAT
generation.
@David Yat Sin, I think we currently support CPU-less and memory-less
NUMA domains already. I know we have CPU-less NUMA domains on MI300A in
NPS4/TPX mode. And I remember we've fixed bugs on NUMA CPU systems with
only one DIMM populated, which would result in a memory-less NUMA domain.
Therefore, I'm not sure whether this patch is really needed and I'd be
concerned about introducing a regression. Is ROCr able to handle the
changes being proposed here? David, could you take a look at this and
maybe even test it, or find someone who could help test it on the system
configurations I described?
@Dhruv B, how much testing have you done on this patch series. Given
that the bug in patch 2 slipped through in v1, I'm worried that it's not
very thorough.
That said, the patch series looks good to me now, in theory.
Regards,
Felix
Co-developed-by: Donet Tom <[email protected]>
Signed-off-by: Donet Tom <[email protected]>
Signed-off-by: Dhruv B <[email protected]>
---
drivers/gpu/drm/amd/amdkfd/kfd_crat.c | 40 +++++++++++++++------------
1 file changed, 22 insertions(+), 18 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_crat.c
b/drivers/gpu/drm/amd/amdkfd/kfd_crat.c
index 7dad8f30ff2a..329931b0edc6 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_crat.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_crat.c
@@ -1944,28 +1944,32 @@ static int kfd_create_vcrat_image_cpu(void
*pcrat_image, size_t *size)
}
/* Fill in Subtype: Compute Unit */
- ret = kfd_fill_cu_for_cpu(numa_node_id, &avail_size,
- crat_table->num_domains,
- (struct crat_subtype_computeunit *)sub_type_hdr);
- if (ret < 0)
- return ret;
- crat_table->length += sub_type_hdr->length;
- crat_table->total_entries++;
+ if (kfd_numa_node_to_apic_id(numa_node_id) != -1) {
+ ret = kfd_fill_cu_for_cpu(numa_node_id, &avail_size,
+ crat_table->num_domains,
+ (struct
crat_subtype_computeunit *)sub_type_hdr);
+ if (ret < 0)
+ return ret;
+ crat_table->length += sub_type_hdr->length;
+ crat_table->total_entries++;
- sub_type_hdr = (typeof(sub_type_hdr))((char *)sub_type_hdr +
- sub_type_hdr->length);
+ sub_type_hdr = (typeof(sub_type_hdr))((char
*)sub_type_hdr +
+ sub_type_hdr->length);
+ }
/* Fill in Subtype: Memory */
- ret = kfd_fill_mem_info_for_cpu(numa_node_id, &avail_size,
- crat_table->num_domains,
- (struct crat_subtype_memory *)sub_type_hdr);
- if (ret < 0)
- return ret;
- crat_table->length += sub_type_hdr->length;
- crat_table->total_entries++;
+ if (node_state(numa_node_id, N_MEMORY)) {
+ ret = kfd_fill_mem_info_for_cpu(numa_node_id,
&avail_size,
+ crat_table->num_domains,
+ (struct
crat_subtype_memory *)sub_type_hdr);
+ if (ret < 0)
+ return ret;
+ crat_table->length += sub_type_hdr->length;
+ crat_table->total_entries++;
- sub_type_hdr = (typeof(sub_type_hdr))((char *)sub_type_hdr +
- sub_type_hdr->length);
+ sub_type_hdr = (typeof(sub_type_hdr))((char
*)sub_type_hdr +
+ sub_type_hdr->length);
+ }
/* Fill in Subtype: IO Link */
#if defined(CONFIG_X86_64) && !defined(CONFIG_UML)
