On 2026-11-17 10:24 +1100, Alistair Popple wrote: > On 2026-11-14 at 07:53 +1100, "Gregory Price (Meta)" <[email protected]> > wrote... > > On 2026-11-13 21:43 +0100, David Hildenbrand (Arm) wrote: > > > On 11/13/26 21:34, Thomas Hellström wrote: > > > > > I recall that John (cc) kept saying "GPUs and NUMA balancing is inherently > > > incompatible, you have to disable NUMA balancing". > > > > > > > If this is a NUMA-hotplugged GPU, then this is resolved by a private > > node, because this: > > > > @@ -6404,6 +6404,9 @@ static vm_fault_t do_numa_page(struct vm_fault *vmf) > > if (!folio || folio_is_zone_device(folio)) > > > > > > changes to > > > > if (!folio || folio_is_common_memory(folio)) > > > > And all folios on the GPU node are never subject to NUMA Balancing > > > > so balancing can stay enabled while the GPU memory is not subject to > > PROT_NONE spew. > > Hi Gregory, just to hopefully help clarify understanding a bit here this isn't > related to folio location and therefore private node won't help much. > > This about a driver getting a snapshot of current PTE state to program a > secondary MMU (in this case on a GPU, but can equally be a NIC or other > device). > > Even CPU memory being subject to NUMA balancing can have quite an adverse > impact > on GPU performance due to increased incidence of GPU faults, hence why John > says > GPUs (using SVM) are inherently incompatible with NUMA balancing. >
Ah, I follow. I'm still working through the performance implications of mmu_notifiers in the presence of PROT_NONE faults, but if i'm understanding correctly it tracks that you'd say it's incompatible. I presume here the driver needs a global view of memory (or at least all memory it can potentially DMA), rather than simply just a subset (some set of VMAs)? ~Gregory -- ~Gregory
