From: Prakash Gupta <[email protected]> System heap currently uses a fixed allocation order, {8, 4, 0} while requesting pages from the buddy allocator which correspond to 1 MiB, 64 KiB and 4 KiB only on systems using 4 KiB page size.
Extending on patch from Juan [1] for ARM64 targets with LPAE, a 1 MiB allocation does not correspond to a PMD block size. For a 4 KiB-granule IOMMU, a 2 MiB allocation can be mapped using a single PMD block entry, reducing IOMMU TLB pressure by requiring fewer translation entries. Use allocation orders corresponding to the ARM64 PMD block and contiguous-PTE sizes for 4 KiB, 16 KiB and 64 KiB page configurations. ARM64 applicable orders are selected for ARM64 targets with default orders being used for rest of the architectures. [1] https://lore.kernel.org/all/[email protected]/ Co-developed-by: Juan Yescas <[email protected]> Signed-off-by: Juan Yescas <[email protected]> Co-developed-by: Pankaj Patil <[email protected]> Signed-off-by: Pankaj Patil <[email protected]> Signed-off-by: Prakash Gupta <[email protected]> --- drivers/dma-buf/heaps/system_heap.c | 27 +++++++++++++++++++++++---- 1 file changed, 23 insertions(+), 4 deletions(-) diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/system_heap.c index c8959eadc71d..7a3e26c3dd43 100644 --- a/drivers/dma-buf/heaps/system_heap.c +++ b/drivers/dma-buf/heaps/system_heap.c @@ -22,6 +22,7 @@ #include <linux/module.h> #include <linux/pgtable.h> #include <linux/scatterlist.h> +#include <linux/sizes.h> #include <linux/slab.h> #include <linux/vmalloc.h> @@ -57,12 +58,30 @@ struct dma_heap_attachment { | __GFP_COMP) static gfp_t order_flags[] = {HIGH_ORDER_GFP, HIGH_ORDER_GFP, LOW_ORDER_GFP}; /* - * The selection of the orders used for allocation (1MB, 64K, 4K) is designed - * to match with the sizes often found in IOMMUs. Using order 4 pages instead - * of order 0 pages can significantly improve the performance of many IOMMUs - * by reducing TLB pressure and time spent updating page tables. + * For ARM64, allocation orders correspond to the PMD block and CONT PTE + * hugepage sizes for each CPU page granule, which also align with the + * TLB entry sizes of a 4K-granule IOMMU (arm-smmu, arm-smmu-v3). + * Each order = (level_shift - PAGE_SHIFT), derived from: + * PMD_SHIFT = (PAGE_SHIFT - 3) * 2 + 3 + * CONT_PTE_SHIFT = CONFIG_ARM64_CONT_PTE_SHIFT + PAGE_SHIFT + * + * 4K CPU (PAGE_SHIFT=12): PMD=2MiB (21-12=9), CONT_PTE=64KiB (16-12=4) + * 16K CPU (PAGE_SHIFT=14): PMD=32MiB (25-14=11), CONT_PTE=2MiB (21-14=7) + * 64K CPU (PAGE_SHIFT=16): PMD=512MiB (29-16=13), CONT_PTE=2MiB (21-16=5) + */ +#if defined(CONFIG_ARM64) && (PAGE_SIZE == SZ_4K) +static const unsigned int orders[] = {21 - PAGE_SHIFT, 16 - PAGE_SHIFT, 0}; +#elif defined(CONFIG_ARM64) && (PAGE_SIZE == SZ_16K) +static const unsigned int orders[] = {25 - PAGE_SHIFT, 21 - PAGE_SHIFT, 0}; +#elif defined(CONFIG_ARM64) && (PAGE_SIZE == SZ_64K) +static const unsigned int orders[] = {29 - PAGE_SHIFT, 21 - PAGE_SHIFT, 0}; +#else +/* Fallback to default allocation order for non-arm64 architectures + * On 4K-page systems, these orders correspond to 1MiB, 64KiB and 4KiB, + * matching sizes commonly supported by IOMMUs. */ static const unsigned int orders[] = {8, 4, 0}; +#endif #define NUM_ORDERS ARRAY_SIZE(orders) static int system_heap_set_page_decrypted(struct page *page) --- base-commit: c2cd463d6ee7d55a3ec0719d93c49ff99022d58f change-id: 20260916-dma_buf_improvement-a095ae567e5f Best regards, -- Pankaj Patil <[email protected]>
