When post_alloc_hook() needs to zero a page for an explicit __GFP_ZERO allocation and user_addr is set, use folio_zero_user() instead of kernel_init_pages(). This zeros near the faulting address last, keeping those cachelines hot for the impending user access.
folio_zero_user() is only used for explicit __GFP_ZERO, not for init_on_alloc. On architectures with virtually-indexed caches (e.g., ARM), clear_user_highpage() performs per-line cache operations; using it for init_on_alloc would add overhead that kernel_init_pages() avoids (the page fault path flushes the cache at PTE installation time regardless). No functional change yet: current callers do not pass __GFP_ZERO for user pages (they zero at the callsite instead). Subsequent patches will convert them. Signed-off-by: Michael S. Tsirkin <[email protected]> Assisted-by: Claude:claude-opus-4-6 --- mm/page_alloc.c | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 04c03c56abec..7791bc1eeefa 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1882,9 +1882,20 @@ inline void post_alloc_hook(struct page *page, unsigned int order, for (i = 0; i != 1 << order; ++i) page_kasan_tag_reset(page + i); } - /* If memory is still not initialized, initialize it now. */ - if (init) - kernel_init_pages(page, 1 << order); + /* + * If memory is still not initialized, initialize it now. + * When __GFP_ZERO was explicitly requested and user_addr is set, + * use folio_zero_user() which zeros near the faulting address + * last, keeping those cachelines hot. For init_on_alloc, use + * kernel_init_pages() to avoid unnecessary cache flush overhead + * on architectures with virtually-indexed caches. + */ + if (init) { + if ((gfp_flags & __GFP_ZERO) && user_addr != USER_ADDR_NONE) + folio_zero_user(page_folio(page), user_addr); + else + kernel_init_pages(page, 1 << order); + } set_page_owner(page, order, gfp_flags); page_table_check_alloc(page, order); -- MST
