As a step towards removing pagetable ctor/dtor calls for kernel
mm's, move the memory accounting from the ctor/dtor to
pagetable_{alloc,free}.The ctor is normally called after pagetable_alloc() (and dtor before pagetable_free()) so this patch should have no functional effect in most cases. Where calls to ctor/dtor are currently missing (e.g. x86's split_large_page()), this patch should improve the accounting. This patch is based on Matthew's earlier implementation linked below, with the difference that when freeing the accounting is updated in __pagetable_free(), meaning that the update is delayed until the pages are actually freed in the CONFIG_ASYNC_KERNEL_PGTABLE_FREE case. This shouldn't matter in practice. Suggested-by: "Matthew Wilcox (Oracle)" <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Kevin Brodsky <[email protected]> --- include/linux/mm.h | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index b336eb32faa0..361a0d7b74c5 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3681,6 +3681,11 @@ static inline struct ptdesc *pagetable_alloc_noprof(gfp_t gfp, unsigned int orde { struct page *page = alloc_pages_noprof(gfp | __GFP_COMP, order); + if (!page) + return NULL; + + lruvec_stat_add_folio(page_folio(page), NR_PAGETABLE); + return page_ptdesc(page); } #define pagetable_alloc(...) alloc_hooks(pagetable_alloc_noprof(__VA_ARGS__)) @@ -3689,6 +3694,8 @@ static inline void __pagetable_free(struct ptdesc *pt) { struct page *page = ptdesc_page(pt); + lruvec_stat_sub_folio(page_folio(page), NR_PAGETABLE); + __free_pages(page, compound_order(page)); } @@ -3797,7 +3804,6 @@ static inline void __pagetable_ctor(struct ptdesc *ptdesc) struct folio *folio = ptdesc_folio(ptdesc); __folio_set_pgtable(folio); - lruvec_stat_add_folio(folio, NR_PAGETABLE); } static inline void pagetable_dtor(struct ptdesc *ptdesc) @@ -3806,7 +3812,6 @@ static inline void pagetable_dtor(struct ptdesc *ptdesc) ptlock_free(ptdesc); __folio_clear_pgtable(folio); - lruvec_stat_sub_folio(folio, NR_PAGETABLE); } static inline void pagetable_dtor_free(struct ptdesc *ptdesc) -- 2.51.2
