(Sorry for multiple mails replying to same, lei/lore are broken again so my setup isn't working properly).
I tried to fixup the conflicts here to run tests locally but there's too many and I messed it up. Could you please resend this series rebased on mm-unstable please? Thanks, Lorenzo On Mon, Dec 01, 2025 at 10:46:11AM -0700, Nico Pache wrote: > The following series provides khugepaged with the capability to collapse > anonymous memory regions to mTHPs. > > To achieve this we generalize the khugepaged functions to no longer depend > on PMD_ORDER. Then during the PMD scan, we use a bitmap to track individual > pages that are occupied (!none/zero). After the PMD scan is done, we use > the bitmap to find the optimal mTHP sizes for the PMD range. The > restriction on max_ptes_none is removed during the scan, to make sure we > account for the whole PMD range in the bitmap. When no mTHP size is > enabled, the legacy behavior of khugepaged is maintained. > > We currently only support max_ptes_none values of 0 or HPAGE_PMD_NR - 1 > (ie 511). If any other value is specified, the kernel will emit a warning > and no mTHP collapse will be attempted. If a mTHP collapse is attempted, > but contains swapped out, or shared pages, we don't perform the collapse. > It is now also possible to collapse to mTHPs without requiring the PMD THP > size to be enabled. These limitiations are to prevent collapse "creep" > behavior. This prevents constantly promoting mTHPs to the next available > size, which would occur because a collapse introduces more non-zero pages > that would satisfy the promotion condition on subsequent scans. > > Patch 1: Refactor/rename hpage_collapse > Patch 2: Refactoring to combine madvise_collapse and khugepaged > Patch 3-8: Generalize khugepaged functions for arbitrary orders and > introduce some helper functions > Patch 9: skip collapsing mTHP to smaller orders > Patch 10-11: Add per-order mTHP statistics and tracepoints > Patch 12: Introduce collapse_allowable_orders > Patch 13-15: Introduce bitmap and mTHP collapse support, fully enabled > Patch 16: Documentation > > --------- > Testing > --------- > - Built for x86_64, aarch64, ppc64le, and s390x > - selftests mm > - I created a test script that I used to push khugepaged to its limits > while monitoring a number of stats and tracepoints. The code is > available here[1] (Run in legacy mode for these changes and set mthp > sizes to inherit) > The summary from my testings was that there was no significant > regression noticed through this test. In some cases my changes had > better collapse latencies, and was able to scan more pages in the same > amount of time/work, but for the most part the results were consistent. > - redis testing. I tested these changes along with my defer changes > (see followup [2] post for more details). We've decided to get the mTHP > changes merged first before attempting the defer series. > - some basic testing on 64k page size. > - lots of general use. > > V13 Changes: > - Lots of minor nits, cleanups, comments, and renames > - Bitmap function simplification and more helpers (Wei, Lorenzo) > - Max_ptes_none (0 or 511) restriction > - commit description expansion > - list all reachable enum values in mthp_collapse() > - Fix ppc64 compile error due to using HPAGE_PMD_ORDER (replace with > ilog2(MAX_PTRS_PER_PTE)) > > V12: https://lore.kernel.org/lkml/[email protected]/ > V11: https://lore.kernel.org/lkml/[email protected]/ > V10: https://lore.kernel.org/lkml/[email protected]/ > V9 : https://lore.kernel.org/lkml/[email protected]/ > V8 : https://lore.kernel.org/lkml/[email protected]/ > V7 : https://lore.kernel.org/lkml/[email protected]/ > V6 : https://lore.kernel.org/lkml/[email protected]/ > V5 : https://lore.kernel.org/lkml/[email protected]/ > V4 : https://lore.kernel.org/lkml/[email protected]/ > V3 : https://lore.kernel.org/lkml/[email protected]/ > V2 : https://lore.kernel.org/lkml/[email protected]/ > V1 : https://lore.kernel.org/lkml/[email protected]/ > > A big thanks to everyone that has reviewed, tested, and participated in > the development process. Its been a great experience working with all of > you on this endeavour. > > [1] - https://gitlab.com/npache/khugepaged_mthp_test > [2] - https://lore.kernel.org/lkml/[email protected]/ > > Baolin Wang (1): > khugepaged: run khugepaged for all orders > > Dev Jain (1): > khugepaged: generalize alloc_charge_folio() > > Nico Pache (14): > khugepaged: rename hpage_collapse_* to collapse_* > introduce collapse_single_pmd to unify khugepaged and madvise_collapse > khugepaged: generalize hugepage_vma_revalidate for mTHP support > khugepaged: introduce is_mthp_order helper > khugepaged: generalize __collapse_huge_page_* for mTHP support > khugepaged: introduce collapse_max_ptes_none helper function > khugepaged: generalize collapse_huge_page for mTHP collapse > khugepaged: skip collapsing mTHP to smaller orders > khugepaged: add per-order mTHP collapse failure statistics > khugepaged: improve tracepoints for mTHP orders > khugepaged: introduce collapse_allowable_orders helper function > khugepaged: Introduce mTHP collapse support > khugepaged: avoid unnecessary mTHP collapse attempts > Documentation: mm: update the admin guide for mTHP collapse > > Documentation/admin-guide/mm/transhuge.rst | 80 ++- > include/linux/huge_mm.h | 5 + > include/trace/events/huge_memory.h | 34 +- > mm/huge_memory.c | 11 + > mm/khugepaged.c | 696 +++++++++++++++------ > mm/mremap.c | 2 +- > 6 files changed, 618 insertions(+), 210 deletions(-) > > -- > 2.51.1 >
