On Wed, May 20, 2026 at 7:55 PM Wei Yang <[email protected]> wrote:
>
> On Wed, May 20, 2026 at 06:05:31AM -0600, Nico Pache wrote:
> >On Tue, May 12, 2026 at 9:44 AM Wei Yang <[email protected]> wrote:
> >>
> >> On Mon, May 11, 2026 at 12:58:11PM -0600, Nico Pache wrote:
> >> >Enable khugepaged to collapse to mTHP orders. This patch implements the
> >> >main scanning logic using a bitmap to track occupied pages and a stack
> >> >structure that allows us to find optimal collapse sizes.
> >> >
> >> >Previous to this patch, PMD collapse had 3 main phases, a light weight
> >> >scanning phase (mmap_read_lock) that determines a potential PMD
> >> >collapse, an alloc phase (mmap unlocked), then finally heavier collapse
> >> >phase (mmap_write_lock).
> >> >
> >> >To enabled mTHP collapse we make the following changes:
> >> >
> >> >During PMD scan phase, track occupied pages in a bitmap. When mTHP
> >> >orders are enabled, we remove the restriction of max_ptes_none during the
> >> >scan phase to avoid missing potential mTHP collapse candidates. Once we
> >> >have scanned the full PMD range and updated the bitmap to track occupied
> >> >pages, we use the bitmap to find the optimal mTHP size.
> >> >
> >> >Implement collapse_scan_bitmap() to perform binary recursion on the bitmap
> >> >and determine the best eligible order for the collapse. A stack structure
> >> >is used instead of traditional recursion to manage the search. This also
> >> >prevents a traditional recursive approach when the kernel stack struct is
> >> >limited. The algorithm recursively splits the bitmap into smaller chunks
> >> >to
> >> >find the highest order mTHPs that satisfy the collapse criteria. We start
> >> >by attempting the PMD order, then moved on the consecutively lower orders
> >> >(mTHP collapse). The stack maintains a pair of variables (offset, order),
> >> >indicating the number of PTEs from the start of the PMD, and the order of
> >> >the potential collapse candidate.
> >> >
> >> >The algorithm for consuming the bitmap works as such:
> >> > 1) push (0, HPAGE_PMD_ORDER) onto the stack
> >> > 2) pop the stack
> >> > 3) check if the number of set bits in that (offset,order) pair
> >> > statisfy the max_ptes_none threshold for that order
> >> > 4) if yes, attempt collapse
> >> > 5) if no (or collapse fails), push two new stack items representing
> >> > the left and right halves of the current bitmap range, at the
> >> > next lower order
> >> > 6) repeat at step (2) until stack is empty.
> >> >
> >> >Below is a diagram representing the algorithm and stack items:
> >> >
> >> > offset mid_offset
> >> > | |
> >> > | |
> >> > v v
> >> > ____________________________________
> >> > | PTE Page Table |
> >> > --------------------------------------
> >> > <-------><------->
> >> > order-1 order-1
> >> >
> >> >mTHP collapses reject regions containing swapped out or shared pages.
> >> >This is because adding new entries can lead to new none pages, and these
> >> >may lead to constant promotion into a higher order mTHP. A similar
> >> >issue can occur with "max_ptes_none > HPAGE_PMD_NR/2" due to a collapse
> >> >introducing at least 2x the number of pages, and on a future scan will
> >> >satisfy the promotion condition once again. This issue is prevented via
> >> >the collapse_max_ptes_none() function which imposes the max_ptes_none
> >> >restrictions above.
> >> >
> >> >We currently only support mTHP collapse for max_ptes_none values of 0
> >> >and HPAGE_PMD_NR - 1. resulting in the following behavior:
> >> >
> >> > - max_ptes_none=0: Never introduce new empty pages during collapse
> >> > - max_ptes_none=HPAGE_PMD_NR-1: Always try collapse to the highest
> >> > available mTHP order
> >> >
> >> >Any other max_ptes_none value will emit a warning and skip mTHP collapse
> >> >attempts. There should be no behavior change for PMD collapse.
> >> >
> >> >Once we determine what mTHP sizes fits best in that PMD range a collapse
> >> >is attempted. A minimum collapse order of 2 is used as this is the lowest
> >> >order supported by anon memory as defined by THP_ORDERS_ALL_ANON.
> >> >
> >> >Currently madv_collapse is not supported and will only attempt PMD
> >> >collapse.
> >> >
> >> >We can also remove the check for is_khugepaged inside the PMD scan as
> >> >the collapse_max_ptes_none() function handles this logic now.
> >> >
> >> >Signed-off-by: Nico Pache <[email protected]>
> >>
> >> [...]
> >>
> >> >+static int mthp_collapse(struct mm_struct *mm, unsigned long address,
> >> >+ int referenced, int unmapped, struct collapse_control *cc,
> >> >+ unsigned long enabled_orders)
> >> >+{
> >> >+ unsigned int nr_occupied_ptes, nr_ptes;
> >> >+ int max_ptes_none, collapsed = 0, stack_size = 0;
> >> >+ unsigned long collapse_address;
> >> >+ struct mthp_range range;
> >> >+ u16 offset;
> >> >+ u8 order;
> >> >+
> >> >+ collapse_mthp_stack_push(cc, &stack_size, 0, HPAGE_PMD_ORDER);
> >> >+
> >> >+ while (stack_size) {
> >> >+ range = collapse_mthp_stack_pop(cc, &stack_size);
> >> >+ order = range.order;
> >> >+ offset = range.offset;
> >> >+ nr_ptes = 1UL << order;
> >> >+
> >> >+ if (!test_bit(order, &enabled_orders))
> >> >+ goto next_order;
> >> >+
> >> >+ max_ptes_none = collapse_max_ptes_none(cc, NULL, order);
> >>
> >> I am thinking whether there is a behavioral change for
> >> userfaultfd_armed(vma).
> >>
> >> collapse_single_pmd()
> >> collapse_scan_pmd
> >> max_ptes_none = collapse_max_ptes_none(cc, vma)
> >> max_ptes_none = KHUGEPAGED_MAX_PTES_LIMIT --- (1)
> >> mthp_collapse
> >> max_ptes_none = collapse_max_ptes_none(cc, NULL) --- (2)
> >> collapse_huge_page(mm)
> >> hugepage_vma_revalidate(&vma)
> >> __collapse_huge_page_isolate(vma)
> >> max_ptes_none = collapse_max_ptes_none(cc, vma)
> >>
> >> Before mthp_collapse() introduced, userfaultfd_armed(vma) is skipped if
> >> there
> >> is any pte_none_or_zero() in collapse_scan_pmd().
> >>
> >> But now, max_ptes_none could be set to KHUGEPAGED_MAX_PTES_LIMIT at (1), so
> >> that we can scan all the pte to get the bitmap. This means
> >> userfaultfd_armed(vma) could continue even with pte_none_or_zero().
> >>
> >> Then in mthp_collapse(), collapse_max_ptes_none() at (2) ignores
> >> userfaultfd_armed(vma), which means it will continue to collapse a
> >> userfaultfd_armed(vma) when there is pte_none_or_zero().
> >>
> >> The good news is we will stop at __collapse_huge_page_isolate(), where we
> >> get collapse_max_ptes_none() with vma. But we already did a lot of work.
> >
> >Good catch!
> >
> >As you stated we eventually ensure we respect the uffd checks. So
> >there are no correctness issues, just the potential for wasted cycles.
> >
> >At (1) we only do this if mTHPs are enabled. If that is the case, the
> >only waste that can arise is at the PMD order, as that order respects
> >the max_ptes_none value.
> >
> >I think one approach is to gate (1) with the uffd check as well. That
> >way, if mTHPs are enabled and its uffd-armed, max_ptes_none will stay
> >at 0, and we bail early on the scan early if any none_ptes are hit.
> >
> >But then we lose the ability to collapse to mTHPs that are uffd-armed,
> >where the PMD has none/zero-ptes and the mTHP fully has 0
> >non-none/zero-ptes.
> >
> >ie) assume a PMD is 16 x's [xxxxxxxx00000000]
> >where x is a populated pte and 0 is not
> >If we guard this scan (1), then we will never check if its possible to
> >collapse to the smaller orders.
> >
> >Let me know if you see a flaw in my logic, I think it's best to keep it as
> >is?
> >
>
> Yes, gate it at (1) is not a proper place.
>
> I am thinking whether we could pass vma to (2)? So that we could respect
> uffd-armed?
Ok, sorry I never replied but i did implement it at (2). Sashiko
brought up a good point that this can result in a UAF; I verified
that. I'm going to send a fixup to the v18 to undo the change.
I added this to my todo list, and will look into optimizing/finding a
solution for this in a future series. The good thing is, as you stated
earlier, this can result in some wasted work, but it is not logically
incorrect overall.
Cheers,
-- Nico
>
> >>
> >> Not sure if I missed something.
> >>
> >> >+
> >> >+ if (max_ptes_none < 0)
> >> >+ return collapsed;
> >> >+
> >> >+ nr_occupied_ptes = collapse_mthp_count_present(cc, offset,
> >> >+ nr_ptes);
> >> >+
> >> >+ if (nr_occupied_ptes >= nr_ptes - max_ptes_none) {
> >> >+ int ret;
> >> >+
> >> >+ collapse_address = address + offset * PAGE_SIZE;
> >> >+ ret = collapse_huge_page(mm, collapse_address,
> >> >referenced,
> >> >+ unmapped, cc, order);
> >> >+ if (ret == SCAN_SUCCEED) {
> >> >+ collapsed += nr_ptes;
> >> >+ continue;
> >> >+ }
> >> >+ }
> >> >+
> >> >+next_order:
> >> >+ if (order > KHUGEPAGED_MIN_MTHP_ORDER) {
> >> >+ const u8 next_order = order - 1;
> >> >+ const u16 mid_offset = offset + (nr_ptes / 2);
> >> >+
> >> >+ collapse_mthp_stack_push(cc, &stack_size,
> >> >mid_offset,
> >> >+ next_order);
> >> >+ collapse_mthp_stack_push(cc, &stack_size, offset,
> >> >+ next_order);
> >> >+ }
> >> >+ }
> >> >+ return collapsed;
> >> >+}
> >> >+
> >> > static enum scan_result collapse_scan_pmd(struct mm_struct *mm,
> >> > struct vm_area_struct *vma, unsigned long start_addr,
> >> > bool *lock_dropped, struct collapse_control *cc)
> >> > {
> >> >- const int max_ptes_none = collapse_max_ptes_none(cc, vma,
> >> >HPAGE_PMD_ORDER);
> >> >+ int max_ptes_none = collapse_max_ptes_none(cc, vma,
> >> >HPAGE_PMD_ORDER);
> >> > const unsigned int max_ptes_shared = collapse_max_ptes_shared(cc,
> >> > HPAGE_PMD_ORDER);
> >> > const unsigned int max_ptes_swap = collapse_max_ptes_swap(cc,
> >> > HPAGE_PMD_ORDER);
> >> >+ enum tva_type tva_flags = cc->is_khugepaged ? TVA_KHUGEPAGED :
> >> >TVA_FORCED_COLLAPSE;
> >> > pmd_t *pmd;
> >> >- pte_t *pte, *_pte;
> >> >- int none_or_zero = 0, shared = 0, referenced = 0;
> >> >+ pte_t *pte, *_pte, pteval;
> >> >+ int i;
> >> >+ int none_or_zero = 0, shared = 0, nr_collapsed = 0, referenced = 0;
> >> > enum scan_result result = SCAN_FAIL;
> >> > struct page *page = NULL;
> >> > struct folio *folio = NULL;
> >> > unsigned long addr;
> >> >+ unsigned long enabled_orders;
> >> > spinlock_t *ptl;
> >> > int node = NUMA_NO_NODE, unmapped = 0;
> >> >
> >> >@@ -1429,8 +1579,19 @@ static enum scan_result collapse_scan_pmd(struct
> >> >mm_struct *mm,
> >> > goto out;
> >> > }
> >> >
> >> >+ bitmap_zero(cc->mthp_bitmap, MAX_PTRS_PER_PTE);
> >> > memset(cc->node_load, 0, sizeof(cc->node_load));
> >> > nodes_clear(cc->alloc_nmask);
> >> >+
> >> >+ enabled_orders = collapse_allowable_orders(vma, vma->vm_flags,
> >> >tva_flags);
> >>
> >> Would it be 0 at this point?
> >
> >If your question relates to the issue you brought up above, then yes,
> >max_ptes_none would be 0 if it's uffd-armed. We must recheck the
> >uffd-armed status before modifying it to 511.
> >
> >>
> >> >+
> >> >+ /*
> >> >+ * If PMD is the only enabled order, enforce max_ptes_none,
> >> >otherwise
> >> >+ * scan all pages to populate the bitmap for mTHP collapse.
> >> >+ */
> >> >+ if (enabled_orders != BIT(HPAGE_PMD_ORDER))
> >> >+ max_ptes_none = KHUGEPAGED_MAX_PTES_LIMIT;
> >> >+
> >> > pte = pte_offset_map_lock(mm, pmd, start_addr, &ptl);
> >> > if (!pte) {
> >> > cc->progress++;
> >> >@@ -1438,11 +1599,13 @@ static enum scan_result collapse_scan_pmd(struct
> >> >mm_struct *mm,
> >> > goto out;
> >> > }
> >> >
> >> >- for (addr = start_addr, _pte = pte; _pte < pte + HPAGE_PMD_NR;
> >> >- _pte++, addr += PAGE_SIZE) {
> >> >+ for (i = 0; i < HPAGE_PMD_NR; i++) {
> >> >+ _pte = pte + i;
> >> >+ addr = start_addr + i * PAGE_SIZE;
> >> >+ pteval = ptep_get(_pte);
> >> >+
> >> > cc->progress++;
> >> >
> >> >- pte_t pteval = ptep_get(_pte);
> >> > if (pte_none_or_zero(pteval)) {
> >> > if (++none_or_zero > max_ptes_none) {
> >> > result = SCAN_EXCEED_NONE_PTE;
> >> >@@ -1522,6 +1685,8 @@ static enum scan_result collapse_scan_pmd(struct
> >> >mm_struct *mm,
> >> > }
> >> > }
> >> >
> >> >+ /* Set bit for occupied pages */
> >> >+ __set_bit(i, cc->mthp_bitmap);
> >> > /*
> >> > * Record which node the original page is from and save
> >> > this
> >> > * information to cc->node_load[].
> >> >@@ -1580,10 +1745,11 @@ static enum scan_result collapse_scan_pmd(struct
> >> >mm_struct *mm,
> >> > if (result == SCAN_SUCCEED) {
> >> > /* collapse_huge_page expects the lock to be dropped
> >> > before calling */
> >> > mmap_read_unlock(mm);
> >> >- result = collapse_huge_page(mm, start_addr, referenced,
> >> >- unmapped, cc, HPAGE_PMD_ORDER);
> >> >+ nr_collapsed = mthp_collapse(mm, start_addr, referenced,
> >> >unmapped,
> >> >+ cc, enabled_orders);
> >> > /* collapse_huge_page will return with the mmap_lock
> >> > released */
> >>
> >> collapse_huge_page will return with mmap_lock released, but mthp_collapse()
> >> may not?
> >
> >We are now releasing the lock before calling mthp_collapse, which
> >subsequently calls collapse_huge_page. Even if `collapse_huge_page` is
> >never called-- say, because enabled_orders is 0 (which should not
> >happen) and all collapse orders are skipped (never calling
> >collapse_huge_page)-- we still return here with the lock dropped.
> >
> >I think this is sound. Let me know if you think differently.
> >
>
> You are right. I missed the lock is released in previous patch.
>
> >Cheers :)
> >-- Nico
> >
> >>
> >> > *lock_dropped = true;
> >> >+ result = nr_collapsed ? SCAN_SUCCEED : SCAN_FAIL;
> >> > }
> >> > out:
> >> > trace_mm_khugepaged_scan_pmd(mm, folio, referenced,
> >> >--
> >> >2.54.0
> >>
> >> --
> >> Wei Yang
> >> Help you, Help me
> >>
>
> --
> Wei Yang
> Help you, Help me
>
