On Sun Aug 16, 2026 at 6:45 PM EDT, Kiryl Shutsemau wrote: > From: "Kiryl Shutsemau (Meta)" <[email protected]> > > Yes, I know, this is a lot of changes. But I'm happy with the overall state > of the patchset and the only reason I tag it as RFC is that it is tricky > to get 57 patches upstream. > > I wanted to give a view of the end state first. I will suggest a possible > way to split it below. > > I would appreciate any feedback. > > TL;DR > ===== > > This replaces khugepaged's anonymous collapse with an engine that > can collapse sub-PMD ranges. It is built around migration entries and > frozen folios instead of heavy locking and isolation, aiming for better > scalability and less disruption to the workload being collapsed. > > Why > === > > mTHP collapse landed in khugepaged in 7.2 and I was glad to see it. We > at Meta run arm64 with 64K base pages, where a PMD is 512M: PMD-order THP > is of limited use at that size, and mTHP is exactly what we want. > > It turned out not to help us. > > khugepaged only ever looks at PMD-aligned windows, and it is not an easy > limitation to lift. > > Fixing the alignment is a one-line change, but what it feeds assumes the > PMD everywhere that matters: collapse_huge_page() clears and flushes the > whole PMD whatever order it is collapsing, installs a PMD leaf because > that is the only thing it can produce, and keeps everyone out with > mmap_write_lock, anon_vma_lock_write() and an IPI broadcast while it > does. > > Which is why hugepage_vma_revalidate() demands that the VMA span the > whole PMD even for an mTHP order -- "we'd need to lock all VMAs in the > PMD range to support this", as the comment there puts it. A PMD-granular > operation is only safe when one VMA owns the PMD, and that is exactly the > restriction in the way. The alignment is the symptom; the PMD is the > design. > > So both roots have to go.
I agree that khugepaged is designed for PMD-aligned collapse and this is a limitation we want to get rid of. It is great you are looking at them. > > Design > ====== > > The old mechanism holds the address space still because it has nothing > else stopping the sources from moving under the copy. The new engine > makes the sources themselves inert instead, with the two barriers > migration already uses, raised in that order: > > 1. migration entries replace the source PTEs. Faults and GUP-slow > now wait on the source folio's lock, which is taken before the > first entry becomes visible. > 2. the source folio's refcount is frozen to its expected value. > GUP-fast, pfn walkers, reclaim, compaction and memory-failure all > fail folio_try_get() and back off. > > Between the two, nothing can reach a source, so the copy runs with no > lock held at all -- and the address space is left alone while it does. > > What that removes from every collapse path: > > mmap_write_lock -> mmap_read > anon_vma_lock_write() -> nothing: an rmap walk needs the folio > locked, and the engine holds that lock > from freeze to putback > tlb_remove_table_sync_one() -> nothing: one ranged flush per round > LRU isolation -> nothing: sources are inert in place I remember we were discussing using migration entry and the issue with mmap_write_lock() in the context of in-place THP promotion and the conclusion was that because MADV_DONTNEED (maybe MADV_REMOVE or MADV_PAGEOUT) works on page table and does not change VMAs, mmap_write_lock() is needed to prevent things being changed under khugepaged. Anything different in normal khugepaged collapse process, so that it is OK to use mmap_read_lock? Let me know if I misremember it. Thanks. -- Best Regards, Yan, Zi
