This series continues the non-temporal demotion work originally posted by Yiannis Nikolakopoulos [1].
Motivation ========== Memory tier demotion moves cold folios from local DRAM to slower memory tiers, such as CXL or remote NUMA nodes. Standard store instructions force the CPU to allocate cache lines for cold data that is unlikely to be referenced in the near term. Using non-temporal stores and prefetching on x86 bypasses CPU cache allocation, protecting active workload working sets and improving demotion throughput. Changes in v3 ============= - Unified MCE-safe NT copy (addressing v2 Open #1): Implement copy_mc_page_nt() in x86 assembly to provide machine-check safe copying using non-temporal stores (movnti) and prefetchnta. - Architecture abstraction (addressing v2 Open #2): v2 relied on memcpy_flushcache(), which lacks non-temporal stores on architectures like arm64. RFC v3 replaces this with a dedicated __HAVE_ARCH_COPY_PAGE_NT interface and generic fallback, enabling architectures to plug in native non-temporal routines independently. - Memory ordering: Add a wmb() barrier at the end of folio_mc_copy_nt() to ensure weakly ordered non-temporal stores are drained before page tables are updated. Discussions =========== 1. Migration mode vs. migration control refactoring: Currently, this series introduces MIGRATE_ASYNC_NONTEMPORAL into enum migrate_mode. In parallel, Shivank Garg's migration refactoring series [2] consolidates migration arguments into migrate_control. Once that lands, demotion can simply rely on the migration reason instead of adding a dedicated migration mode and helper. 2. Large folio copy optimization: Multi-page folios (such as 2MiB THP) are supported by iterating over constituent 4KiB pages via copy_mc_page_nt(). Once the batch page copy series [3] lands, copy_mc_page_nt() can be extended to copy an entire multi-page folio in a single invocation, eliminating per-4KiB loop overhead and keeping the prefetchnta stream uninterrupted. Thanks to Yiannis Nikolakopoulos, Alirad Malek, and Dimitrios Palyvos-Giannas for their original work, feedback, and collaboration. References ========== [1] https://lore.kernel.org/linux-mm/[email protected]/ [2] https://lore.kernel.org/linux-mm/[email protected]/ [3] https://lore.kernel.org/linux-mm/[email protected]/ Signed-off-by: Huan Nguyen <[email protected]> Huan Nguyen (3): mm, x86: support copying a folio using non-temporal stores mm/migrate: introduce async non-temporal migration mode mm/vmscan: demote folios using non-temporal stores arch/x86/include/asm/page_64.h | 2 ++ arch/x86/lib/copy_page_64.S | 65 ++++++++++++++++++++++++++++++++++ fs/nfs/write.c | 2 +- include/linux/highmem.h | 21 +++++++++++ include/linux/migrate.h | 5 +++ include/linux/migrate_mode.h | 1 + include/linux/mm.h | 1 + include/trace/events/migrate.h | 1 + mm/compaction.c | 18 +++++----- mm/migrate.c | 29 +++++++++------ mm/util.c | 29 +++++++++++++++ mm/vmscan.c | 2 +- 12 files changed, 154 insertions(+), 22 deletions(-) base-commit: 05441d697f7f9a927264f4058cbbc6a03dfcac9d -- 2.56.0.398.g882cbad275-goog
