On Tue, May 26, 2026 at 01:37:04PM +0200, Yiannis Nikolakopoulos wrote: > From: Alirad Malek <[email protected]> > > Memory demoted to a lower tier is assumed to be cold and most likely out of > the CPU's last level cache. Additionally, in certain demotion targets (e.g. > CXL devices with compressed memory) the bandwidth can be negatively > impacted by the eviction patterns of the last level cache when standard > memcpy is used. When the feature is enabled, use the > MIGRATE_ASYNC_NON_TEMPORAL_STORES flag in demotions to trigger the folio > copy path using non-temporal stores. > > Signed-off-by: Alirad Malek <[email protected]> > Co-developed-by: Yiannis Nikolakopoulos <[email protected]> > Signed-off-by: Yiannis Nikolakopoulos <[email protected]> > --- > mm/Kconfig | 8 ++++++++ > mm/migrate.c | 9 ++++++++- > 2 files changed, 16 insertions(+), 1 deletion(-) > > diff --git a/mm/Kconfig b/mm/Kconfig > index ebd8ea353687..4b7a75b57f6e 100644 > --- a/mm/Kconfig > +++ b/mm/Kconfig > @@ -645,6 +645,14 @@ config MIGRATION > pages as migration can relocate pages to satisfy a huge page > allocation instead of reclaiming. > > +config DEMOTION_WITH_NON_TEMPORAL_STORES > + bool "Use non-temporal stores for demotion" > + default n > + depends on MIGRATION > + help > + Enable non-temporal stores when migrating pages due to demotion. > + If disabled, demotion uses regular migration copy paths. > +
Do we actually need this config flag or should we just default to this (if the arch supports NT stores)? > config DEVICE_MIGRATION > def_bool MIGRATION && ZONE_DEVICE > > diff --git a/mm/migrate.c b/mm/migrate.c > index ff6cf50e7b0b..368d40dc8772 100644 > --- a/mm/migrate.c > +++ b/mm/migrate.c > @@ -862,7 +862,10 @@ static int __migrate_folio(struct address_space > *mapping, struct folio *dst, > if (folio_ref_count(src) != expected_count) > return -EAGAIN; > > - rc = folio_mc_copy(dst, src); > + if (mode == MIGRATE_ASYNC_NON_TEMPORAL_STORES) > + rc = folio_mc_copy_nt(dst, src); > + else > + rc = folio_mc_copy(dst, src); > if (unlikely(rc)) > return rc; > > @@ -2081,6 +2084,10 @@ int migrate_pages(struct list_head *from, new_folio_t > get_new_folio, > LIST_HEAD(split_folios); > struct migrate_pages_stats stats; > > + if (IS_ENABLED(CONFIG_DEMOTION_WITH_NON_TEMPORAL_STORES) && > + reason == MR_DEMOTION && mode == MIGRATE_ASYNC) > + mode = MIGRATE_ASYNC_NON_TEMPORAL_STORES; > + > trace_mm_migrate_pages_start(mode, reason); > > memset(&stats, 0, sizeof(stats)); > > -- > 2.43.0 >
