AI and code-execution services increasingly run user workloads in short-lived sandbox VMs. These VMs are usually small, densely packed, and backed by sparse disk images, so both memory footprint and VM startup time matter directly for consolidation density.
Using virtio-pmem with FS-DAX for the guest filesystem is one way to reduce that footprint. It avoids keeping a second copy of file data in the guest page cache, and turns that cache into host-side file cache that can be accounted and reclaimed with the host's global memory view. Host-side page-cache reclaim is also more direct than asking the guest to drop its page cache, waiting for the freed pages to be reported back, and then reclaiming the memory from the host side. This makes higher overcommit more practical for dense sandbox deployments. The remaining problem is that virtio-pmem with FS-DAX still pays the guest-side ZONE_DEVICE metadata cost up front. The guest registers the whole pmem aperture and allocates and initializes struct page metadata for every advertised PFN. The vmemmap overhead is about 1.56% of the pmem device size, and initializing all of those struct pages can noticeably slow down startup for lightweight VMs. Much of that private metadata is unnecessary in common sandbox setups. Holes in a sparse rootfs image have no host storage allocated, but the guest still allocates struct page metadata for the corresponding pmem PFNs. Also, many filesystem workloads access files through read(2) and write(2) rather than mmap(2); those DAX blocks are copied through the kernel and do not need to be inserted into userspace page tables, so they do not need private per-PFN struct page state either. The key observation is that when sizeof(struct page) is a power of two, each PAGE_SIZE vmemmap page contains a naturally aligned, repeatable group of struct page slots. For an FS-DAX pmem range at device registration time, those slots only need the same ZONE_DEVICE and dev_pagemap state before a PFN is exposed to userspace. The kernel mainly needs the vmemmap to resolve the PFN back to a valid device page and its dev_pagemap. It does not need independent writable per-PFN state for ranges that are only accessed through the DAX direct-access path, or for ranges that are never accessed at all. This series takes advantage of that split by separating vmemmap population from private metadata allocation. Device registration still gives every advertised PFN a valid struct page representation, but the vmemmap mappings initially point at a shared read-only vmemmap page containing the common ZONE_DEVICE state. This avoids allocating and initializing private metadata for PFNs that may never need it. Private writable metadata is materialized only when it becomes necessary: before a DAX fault inserts the PFN into a userspace mapping. At that point the PFN can participate in the normal page-based MM paths, so the shared vmemmap page is replaced with a private writable copy. PTE faults materialize the corresponding vmemmap page, and PMD faults materialize the whole PMD-sized metadata range. PFNs that are never faulted continue to use the shared vmemmap page, avoiding both the memory cost and the struct page initialization work. A natural follow-up is to make this optimization reversible. Once a DAX entry is removed from the address_space, and after all mappings, references and pins that require private metadata are gone, the corresponding vmemmap page could be remapped back to the shared read-only vmemmap page and the private metadata page could be freed. With that, the guest-side struct page overhead would track the live DAX working set rather than the full advertised pmem device size. Patch 1 renames the architecture opt-in for runtime vmemmap remapping so it describes the generic capability instead of the HugeTLB user. Patch 2 avoids touching PG_hwpoison state for clean pmem pages. This keeps the normal clean-I/O path compatible with read-only shared metadata. Patch 3 adds the shared read-only FS-DAX vmemmap infrastructure and the helper that materializes a private metadata page on demand. Patch 4 opts pmem FS-DAX mappings into the new mode and materializes the metadata before DAX inserts a PFN into a userspace mapping. This does not change FS-DAX data path semantics. It only changes when private struct page metadata is allocated for pmem FS-DAX PFNs. Muchun Song (4): mm: generalize vmemmap remap architecture support nvdimm/pmem: avoid HWPoison flag updates for clean pages mm: add shared read-only vmemmap support for FS-DAX fsdax: materialize pmem vmemmap metadata on faults arch/loongarch/Kconfig | 2 +- arch/riscv/Kconfig | 2 +- arch/x86/Kconfig | 2 +- drivers/nvdimm/pmem.c | 3 +- fs/Kconfig | 2 +- fs/dax.c | 5 ++++ include/linux/memremap.h | 11 ++++++- mm/Kconfig | 6 ++-- mm/memremap.c | 38 ++++++++++++++++++++++-- mm/mm_init.c | 11 +++++++ mm/sparse-vmemmap.c | 64 +++++++++++++++++++++++++++++++++++++--- 11 files changed, 131 insertions(+), 15 deletions(-) base-commit: 32b6ef9a5d0eca44f9cd91f52f4faa89f145a0de -- 2.54.0
