This is a first batch of userfaultfd fixes, with more to come.
These were found by a mix of regular old LLM review and by having an LLM
write a TLA+ model of uffd interactions with PTE bits and other state
and seeing what issues fell out.
Most of these fix places where mm code loses uffd-wp or RWP protection
while moving or migrating an entry. Userspace then either misses writes
and accesses, or gets pages reported that nobody touched.
There is also a hang in UFFDIO_MOVE, and UFFDIO_ZEROPAGE modifying a
file through a private mapping.
Patches 1-2: UFFDIO_ZEROPAGE and UFFDIO_MOVE fixes, cc stable
Patches 3-5: PAGEMAP_SCAN write-protect vs. migration and
device-private entries, patch 3 cc stable
Patch 6: UFFDIO_MOVE of the huge zero page from an RWP source
Patch 7: mremap() losing uffd state when a move fails halfway
Patch 8: migrate_vma losing uffd state of a zero page
Patch 9: UFFDIO_UNREGISTER with a misaligned hugetlb end
A few of the fixes change what userspace sees:
- UFFDIO_ZEROPAGE on a private file mapping no longer fails with
-EEXIST where the file already has a page, on an mm that cannot use
the zero page (patch 1)
- UFFDIO_MOVE returns -EEXIST for any huge destination PMD, not only
for a present THP (patch 2)
- UFFDIO_UNREGISTER with a misaligned hugetlb end fails before
unregistering anything (patch 9)
Patch 6 has little user impact on its own. It is included because,
together with patch 7, it allows removing clear_uffd_wp_pmd().
Patch 7 adds spurious write faults on shmem and hugetlb after mremap():
when dropping uffd protection there, change_protection() makes writable
pages that were never protected read-only. UFFDIO_UNREGISTER and
UFFDIO_WRITEPROTECT already behave this way and lead to the same
spurious faults. This will be fixed in change_protection() in a separate
series.
Based on mm-new.
---
Changes in v2:
- Collect tags (Thanks Kiryl, Lorenzo, Gregory, Alistair!)
- Patch 2: say "softleaf entry" in the commit message, per Lorenzo
- Patch 3: add softleaf_is_writable() and softleaf_mkreadable() to
leafops.h and rename the helpers to say softleaf, per Lorenzo
- Patch 3: add Reported-by and Closes for Breno's report, and cc stable
- Patch 4: explain in the commit message how PAGEMAP_SCAN set the uffd
bit on writable migration entries, per Lorenzo
- Patch 5: use pmd_is_valid_softleaf() rather than open-coding it, per
Lorenzo
- Patch 7: say "write faults" rather than "minor faults" in the commit
message, per Kiryl
- Patch 7: trim the commit message, per Lorenzo and Gregory
- Patch 8: trim the commit message, per Gregory
- Patch 8: Cc the GPU driver maintainers and lists, per Gregory and
Alistair
- Patch 9: use IS_ALIGNED() and a local variable for the VMA's ctx, and
shorten the comment, per Lorenzo
- Drop the three guard region patches, per Lorenzo
- Link to v1:
https://patch.msgid.link/[email protected]
---
Tal Zussman (9):
userfaultfd: use anonymous memory for UFFDIO_ZEROPAGE on private file
mappings
userfaultfd: refuse UFFDIO_MOVE into a non-present huge destination PMD
fs/proc/task_mmu: downgrade writable migration entries in PAGEMAP_SCAN
write-protect
mm/migrate: keep the uffd bit when restoring a writable migration entry
fs/proc/task_mmu: write-protect device-private THPs in PAGEMAP_SCAN
mm/huge_memory: build the moved huge zero PMD from the destination VMA
mm/mremap: resolve uffd state when the context is dropped, not during the
move
mm/migrate_device: do not migrate a uffd-protected zero page
mm/userfaultfd: check hugetlb end alignment in UFFDIO_UNREGISTER pre-scan
fs/proc/task_mmu.c | 42 +++++++++++++++++++++++++++---
include/linux/leafops.h | 51 ++++++++++++++++++++++++++++++++++++
mm/huge_memory.c | 31 +++-------------------
mm/hugetlb.c | 27 +------------------
mm/migrate.c | 2 +-
mm/migrate_device.c | 23 +++++++++++++++--
mm/mremap.c | 46 +--------------------------------
mm/userfaultfd.c | 69 ++++++++++++++++++++++++-------------------------
8 files changed, 150 insertions(+), 141 deletions(-)
---
base-commit: 67f3b04b277373438f17300e056306adbd9edf7b
change-id: 20261106-uffd-state-fixes-d2111e2abd48
Best regards,
--
Tal Zussman <[email protected]>