From: John Groves <[email protected]>
This is v13 of famfs, as a standalone file system. v12 is at [7].
Most of this cover letter repeats v12. If you have read it, skip to "Changes
since v12" below and to the per-patch changelogs under the "---" line of each
patch. The history goes back to 2023, and is briefly covered in the v12
cover.
The most important thing to know about famfs is that it CANNOT be used as a
general purpose file system. It is for enabling file-based byte-level access
(including direct mmap) to very large (e.g. 100TB) shared/disaggregated
memory appliances - which have become available during this long process,
and which are in need of Linux support.
So famfs cannot be used by anybody who doesn't know why they need it. Making
famfs standalone means it can't affect users who don't use it. Micron, as
well as other memory companies, need it, and do not intend to abandon it -
but imagine if we did: fs/famfs/ should be removed if the memory companies
can't be bothered to maintain it.
This code base has been in active CI and used globally by early adopters and
testers of disaggregated memory since 2024. I believe it is solid.
Why standalone (short version)
famfs was introduced at LPC 2023 and LSFMM 2024 [1] as a standalone file
system, then spent close to a year being ported to fuse/libfuse. After
maintaining famfs in both forms, I have concluded it makes more sense as a
standalone file system: fuse adds complexity to famfs (more complex, less
adaptable, less performant - and famfs files are memory, so access must run
at memory speed), while famfs adds complexity to fuse that is unlikely to see
constructive re-use. The full history and argument are in the v12 cover
letter [7].
Changes since v12
-----------------
- Patch 01 split, per Alison Schofield: the removal of the now-unused
dax_dev_get() export is a standalone DAX cleanup that goes via the DAX tree
and is posted separately [8]. Patch 01 of this series is reduced to adding
the non-allocating lookup helper dax_dev_find().
- Darrick Wong's review comments are addressed across the series: the
kill_*_super() rationale in the mount patch, the page_mkwrite/pfn_mkwrite
write-fault comment, dropping the operation flags from iomap->flags in the
resolver, and more.
- A number of findings from the Sashiko review bot are fixed:
* mprotect() could bypass the FAMFS_OPT_WRITE gate (the check looked at
VM_WRITE only, not VM_MAYWRITE);
* rename() over an existing target bypassed the FAMFS_OPT_UNLINK /
FAMFS_OPT_RMDIR policy that unlink()/rmdir() enforce;
* splice()/sendfile() returned zeroes on these DAX files
(filemap_splice_read -> copy_splice_read);
* create/mkdir/mknod updated the child inode's timestamps instead of the
parent directory's;
* .map_pages (filemap_map_pages) was a no-op on DAX and is removed;
* mount-parameter and module-namespace hygiene (reject unknown mount
options; prefix the non-static lookup_daxdev()).
A few Sashiko findings were considered and declined with rationale - for
example generic_write_sync(), which is inert for a filesystem with no page
cache, no writeback, and a noop_fsync ->fsync. Those are noted under the
relevant patch.
- famfs now accepts 4 KiB extent alignment in addition to 2 MiB: the
allocation unit may be 4 KiB or 2 MiB. 2 MiB-aligned extents still map with
huge pages; 4 KiB-granular extents map with PTEs (the DAX fault path already
falls back PMD -> PTE).
- MAP_SYNC is now supported (FOP_MMAP_SYNC). famfs fmap metadata is immutable
after MAP_CREATE, so the MAP_SYNC durability guarantee is met with no
metadata sync required.
- famfs is explicitly 64-bit only now (depends on 64BIT); it targets
CXL/fabric-attached memory, which does not exist on 32-bit systems.
Famfs Overview
Famfs exposes sharable disaggregated memory as a file system. Famfs consumes
shared memory from [usually shared memory] dax devices, and provides
memory-mappable files that map directly to the memory - no page cache
involvement. Famfs differs from conventional file systems in fs-dax mode,
in that it handles in-memory metadata in a sharable way (which begins with
never caching dirty shared metadata). So a famfs file system can be mounted
from multiple nodes, provided they have access to the memory.
The key performance requirement is that famfs must resolve mapping faults
with minimal overhead. This is achieved by fully caching the file-to-devdax
metadata for all active files.
Famfs remains the first fs-dax file system that is backed by devdax rather
than pmem in fs-dax mode (hence the need for the new dax mode).
Famfs depends on the 'fsdev' dax mode which landed in 7.1 - it will only run
with an fsdev-mode (aka famfs-mode) daxdev as its backing device(s).
The famfs user space can be found at [6].
[1] https://lwn.net/Articles/983105/ (Famfs at LSFMM 2024)
[2] https://lwn.net/Articles/1020170/ (Famfs at LSFMM 2025, with patch link)
[3] https://lwn.net/Articles/1068686/ (LWN coverage of the patch thread)
[4] https://lwn.net/Articles/1082687/ (Famfs at LSFMM 2026)
[5]
https://lore.kernel.org/linux-fsdevel/0100019f7d9fbe81-6cb16662-2522-47ea-a152-fab0ee3d9b35-000...@email.amazonses.com/#b
[6] https://famfs.org
[7]
https://lore.kernel.org/linux-fsdevel/20260806053409.GL3560084@frogsfrogsfrogs/T/#t
(v12)
[8]
https://lore.kernel.org/linux-cxl/0100019fd478eb93-a6bd48e4-b5f1-4871-93d4-582802956e6c-000...@email.amazonses.com/T/#u
(Partial dax patch)
John Groves (12):
dax: replace exported dax_dev_get() with non-allocating dax_dev_find()
famfs: Module operations, fs_context, and mount
famfs: Add daxdev table and dax notify_failure support
famfs: Introduce inode_operations and super_operations
famfs: Introduce file_operations read/write
famfs: Introduce mmap and VM fault handling
famfs: MAP_CREATE ioctl and fmap ingest (ABI 44)
famfs: iomap_begin and file-to-dax offset resolution
famfs: Register secondary daxdevs by path (FAMFSIOC_DAXDEV_OPEN)
famfs: Add runtime operation-permission (opts) framework
famfs: Report device capacity via statfs so df works
famfs: Add documentation
Documentation/filesystems/famfs.rst | 142 +++
Documentation/filesystems/index.rst | 1 +
.../userspace-api/ioctl/ioctl-number.rst | 1 +
MAINTAINERS | 8 +
drivers/dax/fsdev.c | 19 +
drivers/dax/super.c | 38 +-
fs/Kconfig | 2 +
fs/Makefile | 1 +
fs/famfs/Kconfig | 12 +
fs/famfs/Makefile | 5 +
fs/famfs/famfs_file.c | 1007 +++++++++++++++++
fs/famfs/famfs_inode.c | 869 ++++++++++++++
fs/famfs/famfs_internal.h | 162 +++
fs/namei.c | 1 +
fs/super.c | 7 +
include/linux/dax.h | 7 +-
include/linux/fs.h | 1 +
include/uapi/linux/famfs_ioctl.h | 160 +++
include/uapi/linux/magic.h | 1 +
19 files changed, 2441 insertions(+), 3 deletions(-)
create mode 100644 Documentation/filesystems/famfs.rst
create mode 100644 fs/famfs/Kconfig
create mode 100644 fs/famfs/Makefile
create mode 100644 fs/famfs/famfs_file.c
create mode 100644 fs/famfs/famfs_inode.c
create mode 100644 fs/famfs/famfs_internal.h
create mode 100644 include/uapi/linux/famfs_ioctl.h
--
2.53.0
