casaroli opened a new pull request, #19536:
URL: https://github.com/apache/nuttx/pull/19536
## Summary
Adds **XIPFS**, a writable file system that can serve execute-in-place
mappings, so a loadable module can arrive on the board at run time and still
be executed out of flash rather than copied into RAM.
Today ROMFS is the only in-tree file system able to serve the XIP mappings
the NXFLAT loader needs (`Documentation/components/nxflat.rst` says so
explicitly). A ROMFS image is built on the host and is read-only, so on a
NOMMU target with memory-mapped NOR there is no way to download a module and
run it in place. XIPFS closes that gap; this PR also updates the NXFLAT page,
which no longer has only one answer to point at.
How it works:
* **Contiguous extents.** Each file is one physically contiguous,
erase-block-aligned run, so `mmap()` resolves to `flash_base + offset` and
the loader executes the file where it already lies. This needs the MTD
driver underneath to answer `BIOC_XIPBASE`.
* **Write once.** A file is created, its size declared, written sequentially,
closed, and is immutable until deleted — the whole life cycle of a
downloaded module. That is what licenses reserving the exact extent up
front, which makes intra-file fragmentation impossible by construction.
Random writes, appends and truncation of a written file are refused.
* **Manual, best-effort, pin-aware defragmentation.** Allocation returns
`-ENOSPC` and never compacts on its own; the caller decides, via
`XIPFSIOC_DEFRAG` on a descriptor for the mountpoint directory (using the
`ioctldir` method added in #19512). A pass is a loop of atomic
single-extent relocations, so every stop point is a consistent layout, and
it reports the largest contiguous run it achieved — which is what tells the
caller whether a retry will fit.
* **Power-safe metadata.** A generation is committed as write the body, then
write the header that carries the sequence number and CRC. That single
header program is the commit point, so a torn write costs the interrupted
operation and nothing else; mount selects the last fully valid generation.
* **Pinning.** A mapping that aliases flash takes a pin on the extent, not on
the descriptor, so N running instances of one module hold N pins and the
extent becomes movable only when the last goes. Defragmentation skips
pinned extents, which is what stops it relocating code that is executing.
Pins are released by `munmap()` **or** by task teardown, so a module that
faults without unmapping does not leak one.
* **Directories** are records in the same metadata generation as files, never
objects in the data region. That placement is deliberate: `mkdir` and
`rmdir` add or remove a record and commit one generation, exactly as create
and unlink do, so there is no multi-object update to journal and no orphan
to collect at mount. An empty directory therefore exists and survives a
remount, and costs one entry out of the volume's fixed supply and no flash
blocks. Mount rebuilds the tree and verifies it *is* a tree — unique
identities, unique names per directory, every parent a live directory, and
parents reaching the root — because a cycle on the medium would otherwise
hang a path walk rather than merely answer wrongly.
`MAP_XIP_STRICT` is added to `sys/mman.h`: with it a mapping either resolves
in place or fails with `-ENXIO`, never silently falling back to the
`CONFIG_FS_RAMMAP` copy. A module loader must not get a RAM copy by accident,
since that defeats the entire point.
The applications that go with this — the `xipfs` compaction command, the test
suite whose results are quoted below, and the NXFLAT execute-in-place demo —
are in the companion PR:
**Companion PR: apache/nuttx-apps (link added below once opened)**
Both are needed together: the test suite and demo in that PR are what
exercise
this one, and its Kconfig options depend on `CONFIG_FS_XIPFS` from here.
## Impact
* **New feature, off by default.** `CONFIG_FS_XIPFS` defaults to `n` and
depends on `MTD` and `!DISABLE_MOUNTPOINT`. Nothing changes for existing
configurations.
* **Users:** a second answer to "how do I execute a module in place", and the
first writable one. The `Limitations` section of the docs is explicit about
what XIPFS is not: no random writes, no appends, no growth, no rename, and
a
file occupies a whole number of erase blocks.
* **Existing code touched:**
* `include/sys/mman.h` — one new flag, `MAP_XIP_STRICT` (bit 27).
* `include/sys/statfs.h` and `fs/mount/fs_gettype.c` — a magic and a case
so
`df` names the file system instead of printing "Unrecognized".
* `fs/mount/fs_mount.c` — registers `xipfs` among the MTD-backed file
systems.
* `Documentation/components/nxflat.rst` — the "ROMFS is the only
XIP-capable
file system" limitation is now two, and says which one is writable.
* The rp23xx board `Make.defs` files gain `MKNXFLAT`/`LDNXFLAT`, which
every
other ARM board with NXFLAT support already names.
* **Hardware:** no change to any driver. The RP2350 flash MTD driver this was
developed against is already upstream (#19531).
* **Build:** new files only under `fs/xipfs/`; the fs `CMakeLists.txt` picks
the directory up automatically.
* **Security/compatibility:** the on-media format carries a version and mount
refuses anything else with `-EFTYPE`, which `autoformat` turns into a
reformat where the mount asked for it.
## Testing
Host: macOS 15 (arm64), `arm-none-eabi-gcc 14.2.rel1`.
Targets:
* `sim:xipfs` — new configuration, XIPFS on `rammtd` (which answers
`BIOC_XIPBASE`, so the in-place path is exercised without flash).
* `pimoroni-pico-2-plus:xipfs` — new configuration, XIPFS on the real QSPI
NOR via `/dev/rpflash`, i.e. genuine erases and programs throughout.
Both were verified with the test suite from the companion PR. Every run below
was bracketed by a `uname -a` check of the git revision and board name,
before
the first command and after the last, so the logs cannot be from a different
build.
### `pimoroni-pico-2-plus:xipfs` — full suite, real flash
```
[identity ok before] rev 0e9e189444, board pimoroni-pico-2-plus
-- basic VFS -- 12 checks
-- write-once enforcement -- 5 checks
-- strict mapping semantics -- 3 checks
-- XIP mapping -- 9 checks
-- concurrent mappings -- 4 checks
-- cross-task pin refcounting -- 4 checks
-- pin release on task exit -- 3 checks
-- directories -- 28 checks
-- defragmentation -- 9 checks
-- power loss during create (clean) --
PASS filesystem stays consistent across every injection point (clean)
-- power loss during create (torn) --
PASS filesystem stays consistent across every injection point (torn)
-- power loss during unlink --
swept 4 injection points, covering the whole unlink
-- power loss during mkdir and rmdir --
PASS mkdir and rmdir stay consistent across every injection point
-- power loss during defrag --
swept 110 injection points, covering every operation of the whole pass
==== 90 passed, 0 failed ====
[identity ok after] rev 0e9e189444, board pimoroni-pico-2-plus
```
The power-loss sections fail the Nth flash write or erase, remount, and
assert
the volume is consistent and every committed file byte-for-byte intact — with
the failing operation left *torn* (half a page programmed, half a sector
erased) as well as cleanly refused, because a torn generation is what forces
the mount-time CRC to do real work.
### `sim:xipfs`
```
==== 90 passed, 0 failed ====
```
### Execute-in-place, end to end on hardware
An NXFLAT module written into the volume at run time and run twice
concurrently:
```
staged 268 bytes to /mnt/xipfs/xipmod
extent: block 97 x1, size 268, flash addr 0x10161000
[while running] pin count on the shared text = 2
instance seed 1: text 0x10161070, stack 0x20006980, sum 2080 -- data
private and intact
instance seed 2: text 0x10161070, stack 0x20007588, sum 4160 -- data
private and intact
```
Both instances report the same text address — inside the flash window, equal
to where the file lies on the media — and different stacks.
### Defragmentation on real NOR
```
[3] requesting a 209-block (836 KB) contiguous file
result: FAILED -ENOSPC (free space is not contiguous)
[4] compacting...
elapsed : 1230 ms
extents relocated: 27
blocks reclaimed : 27
largest free run : 913408 bytes (223 blocks)
stop reason : 0 (done)
[5] retrying, sized to the reported largest run (223 blocks)
result: OK -- allocation now fits
[6] verifying every relocated file byte-for-byte
all survivors intact
[7] remounting to prove it is durable
all files intact after remount
```
### Directories from the shell
```
nsh> mkdir /mnt/xipfs/bin
nsh> mkdir /mnt/xipfs/bin/sub
nsh> echo deep > /mnt/xipfs/bin/sub/deep
nsh> mkdir /mnt/xipfs/empty
nsh> ls -lR /mnt/xipfs
/mnt/xipfs:
dr-xr-xr-x 0 empty/
dr-xr-xr-x 0 bin/
/mnt/xipfs/empty:
/mnt/xipfs/bin:
-r--r--r-- 6 greet
dr-xr-xr-x 0 sub/
/mnt/xipfs/bin/sub:
-r--r--r-- 5 deep
nsh> rmdir /mnt/xipfs/bin
nsh: rmdir: rmdir failed: 39 <- ENOTEMPTY
nsh> df -h
Filesystem Size Used Available Mounted on
xipfs 1000K 4K 996K /mnt/xipfs
```
`tools/checkpatch.sh -c -u -m -g` passes on every commit in the branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
