casaroli opened a new pull request, #21613: URL: https://github.com/apache/nuttx/pull/21613
## Summary POSIX says: `open()` of a directory with O_RDONLY gives a directory descriptor, `open()` of a directory with O_WRONLY or O_RDWR fails with EISDIR, and `open()` of a file with O_DIRECTORY fails with ENOTDIR. NuttX did not do this on the pseudo-fs, hostfs, rpmsgfs and v9fs. This PR fixes each case in its own commit: - **fs/vfs: Fail an open of a file with O_DIRECTORY.** `open(path, O_RDONLY | O_DIRECTORY)`, which `opendir()` uses, did not check that the path is a directory. A regular file on v9fs and a driver such as `/dev/null` gave a directory descriptor. The VFS now checks the path with `file_checkdir()` for every access mode, before the O_RDONLY case calls `dir_allocate()` (the O_WRONLY/O_RDWR case already made this check). The check is in the VFS because several `opendir` methods accept a file (the pseudo-fs, spiffs, zipfs, v9fs), and hostfs and rpmsgfs fail with ENOENT. A file system without a `stat` method cannot tell, so its `opendir` method still decides. - **fs/vfs: Open a pseudo-fs directory without O_DIRECTORY as a directory.** `open()` of a pseudo-fs directory such as `/dev` or `/` failed with ENXIO for every access mode. `file_vopen()` now uses `-EISDIR` for it, as the file systems do. The existing code then opens the directory with the directory operations for O_RDONLY and returns EISDIR for the other access modes. - **fs/hostfs: Open a directory without O_DIRECTORY as a directory.** The host opened the directory, and the descriptor had the hostfs file operations: `readdir()` after `fdopendir()` failed with EISDIR and `openat()` failed with ENOENT. For O_RDONLY, `hostfs_open()` now asks the host for the type (`host_fstat()`) and returns `-EISDIR` for a directory. - **fs/rpmsgfs: Open a directory without O_DIRECTORY as a directory.** The server opened the directory and the client used the rpmsgfs file operations: `readdir()` read the server's `struct dirent` through read messages, and F_GETPATH gave the server's path, so `openat()` used a wrong path. The server now returns EISDIR when an O_RDONLY open gives a directory (a local `file_fstat()`, no new message). The client VFS then uses the rpmsgfs directory messages. - **fs/v9fs: Open a directory without O_DIRECTORY as a directory.** The descriptor had the v9fs file operations: the server refused the Tread of the directory, so `readdir()` failed, and O_WRONLY/O_RDWR did not fail. Rlopen gives the type: `v9fs_vfs_open()` now returns `-EISDIR` for a directory. - **fs/rpmsgfs: Close the remote file when the O_APPEND seek fails.** In the same open path: when the seek to the end failed (for example ENOSYS on a character driver of the server), `rpmsgfs_open()` freed its file structure but left the file open on the server. Each such open leaked a file on the server. - **Documentation/filesystem: Describe the open() of a directory.** The `open` method returns `-EISDIR` for a directory; for O_DIRECTORY the VFS checks the path with `stat()` before `opendir()`. ## Impact Each line is a change that a program sees. The code is the same in flat, protected and kernel builds, on every architecture. - `open(file, O_RDONLY | O_DIRECTORY)` and `opendir(file)` fail with ENOTDIR on every file system with a `stat` method. Before, v9fs gave a directory descriptor, and hostfs and rpmsgfs failed with ENOENT. tmpfs already gave ENOTDIR. - `open(driver, O_RDONLY | O_DIRECTORY)` and `opendir(driver)`, for example `/dev/null`, fail with ENOTDIR. Before, they gave a directory descriptor. - Each `open()` with O_DIRECTORY on a mounted file system, so each `opendir()`, calls the `stat` method once before the `opendir` method. - `open()` of a pseudo-fs directory: O_RDONLY gives a directory descriptor, O_WRONLY and O_RDWR fail with EISDIR. Before, all three failed with ENXIO. Other inode types (block drivers without BCH, named semaphores, message queues, shared memory) still fail with ENXIO. - `open(dir, O_RDONLY)` on hostfs, v9fs and rpmsgfs gives a directory descriptor: `fdopendir()`, `readdir()` and `openat()` work. Before, `openat()` failed with ENOENT, and `readdir()` failed on hostfs (EISDIR) and on v9fs (EOPNOTSUPP). - `open(dir, O_WRONLY)` and `open(dir, O_RDWR)` on v9fs fail with EISDIR. Before, they succeeded. - `read()` on the new directory descriptors gives `struct dirent` records, as `read()` on a tmpfs directory descriptor does today (EINVAL for a buffer smaller than `struct dirent`). For example, `cat /dev` now prints the entries instead of "open failed". This PR does not change `read()` of a directory. - hostfs makes one more host call (`host_fstat()`) for each O_RDONLY open. The semihosting hosts report a regular file there, so their behaviour does not change. - rpmsgfs: the protocol does not change. A new server with an old client and an old server with a new client work (tested, see below). - rpmsgfs: an O_APPEND open whose seek to the end fails still fails with the error of the seek, but the file on the server is now closed. Before, each such open left a file open on the server: after 255 opens of the server console, the next opens failed with EMFILE. Potential breaking change: the results above change in the flat build too. They are POSIX fixes. In nuttx and nuttx-apps, no caller tests for ENXIO from an `open()` of a directory, and each `open()` with O_DIRECTORY is on a directory (`opendir()`, the mount points in xipfs and nxflatxip, the directory list of the web server). `Documentation/os/filesystem/index.rst` now describes the `open` and `opendir` contract. ## Testing The test is `testing/fs/diropen` (apache/nuttx-apps#NNNN): for each directory it checks `open()` with O_WRONLY, O_RDWR and O_RDONLY, `fstat()`, `openat()`, `fdopendir()`/`readdir()` and O_DIRECTORY; for each file `open()` with O_DIRECTORY and `opendir()`. It runs on the pseudo-fs (`/dev`, `/` and a new directory), on a tmpfs, and in the directory given as an argument. nuttx master vs this PR, pass / fail: | Configuration | Argument | master | this PR | ostest | |---|---|---|---|---| | sim:nsh (flat, macOS host), make | hostfs `/host` | 23 / 15 | 47 / 0 | status 0 / 0 | | qemu-armv8a:nsh (flat), HVF | v9fs `/system` (9p share) | 21 / 17 | 47 / 0 | status 0 / 0 | | qemu-armv8a:knsh (kernel), HVF | v9fs `/system` | 21 / 17 | 47 / 0 | status 0 / 0 | | mps2-an500:knsh (protected), QEMU TCG | none | 15 / 11 | 35 / 0 | not run | | qemu-armv8a:rpserver + rpproxy over ivshmem, QEMU TCG on a Linux host | rpmsgfs `/srv` (a tmpfs of the server) | 24 / 14 | 47 / 0 | not run | The failures on master, by commit: - O_DIRECTORY: `/dev/null` opens as a directory; a file on v9fs opens as a directory; a file on hostfs and rpmsgfs gives ENOENT. - pseudo-fs: `/dev`, `/` and the new directory give ENXIO for each access mode. - hostfs: `openat()` ENOENT, `readdir()` EISDIR. - v9fs: O_WRONLY and O_RDWR succeed, `openat()` ENOENT, `readdir()` EOPNOTSUPP. - rpmsgfs: `openat()` ENOENT. `readdir()` passes, because both sides have the same `struct dirent`. The tmpfs cases pass on master and with this PR. The mps2-an500 and rpmsgfs rows ran on an earlier version of this PR with the same behaviour: only the place of the `file_checkdir()` call in `fs_open.c` changed since. rpmsgfs mixed pairs: a server with this PR and a master proxy gives 25 / 13 (only the proxy-side failures stay); a master server and a proxy with this PR gives 46 / 1 (`openat()` needs the server fix). O_APPEND leak, same rpmsgfs pair: the proxy mounts the server root and opens `/dev/console` of the server with O_WRONLY | O_APPEND 200 times, twice; `free` on the server before and after: | proxy | first 200 opens | second 200 opens | server heap used | |---|---|---|---| | master | fail, ENOSYS | fail, EMFILE | 63304 / 70112 / 72016 | | this PR | fail, ENOSYS | fail, ENOSYS | 63304 / 63608 / 63608 | Builds: qemu-armv8a:rpserver and :rpproxy with `-Wno-cpp -Werror`, 0 warnings; qemu-armv8a:nsh with CMake and the test, 0 warnings; mps2-an500:knsh with arm-none-eabi-gcc 15.3, 0 warnings. `tools/checkpatch.sh -c -u -m -g` passes. Not run: the Windows simulator, semihosting hostfs, xtensa and risc-v, SMP. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
