Hey, The idea in this series is old-ish and really never let go of me and so I wanted to at least dump it onto the list once even if it's just for illustrative purposes.
I still had parts of an implementation laying around that I started montsh ago alongside FD_PREPARE()/FD_ADD(). I sat down and finished it. We've spoken about this idea a few times over the years that we could reserve fds and files on a task and then install or a clean them up on syscall success or error and get rid of most of the complicated cleanup dance that we have in a lot of code. In particular drm. It last came up during the Rust file descriptor reservation discussion quite some time ago. TL;DR, this lets arch code handle fd install and cleanup. fd_prepare() allocates a descriptor like get_unused_fd_flags() does and records it in a slot on the task. fd_stage() attaches the file to that slot and returns the number. When the syscall returns success the exit path installs every staged file. When it returns an error it drops the descriptors and the files. So a caller reserves, hands the number to userspace whenever it wants, creates the file, stages it and returns errors without unwinding anything. fd_prepare() returns the slot itself, as a const pointer. The preexisting fd_prepare_fd() and fd_prepare_file() give access to the fd and file. get_unused_fd_flags() and fd_install() don't change. A descriptor is only reserved where a caller asks for it. And open(), dup() and all other syscalls that maximize speed simply use FD_ADD(). The task keeps two slots inline. For SCM_RIGHTS and multi-descriptor ioctls a spill array is added. It stick with the task. Reservations belong to the thread and the syscall that made them. A child of fork() starts without any. A thread can't unshare its fdtagble with outstanding reservations. Kernel threads never return to userspace so nothing would commit. Anything left at exit is a bug and gets warned about and dropped. Christian Signed-off-by: Christian Brauner (Amutable) <[email protected]> --- Christian Brauner (50): file: install files on syscall exit entry: commit fds on syscall exit alpha: commit fds on syscall exit ARC: commit fds on syscall exit ARM: commit fds on syscall exit arm64: commit fds on syscall exit csky: commit fds on syscall exit hexagon: commit fds on syscall exit m68k: commit fds on syscall exit microblaze: commit fds on syscall exit MIPS: commit fds on syscall exit nios2: commit fds on syscall exit openrisc: commit fds on syscall exit parisc: commit fds on syscall exit sh: commit fds on syscall exit sparc: commit fds on syscall exit um: commit fds on syscall exit xtensa: commit fds on syscall exit file: require the syscall exit hook from every architecture file: warn when the descriptor table is unshared with slots io_uring: commit fds per request net: install SCM_RIGHTS descriptors when recvmsg() returns file: open-code receive_fd()'s immediate install file: make FD_ADD() a standalone immediate install file: reimplement FD_PREPARE() on the deferred fd_prepare() path dma-buf: stop unwinding sync file descriptors by hand drm/amdkfd: stop collecting CRIU dma-buf descriptors for a final install drm/msm: install the out-fence descriptor when the ioctl returns drm/virtio: install the out-fence descriptor when the ioctl returns drm/vmwgfx: install the out-fence descriptor when the ioctl returns vfio: install the migration data descriptor when the ioctl returns liveupdate: install the session descriptors when the ioctl returns io_uring/zcrx: install the exported descriptor when the request returns sctp: install the peeloff descriptor when the syscall returns ALSA: compress: install the task descriptors when the ioctl returns nitro_enclaves: install the enclave descriptor when the ioctl returns tpm: vtpm_proxy: install the server descriptor when the ioctl returns perf: stop putting the event descriptor back on failure seccomp: stop putting the listener descriptor back on failure KVM: stop putting descriptors back on failure KVM: guest_memfd: stop putting the descriptor back on failure drm: stop unwinding descriptors by hand drm/amdgpu: stop unwinding the fence descriptor by hand drm/etnaviv: install the out-fence descriptor when the ioctl returns accel/habanalabs: stop putting the dma-buf descriptor back on failure xen/gntdev-dmabuf: stop putting the descriptor back on failure iio: buffer: install the buffer descriptor when the ioctl returns misc: fastrpc: install the dma-buf descriptor when the ioctl returns iommufd: stop putting descriptors back on failure Drivers: hv: mshv: stop putting descriptors back on failure arch/alpha/include/asm/thread_info.h | 2 + arch/alpha/kernel/entry.S | 24 ++- arch/alpha/kernel/ptrace.c | 3 + arch/arc/include/asm/thread_info.h | 2 + arch/arc/kernel/entry.S | 5 + arch/arc/kernel/ptrace.c | 4 + arch/arm/include/asm/thread_info.h | 2 + arch/arm/kernel/entry-common.S | 2 + arch/arm/kernel/ptrace.c | 4 + arch/arm64/include/asm/thread_info.h | 4 +- arch/arm64/kernel/ptrace.c | 4 + arch/csky/include/asm/thread_info.h | 2 + arch/csky/kernel/entry.S | 9 + arch/csky/kernel/ptrace.c | 4 + arch/hexagon/include/asm/thread_info.h | 2 + arch/hexagon/kernel/traps.c | 4 + arch/m68k/68000/entry.S | 10 ++ arch/m68k/coldfire/entry.S | 12 ++ arch/m68k/include/asm/thread_info.h | 2 + arch/m68k/kernel/entry.S | 2 + arch/m68k/kernel/ptrace.c | 3 + arch/microblaze/include/asm/thread_info.h | 5 +- arch/microblaze/kernel/ptrace.c | 4 + arch/mips/include/asm/thread_info.h | 5 +- arch/mips/kernel/ptrace.c | 4 + arch/nios2/include/asm/thread_info.h | 2 + arch/nios2/kernel/entry.S | 5 + arch/nios2/kernel/ptrace.c | 3 + arch/openrisc/include/asm/thread_info.h | 5 +- arch/openrisc/kernel/entry.S | 5 +- arch/openrisc/kernel/ptrace.c | 4 + arch/parisc/include/asm/thread_info.h | 2 + arch/parisc/kernel/entry.S | 12 ++ arch/parisc/kernel/ptrace.c | 4 + arch/powerpc/platforms/cell/spufs/inode.c | 12 +- arch/sh/include/asm/thread_info.h | 10 +- arch/sh/kernel/entry-common.S | 8 +- arch/sh/kernel/ptrace_32.c | 4 + arch/sparc/include/asm/thread_info_32.h | 2 + arch/sparc/include/asm/thread_info_64.h | 9 +- arch/sparc/kernel/entry.S | 2 +- arch/sparc/kernel/ptrace_32.c | 4 + arch/sparc/kernel/ptrace_64.c | 4 + arch/sparc/kernel/syscalls.S | 4 +- arch/um/include/asm/thread_info.h | 2 + arch/um/kernel/ptrace.c | 4 + arch/xtensa/include/asm/thread_info.h | 2 + arch/xtensa/kernel/entry.S | 5 + arch/xtensa/kernel/ptrace.c | 4 + drivers/accel/habanalabs/common/memory.c | 13 +- drivers/char/tpm/tpm_vtpm_proxy.c | 30 +--- drivers/dma-buf/dma-buf.c | 33 ++-- drivers/dma-buf/sw_sync.c | 40 ++--- drivers/dma-buf/sync_file.c | 52 ++---- drivers/gpio/gpiolib-cdev.c | 18 +- drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 16 +- drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 67 ++------ drivers/gpu/drm/drm_lease.c | 20 +-- drivers/gpu/drm/drm_prime.c | 13 +- drivers/gpu/drm/drm_syncobj.c | 44 ++--- drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c | 14 +- drivers/gpu/drm/msm/msm_gem_submit.c | 22 +-- drivers/gpu/drm/msm/msm_gem_vma.c | 22 +-- drivers/gpu/drm/msm/msm_perfcntr.c | 6 +- drivers/gpu/drm/virtio/virtgpu_submit.c | 32 ++-- drivers/gpu/drm/vmwgfx/vmwgfx_execbuf.c | 29 ++-- drivers/hv/mshv_root_main.c | 27 +-- drivers/hv/mshv_vtl_main.c | 20 +-- drivers/iio/industrialio-buffer.c | 26 +-- drivers/iommu/iommufd/eventq.c | 39 ++--- drivers/media/mc/mc-request.c | 6 +- drivers/misc/fastrpc.c | 19 +-- drivers/misc/ntsync.c | 6 +- drivers/vfio/vfio_main.c | 25 +-- drivers/virt/nitro_enclaves/ne_misc_dev.c | 33 ++-- drivers/xen/gntdev-dmabuf.c | 14 +- fs/eventfd.c | 6 +- fs/eventpoll.c | 6 +- fs/exec.c | 3 + fs/file.c | 245 +++++++++++++++++++++++++-- fs/namespace.c | 12 +- fs/nsfs.c | 6 +- fs/xfs/xfs_handle.c | 6 +- include/linux/entry-common.h | 8 +- include/linux/file.h | 160 +++++------------ include/linux/sched.h | 21 +++ include/linux/thread_info.h | 2 + io_uring/io_uring.c | 27 +++ io_uring/mock_file.c | 5 +- io_uring/zcrx.c | 25 +-- kernel/bpf/bpf_iter.c | 6 +- kernel/bpf/token.c | 6 +- kernel/events/core.c | 31 ++-- kernel/exit.c | 1 + kernel/fork.c | 3 + kernel/liveupdate/luo_core.c | 48 ++---- kernel/liveupdate/luo_session.c | 24 +-- kernel/seccomp.c | 13 +- mm/userfaultfd.c | 6 +- net/core/scm.c | 13 +- net/handshake/netlink.c | 20 ++- net/kcm/kcmsock.c | 5 +- net/sctp/socket.c | 78 ++++----- sound/core/compress_offload.c | 24 ++- virt/kvm/guest_memfd.c | 20 +-- virt/kvm/kvm_main.c | 65 +++---- 106 files changed, 962 insertions(+), 861 deletions(-) --- base-commit: cee9395acd8043be0644b25c34bfa86623f2b935 change-id: 20260915-work-fd-reserve-unify-folded-b89d3154b3d0
