This is an automated email from the ASF dual-hosted git repository. acassis pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/nuttx.git
commit 70c2ef5911cc281b4a6e8a6f7412cf68c9e3a99a Author: Marco Casaroli <[email protected]> AuthorDate: Fri Aug 7 12:12:59 2026 +0200 !sched/arch/libc: Give fork() and vfork() their real, separate semantics. NuttX implemented fork() and vfork() as the same function. Both were libc wrappers around a single up_fork() syscall; vfork() differed only by a trailing waitpid(). Underneath, the child joined the parent's address environment -- the same addrenv_join() that pthread_create() uses -- and got a private copy of the stack. So the child shared .data, .bss and the heap with its parent and ran concurrently with it. That is not fork(). It is vfork()-with-a-private-stack under fork()'s name, and the history says so: today's fork() is NuttX's old vfork(), renamed in c33d1c9c97 (2023) without any change of behaviour. The failure was silent -- a program written against POSIX fork() compiled, ran, and had its child's writes land in the parent's variables. Separate them into two primitives, chosen by which function the caller called rather than by what the hardware happens to be: fork() child gets its own copy of the parent's memory at the same virtual addresses; runs concurrently. Only where an address environment can be duplicated -- elsewhere it is not declared at all, so calling it is a build error naming the function. vfork() child shares the parent's memory; parent suspended until the child _exit()s or exec()s. Implementable everywhere. Below libc there is still one syscall. up_fork() gains a bool saying which primitive the caller used, since the per-architecture register snapshot is the same for both, and passes it to nxtask_setup_fork(), which is the single place the memory semantics are decided. The argument arrives in the first argument register and is never touched: each architecture's snapshot takes some other call-clobbered register for its scratch, so the flag is simply still there when the C worker is called. The vfork() parent suspension moves out of libc into nxtask_start_fork(), released from nxsched_release_tcb() by nxtask_resume_vfork(). Two things follow: the parent is resumed at exec(), since exec_swap() has already handed the child's pid to the loaded program by the time the vfork stub exits, and vfork() no longer depends on CONFIG_SCHED_WAITPID. Releasing there requires one fix in nxtask_exit(). It raises rtcb->lockcount directly rather than through sched_lock() while it tears the TCB down, so the nxsem_post() that wakes the vfork() parent leaves it queued where a blocked task collects while pre-emption is off -- g_pendingtasks, or g_readytorun on SMP -- and the matching raw lockcount-- does not publish it the way sched_unlock() would, leaving the parent stranded with nothing to move it on. The fix mirrors sched_unlock() for each case: nxsched_merge_pending(), or nxsched_deliver_task() under CONFIG_SMP. Both are no-ops while pre-emption is still disabled, and up_exit() re-reads this_task() afterwards, so a change of the ready-to-run head is honoured. Without it vfork() deadlocks wherever no other task happens to call sched_unlock() afterwards -- rv-virt:nsh64 and rv-virt:pnsh64, where NSH is blocked in waitpid() holding the lock, and qemu-armv8a:citest_smp, which hangs the moment the vfork() test runs. fork() is built on a new addrenv_fork(), backed by an up_addrenv_fork() hook that duplicates an address environment into freshly allocated pages mapped at the same virtual addresses -- unlike up_addrenv_clone(), which copies only the representation and leaves both pointing at the same page tables. The child then adopts the parent's stack geometry rather than being given a relocated copy: a pointer to a stack local taken before fork() must name the same object in the child that it named in the parent, and the parent's stack is already in the duplicate, with its contents, at the parent's address. No architecture implements up_addrenv_fork() yet, so this commit leaves fork() unavailable everywhere. That is the intended state. It withdraws fork() from ARCH_ARM, flat ARCH_ARM64, ARCH_RISCV, ARCH_SIM and ARCH_X86_64, where until now it named the sharing primitive; per-architecture patches restore it, with POSIX semantics, as up_addrenv_fork() lands. In the meantime the sharing primitive is still there under the name that describes it: vfork() for a child that runs a program, pthread_create() for a second flow of control that shares memory, posix_spawn() for both at once. Kconfig: ARCH_HAVE_VFORK inherits ARCH_HAVE_FORK's select lines, conditions included, so no configuration gains machinery; ARCH_HAVE_FORK is redefined to mean "can provide POSIX fork() semantics" and now depends on ARCH_ADDRENV. There is one deliberate departure from "verbatim". ARCH_ARM selected the fork family unconditionally, BUILD_KERNEL included, and that has never worked: on a kernel build the architecture's fork entry point sees the kernel's return address and stack pointer rather than the caller's, so the child resumes at a kernel address. On qemu-armv7a:knsh master faults in ostest's fork case with "Child did not run" and then a data abort; without the condition this change faults the same way through vfork(). ARCH_ARM64 and ARCH_X86_64 already carried "if !BUILD_KERNEL" for exactly this reason -- ARM was the outlier. Conditioning it turns a runtime fault into an honest absence, which is the whole point of the change; arch/arm takes the condition off again in the patch that adds its saved-syscall-frame path. Only the MMU-capable ARM ports are affected, since Cortex-M cannot build BUILD_KERNEL at all. Also fixes two latent syntax errors found on the way: a missing comma in riscv_fork.c and mips_fork.c, both in *_FRAMEPOINTER && !SAVE_GP branches that are never compiled today. BREAKING CHANGE: fork() is withdrawn from every architecture. It is no longer declared in unistd.h, so code that calls it fails to build with an error naming the function, and the sharing behaviour it used to have is gone rather than renamed. CONFIG_ARCH_HAVE_FORK no longer means "fork() exists"; it means "this configuration can provide POSIX fork() semantics", and no architecture selects it yet. Quick fix, chosen by why the call was made: to run a program vfork() + exec*(), or better posix_spawn() a second flow of control that pthread_create() shares the caller's memory a genuinely independent copy keep fork(), and wait for the per-arch patch of the process that implements up_addrenv_fork() and selects CONFIG_ARCH_HAVE_FORK Out-of-tree code that tests CONFIG_ARCH_HAVE_FORK to decide whether a fork-then-exec path is available wants CONFIG_ARCH_HAVE_VFORK instead, which is selected in exactly the places CONFIG_ARCH_HAVE_FORK used to be. The full migration guide is Documentation/guides/fork_vfork_migration.rst. Assisted-by: Claude Code:claude-opus-5 Signed-off-by: Marco Casaroli <[email protected]> --- arch/Kconfig | 40 ++- arch/arm/src/common/arm_fork.c | 126 +++++---- arch/arm/src/common/gnu/fork.S | 58 ++--- arch/arm/src/common/iar/fork.S | 52 ++-- arch/arm64/src/common/arm64_fork.c | 78 +++--- arch/arm64/src/common/arm64_fork_func.S | 57 +++-- arch/ceva/src/common/ceva_fork.c | 18 +- arch/mips/Kconfig | 2 +- arch/mips/src/mips32/Kconfig | 2 +- arch/mips/src/mips32/fork.S | 38 +-- arch/mips/src/mips32/mips_fork.c | 11 +- arch/risc-v/src/common/CMakeLists.txt | 2 +- arch/risc-v/src/common/Make.defs | 2 +- arch/risc-v/src/common/fork.S | 68 +++-- arch/risc-v/src/common/riscv_fork.c | 143 ++++++----- arch/sim/src/Makefile | 2 +- arch/sim/src/sim/CMakeLists.txt | 2 +- arch/sim/src/sim/sim_fork.c | 24 +- arch/sim/src/sim/sim_fork_arm.S | 42 +-- arch/sim/src/sim/sim_fork_arm64.S | 47 ++-- arch/sim/src/sim/sim_fork_x86.S | 44 ++-- arch/sim/src/sim/sim_fork_x86_64.S | 45 ++-- arch/x86_64/src/common/CMakeLists.txt | 2 +- arch/x86_64/src/common/Make.defs | 2 +- arch/x86_64/src/common/fork.S | 41 +-- arch/x86_64/src/common/x86_64_fork.c | 82 +++--- include/nuttx/addrenv.h | 25 ++ include/nuttx/arch.h | 47 +++- include/nuttx/sched.h | 40 ++- include/sys/syscall_lookup.h | 4 +- include/unistd.h | 9 + libs/libbuiltin/libgcc/gcov.c | 6 + libs/libc/libc.csv | 1 + libs/libc/unistd/CMakeLists.txt | 2 +- libs/libc/unistd/Make.defs | 2 +- libs/libc/unistd/lib_fork.c | 68 +++-- sched/addrenv/addrenv.c | 65 +++++ sched/sched/sched.h | 10 + sched/sched/sched_releasetcb.c | 9 + sched/task/CMakeLists.txt | 2 +- sched/task/Make.defs | 2 +- sched/task/task_exit.c | 14 + sched/task/task_fork.c | 437 +++++++++++++++++++++++++------- syscall/syscall.csv | 2 +- 44 files changed, 1202 insertions(+), 573 deletions(-) diff --git a/arch/Kconfig b/arch/Kconfig index ab0a0c066c7..13e0db6ea98 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -11,7 +11,7 @@ config ARCH_ARM bool "ARM" select ARCH_HAVE_BACKTRACE select ARCH_HAVE_INTERRUPTSTACK - select ARCH_HAVE_FORK + select ARCH_HAVE_VFORK if !BUILD_KERNEL select ARCH_HAVE_STACKCHECK select ARCH_HAVE_CUSTOMOPT select ARCH_HAVE_STDARG_H @@ -29,7 +29,7 @@ config ARCH_ARM64 select ARCH_64BIT select ARCH_HAVE_BACKTRACE select ARCH_HAVE_INTERRUPTSTACK - select ARCH_HAVE_FORK if !BUILD_KERNEL && !BUILD_PROTECTED + select ARCH_HAVE_VFORK if !BUILD_KERNEL && !BUILD_PROTECTED select ARCH_HAVE_STACKCHECK select ARCH_HAVE_CUSTOMOPT select ARCH_HAVE_STDARG_H @@ -87,7 +87,7 @@ config ARCH_RISCV select ARCH_HAVE_CPUINFO select ARCH_HAVE_INTERRUPTSTACK select ARCH_HAVE_STACKCHECK - select ARCH_HAVE_FORK + select ARCH_HAVE_VFORK select ARCH_HAVE_CUSTOMOPT select ARCH_HAVE_SETJMP select ARCH_HAVE_STDARG_H @@ -111,7 +111,7 @@ config ARCH_SIM select ARCH_HAVE_TICKLESS select ARCH_HAVE_POWEROFF select ARCH_HAVE_TESTSET - select ARCH_HAVE_FORK if !HOST_WINDOWS + select ARCH_HAVE_VFORK if !HOST_WINDOWS select ARCH_HAVE_SETJMP select ARCH_HAVE_CUSTOMOPT select ARCH_HAVE_TCBINFO @@ -147,7 +147,7 @@ config ARCH_X86_64 select PCI_LATE_DRIVERS_REGISTER if PCI select ARCH_TOOLCHAIN_GNU select ARCH_HAVE_BACKTRACE - select ARCH_HAVE_FORK if !BUILD_KERNEL + select ARCH_HAVE_VFORK if !BUILD_KERNEL select ARCH_HAVE_SETJMP select ARCH_HAVE_PERF_EVENTS select ARCH_HAVE_POWEROFF @@ -482,9 +482,39 @@ config ARCH_HAVE_CPUID_MAPPING default n depends on ARCH_HAVE_MULTICPU +config ARCH_HAVE_VFORK + bool + default n + ---help--- + The architecture can implement POSIX vfork(): the child shares the + parent's memory and the parent is suspended until the child calls + _exit() or one of the exec family of functions. + config ARCH_HAVE_FORK bool default n + depends on ARCH_ADDRENV + ---help--- + The architecture can implement POSIX fork(): the child receives its + own copy of the parent's memory at the same virtual addresses, may + modify anything, may return from the function that called fork(), and + runs concurrently with the parent. + + This requires an address environment to duplicate, and an + up_addrenv_fork() to duplicate it with: the copy is backed by freshly + allocated pages holding a copy of the parent's contents, mapped at the + same virtual addresses. + + No architecture selects this yet. Two things are needed. First, + up_addrenv_fork() itself. Second, the architecture must build the + child's register context from the *user's* saved system call frame: + in a kernel build fork() is reached through a system call, so the + return address and stack pointer the architecture's fork entry point + can see for itself are the kernel's, not the caller's, and a child + built from those resumes at a kernel address. + + Where this is not selected fork() is not provided at all, and code + that calls it fails to build. config ARCH_HAVE_CRC32 bool diff --git a/arch/arm/src/common/arm_fork.c b/arch/arm/src/common/arm_fork.c index f98f8ec896a..db822caee75 100644 --- a/arch/arm/src/common/arm_fork.c +++ b/arch/arm/src/common/arm_fork.c @@ -49,47 +49,56 @@ * Name: arm_fork * * Description: - * The fork() function has the same effect as posix fork(), except that the - * behavior is undefined if the process created by fork() either modifies - * any data other than a variable of type pid_t used to store the return - * value from fork(), or returns from the function in which fork() was - * called, or calls any other function before successfully calling _exit() - * or one of the exec family of functions. + * The common ARM worker behind up_fork(). vfork() and fork() snapshot + * the caller's registers identically; `vfork' says which primitive was + * called, and is passed straight through to nxtask_setup_fork(), which is + * where the memory semantics are decided. + * + * What differs here is only the stack. Normally the child has a stack of + * its own, and this function fills it with a relocated copy of the + * parent's, rebasing the stack and frame pointers to match. When the + * child shares the parent's stack addresses -- a fork() child, inside its + * duplicated address environment -- there is nothing to relocate and the + * pointers are carried over unchanged. * * The overall sequence is: * - * 1) User code calls fork(). fork() collects context information and - * transfers control up arm_fork(). - * 2) arm_fork() and calls nxtask_setup_fork(). + * 1) User code calls vfork() or fork(). The libc wrapper enters + * up_fork(), which collects context information and transfers control + * to arm_fork(). + * 2) arm_fork() calls nxtask_setup_fork(). * 3) nxtask_setup_fork() allocates and configures the child task's TCB. * This consists of: * - Allocation of the child task's TCB. * - Initialization of file descriptors and streams * - Configuration of environment variables - * - Allocate and initialize the stack + * - Establishing the child's address environment: joined to the + * parent's for vfork(), duplicated from it for fork() + * - Allocating the stack, or inheriting the parent's for fork() * - Setup the input parameters for the task. * - Initialization of the TCB (including call to up_initial_state()) * 4) arm_fork() provides any additional operating context. arm_fork must: * - Initialize special values in any CPU registers that were not * already configured by up_initial_state() - * 5) arm_fork() then calls nxtask_start_fork() - * 6) nxtask_start_fork() then executes the child thread. + * 5) arm_fork() then calls nxtask_start_fork(), which for vfork() + * additionally suspends the caller. + * 6) which executes the child thread. * * nxtask_abort_fork() may be called if an error occurs between steps 3 and * 6. * * Input Parameters: - * context - Caller context information saved by fork() + * vfork - true for vfork(), false for fork() + * context - Caller context information saved by the entry point * * Returned Value: - * Upon successful completion, fork() returns 0 to the child process and - * returns the process ID of the child process to the parent process. - * Otherwise, -1 is returned to the parent, no child process is created, - * and errno is set to indicate the error. + * Upon successful completion, 0 is returned to the child and the process + * ID of the child is returned to the parent. Otherwise, -1 is returned to + * the parent, no child is created, and errno is set to indicate the error. * ****************************************************************************/ -pid_t arm_fork(const struct fork_s *context) +pid_t arm_fork(bool vfork, const struct fork_s *context) { struct tcb_s *parent = this_task(); struct tcb_s *child; @@ -115,7 +124,7 @@ pid_t arm_fork(const struct fork_s *context) /* Allocate and initialize a TCB for the child task. */ - child = nxtask_setup_fork((start_t)(context->lr & ~1)); + child = nxtask_setup_fork((start_t)(context->lr & ~1), vfork); if (!child) { serr("ERROR: nxtask_setup_fork failed\n"); @@ -137,43 +146,60 @@ pid_t arm_fork(const struct fork_s *context) sinfo("Parent: stackutil:%" PRIu32 "\n", stackutil); - /* Make some feeble effort to preserve the stack contents. This is - * feeble because the stack surely contains invalid pointers and other - * content that will not work in the child context. However, if the - * user follows all of the caveats of fork() usage, even this feeble - * effort is overkill. - */ + if (child->stack_base_ptr == parent->stack_base_ptr) + { + /* The child is running at the parent's stack addresses, inside its + * own duplicated address environment. There is nothing to relocate: + * every stack address the child inherits is still the address it + * names. + */ - newtop = (uint32_t)child->stack_base_ptr + - child->adj_stack_size; + newsp = oldsp; + newfp = context->fp; + } + else + { + /* Make some feeble effort to preserve the stack contents. This is + * feeble because the stack surely contains invalid pointers and other + * content that will not work in the child context. However, if the + * user follows all of the caveats of vfork() usage, even this feeble + * effort is overkill. + * + * For a POSIX fork() child the stack contents are not merely a feeble + * effort: the child is entitled to use them, and it does. + */ - newsp = newtop - stackutil; + newtop = (uint32_t)child->stack_base_ptr + + child->adj_stack_size; - /* Move the register context to newtop. */ + newsp = newtop - stackutil; - memcpy((void *)(newsp - XCPTCONTEXT_SIZE), - child->xcp.regs, XCPTCONTEXT_SIZE); + /* Move the register context to newtop. */ - child->xcp.regs = (void *)(newsp - XCPTCONTEXT_SIZE); + memcpy((void *)(newsp - XCPTCONTEXT_SIZE), + child->xcp.regs, XCPTCONTEXT_SIZE); - memcpy((void *)newsp, (const void *)oldsp, stackutil); + child->xcp.regs = (void *)(newsp - XCPTCONTEXT_SIZE); - /* Was there a frame pointer in place before? */ + memcpy((void *)newsp, (const void *)oldsp, stackutil); - if (context->fp >= oldsp && context->fp < stacktop) - { - uint32_t frameutil = stacktop - context->fp; - newfp = newtop - frameutil; - } - else - { - newfp = context->fp; - } + /* Was there a frame pointer in place before? */ - sinfo("Old stack top:%08" PRIx32 " SP:%08" PRIx32 " FP:%08" PRIx32 "\n", - stacktop, oldsp, context->fp); - sinfo("New stack top:%08" PRIx32 " SP:%08" PRIx32 " FP:%08" PRIx32 "\n", - newtop, newsp, newfp); + if (context->fp >= oldsp && context->fp < stacktop) + { + uint32_t frameutil = stacktop - context->fp; + newfp = newtop - frameutil; + } + else + { + newfp = context->fp; + } + + sinfo("Old stack top:%08" PRIx32 " SP:%08" PRIx32 + " FP:%08" PRIx32 "\n", stacktop, oldsp, context->fp); + sinfo("New stack top:%08" PRIx32 " SP:%08" PRIx32 + " FP:%08" PRIx32 "\n", newtop, newsp, newfp); + } /* Update the stack pointer, frame pointer, and volatile registers. When * the child TCB was initialized, all of the values were set to zero. @@ -245,9 +271,9 @@ pid_t arm_fork(const struct fork_s *context) } #endif - /* And, finally, start the child task. On a failure, nxtask_start_fork() - * will discard the TCB by calling nxtask_abort_fork(). + /* And, finally, start the child task. A vfork() additionally suspends us + * until the child calls _exit() or exec(). */ - return nxtask_start_fork(child); + return nxtask_start_fork(child, vfork); } diff --git a/arch/arm/src/common/gnu/fork.S b/arch/arm/src/common/gnu/fork.S index 83e22326c26..60a1a2f07be 100644 --- a/arch/arm/src/common/gnu/fork.S +++ b/arch/arm/src/common/gnu/fork.S @@ -1,5 +1,5 @@ /**************************************************************************** - * arch/arm/src/common/fork.S + * arch/arm/src/common/gnu/fork.S * * SPDX-License-Identifier: Apache-2.0 * @@ -38,43 +38,41 @@ * Name: up_fork * * Description: - * The up_fork() function is the base of fork() function that provided in - * libc, and fork() is implemented as a wrapper of up_fork() function. - * The fork() function has the same effect as posix fork(), except that the - * behavior is undefined if the process created by fork() either modifies - * any data other than a variable of type pid_t used to store the return - * value from fork(), or returns from the function in which fork() was - * called, or calls any other function before successfully calling _exit() - * or one of the exec family of functions. + * The architecture-specific entry point of both of NuttX's cloning + * primitives. Both need exactly the same thing from assembly -- a + * snapshot of the caller's callee-saved registers, stack pointer and + * return address -- and differ only in what the C code then does with it, + * so there is one entry point and the caller's r0 says which primitive was + * called. It is passed straight through to arm_fork(). * - * This thin layer implements fork by simply calling up_fork() with the - * fork() context as an argument. The overall sequence is: + * The overall sequence is: * - * 1) User code calls fork(). fork() collects context information and - * transfers control up up_fork(). - * 2) arm_fork() and calls nxtask_setup_fork(). - * 3) nxtask_setup_fork() allocates and configures the child task's TCB. - * This consists of: + * 1) User code calls vfork() or fork(). Both are libc wrappers around + * this entry point. + * 2) The entry point collects the context and calls arm_fork(). + * 3) arm_fork() calls nxtask_setup_fork(), which allocates and configures + * the child task's TCB. This consists of: * - Allocation of the child task's TCB. * - Initialization of file descriptors and streams * - Configuration of environment variables - * - Allocate and initialize the stack + * - Establishing the child's address environment + * - Allocating the stack, or inheriting the parent's for fork() * - Setup the input parameters for the task. * - Initialization of the TCB (including call to up_initial_state()) - * 4) arm_fork() provides any additional operating context. arm_fork must: + * 4) arm_fork() provides any additional operating context: * - Initialize special values in any CPU registers that were not * already configured by up_initial_state() + * - Relocate the copied stack, unless the child shares the parent's * 5) arm_fork() then calls nxtask_start_fork() - * 6) nxtask_start_fork() then executes the child thread. + * 6) which executes the child thread. * * Input Parameters: - * None + * r0 - true for vfork(), false for fork() * * Returned Value: - * Upon successful completion, fork() returns 0 to the child process and - * returns the process ID of the child process to the parent process. - * Otherwise, -1 is returned to the parent, no child process is created, - * and errno is set to indicate the error. + * Upon successful completion, 0 is returned to the child and the process + * ID of the child is returned to the parent. Otherwise, -1 is returned to + * the parent, no child is created, and errno is set to indicate the error. * ****************************************************************************/ @@ -88,7 +86,7 @@ up_fork: /* Create a stack frame */ - mov r0, sp /* Save the value of the stack on entry */ + mov r3, sp /* Save the value of the stack on entry */ sub sp, sp, #FORK_SIZEOF /* Allocate the structure on the stack */ /* CPU registers */ @@ -102,11 +100,13 @@ up_fork: mov r7, r11 stmia r1!, {r4-r7} /* Save r8-r11 in the structure */ mov r5, lr /* Copy lr to a low register */ - stmia r1!, {r0,r5} /* Save sp and lr in the structure */ + stmia r1!, {r3,r5} /* Save sp and lr in the structure */ - /* Then, call arm_fork(), passing it a pointer to the stack structure */ + /* Then, call arm_fork(). r0 still holds the vfork flag: nothing above + * touches it. + */ - mov r0, sp + mov r1, sp bl arm_fork /* Recover r4-r7 that were destroyed before arm_fork was called */ @@ -114,7 +114,7 @@ up_fork: mov r1, sp ldmia r1!, {r4-r7} - /* Release the stack data and return the value returned by up_fork */ + /* Release the stack data and return the value returned by arm_fork */ ldr r1, [sp, #FORK_LR_OFFSET] mov r14, r1 diff --git a/arch/arm/src/common/iar/fork.S b/arch/arm/src/common/iar/fork.S index 3f16484a365..53e3e221e3a 100644 --- a/arch/arm/src/common/iar/fork.S +++ b/arch/arm/src/common/iar/fork.S @@ -50,43 +50,41 @@ * Name: up_fork * * Description: - * The up_fork() function is the base of fork() function that provided in - * libc, and fork() is implemented as a wrapper of up_fork() function. - * The fork() function has the same effect as posix fork(), except that the - * behavior is undefined if the process created by fork() either modifies - * any data other than a variable of type pid_t used to store the return - * value from fork(), or returns from the function in which fork() was - * called, or calls any other function before successfully calling _exit() - * or one of the exec family of functions. + * The architecture-specific entry point of both of NuttX's cloning + * primitives. Both need exactly the same thing from assembly -- a + * snapshot of the caller's callee-saved registers, stack pointer and + * return address -- and differ only in what the C code then does with it, + * so there is one entry point and the caller's r0 says which primitive was + * called. It is passed straight through to arm_fork(). * - * This thin layer implements fork by simply calling up_fork() with the - * fork() context as an argument. The overall sequence is: + * The overall sequence is: * - * 1) User code calls fork(). fork() collects context information and - * transfers control up up_fork(). - * 2) arm_fork() and calls nxtask_setup_fork(). - * 3) nxtask_setup_fork() allocates and configures the child task's TCB. - * This consists of: + * 1) User code calls vfork() or fork(). Both are libc wrappers around + * this entry point. + * 2) The entry point collects the context and calls arm_fork(). + * 3) arm_fork() calls nxtask_setup_fork(), which allocates and configures + * the child task's TCB. This consists of: * - Allocation of the child task's TCB. * - Initialization of file descriptors and streams * - Configuration of environment variables - * - Allocate and initialize the stack + * - Establishing the child's address environment + * - Allocating the stack, or inheriting the parent's for fork() * - Setup the input parameters for the task. * - Initialization of the TCB (including call to up_initial_state()) * 4) arm_fork() provides any additional operating context. arm_fork must: * - Initialize special values in any CPU registers that were not * already configured by up_initial_state() + * - Relocate the copied stack, unless the child shares the parent's * 5) arm_fork() then calls nxtask_start_fork() - * 6) nxtask_start_fork() then executes the child thread. + * 6) which executes the child thread. * * Input Parameters: - * None + * r0 - true for vfork(), false for fork() * * Returned Value: - * Upon successful completion, fork() returns 0 to the child process and - * returns the process ID of the child process to the parent process. - * Otherwise, -1 is returned to the parent, no child process is created, - * and errno is set to indicate the error. + * Upon successful completion, 0 is returned to the child and the process + * ID of the child is returned to the parent. Otherwise, -1 is returned to + * the parent, no child is created, and errno is set to indicate the error. * ****************************************************************************/ @@ -95,7 +93,7 @@ up_fork: /* Create a stack frame */ - mov r0, sp /* Save the value of the stack on entry */ + mov r3, sp /* Save the value of the stack on entry */ sub sp, sp, #FORK_SIZEOF /* Allocate the structure on the stack */ /* CPU registers */ @@ -112,14 +110,16 @@ up_fork: /* Save the frame pointer, stack pointer, and return address */ str r11, [sp, #FORK_FP_OFFSET] /* fp not defined. use r11 */ - str r0, [sp, #FORK_SP_OFFSET] + str r3, [sp, #FORK_SP_OFFSET] str lr, [sp, #FORK_LR_OFFSET] /* Floating point registers (not yet) */ - /* Then, call arm_fork(), passing it a pointer to the stack structure */ + /* Then, call arm_fork(). r0 still holds the vfork flag: nothing above + * touches it. + */ - mov r0, sp + mov r1, sp bl arm_fork /* Release the stack data and return the value returned by arm_fork */ diff --git a/arch/arm64/src/common/arm64_fork.c b/arch/arm64/src/common/arm64_fork.c index fbbf6a56da0..b7d5af1f9f7 100644 --- a/arch/arm64/src/common/arm64_fork.c +++ b/arch/arm64/src/common/arm64_fork.c @@ -69,20 +69,18 @@ void arm64_fork_fpureg_save(struct fork_s *context) #endif /**************************************************************************** - * Name: fork + * Name: arm64_fork * * Description: - * The fork() function has the same effect as posix fork(), except that the - * behavior is undefined if the process created by fork() either modifies - * any data other than a variable of type pid_t used to store the return - * value from fork(), or returns from the function in which fork() was - * called, or calls any other function before successfully calling _exit() - * or one of the exec family of functions. + * The common ARM64 worker behind up_fork(). vfork() and fork() snapshot + * the caller's registers identically; `vfork' says which primitive was + * called, and is passed straight through to nxtask_setup_fork(), which is + * where the memory semantics are decided. * * The overall sequence is: * - * 1) User code calls fork(). fork() collects context information and - * transfers control up arm64_fork(). + * 1) User code calls vfork() or fork(). up_fork() collects context + * information and transfers control to arm64_fork(). * 2) arm64_fork() and calls nxtask_setup_fork(). * 3) nxtask_setup_fork() allocates and configures the child task's TCB. * This consists of: @@ -103,7 +101,8 @@ void arm64_fork_fpureg_save(struct fork_s *context) * 6. * * Input Parameters: - * context - Caller context information saved by fork() + * vfork - true for vfork(), false for fork() + * context - Caller context information saved by up_fork() * * Returned Value: * Upon successful completion, fork() returns 0 to the child process and @@ -113,7 +112,7 @@ void arm64_fork_fpureg_save(struct fork_s *context) * ****************************************************************************/ -pid_t arm64_fork(const struct fork_s *context) +pid_t arm64_fork(bool vfork, const struct fork_s *context) { struct tcb_s *parent = this_task(); struct tcb_s *child; @@ -125,7 +124,7 @@ pid_t arm64_fork(const struct fork_s *context) /* Allocate and initialize a TCB for the child task. */ - child = nxtask_setup_fork((start_t)context->lr); + child = nxtask_setup_fork((start_t)context->lr, vfork); if (!child) { serr("ERROR: nxtask_setup_fork failed\n"); @@ -143,28 +142,45 @@ pid_t arm64_fork(const struct fork_s *context) DEBUGASSERT(stacktop > context->sp); stackutil = stacktop - context->sp; - /* Make some feeble effort to preserve the stack contents. This is - * feeble because the stack surely contains invalid pointers and other - * content that will not work in the child context. However, if the - * user follows all of the caveats of fork() usage, even this feeble - * effort is overkill. - */ - - newtop = (uint64_t)child->stack_base_ptr + - child->adj_stack_size; - newsp = newtop - stackutil; - memcpy((void *)newsp, (const void *)context->sp, stackutil); - - /* Was there a frame pointer in place before? */ - - if (context->fp >= context->sp && context->fp < stacktop) + if (child->stack_base_ptr == parent->stack_base_ptr) { - uint64_t frameutil = stacktop - context->fp; - newfp = newtop - frameutil; + /* The child is running at the parent's stack addresses, inside its + * own duplicated address environment. There is nothing to relocate: + * every stack address the child inherits is still the address it + * names. + */ + + newsp = context->sp; + newfp = context->fp; } else { - newfp = context->fp; + /* Make some feeble effort to preserve the stack contents. This is + * feeble because the stack surely contains invalid pointers and other + * content that will not work in the child context. However, if the + * user follows all of the caveats of vfork() usage, even this feeble + * effort is overkill. + * + * For a POSIX fork() child the stack contents are not merely a feeble + * effort: the child is entitled to use them, and it does. + */ + + newtop = (uint64_t)child->stack_base_ptr + + child->adj_stack_size; + newsp = newtop - stackutil; + memcpy((void *)newsp, (const void *)context->sp, stackutil); + + /* Was there a frame pointer in place before? */ + + if (context->fp >= context->sp && context->fp < stacktop) + { + uint64_t frameutil = stacktop - context->fp; + newfp = newtop - frameutil; + } + else + { + newfp = context->fp; + } } /* Update the stack pointer, frame pointer, and volatile registers. When @@ -235,5 +251,5 @@ pid_t arm64_fork(const struct fork_s *context) * will discard the TCB by calling nxtask_abort_fork(). */ - return nxtask_start_fork(child); + return nxtask_start_fork(child, vfork); } diff --git a/arch/arm64/src/common/arm64_fork_func.S b/arch/arm64/src/common/arm64_fork_func.S index 79cd9566076..83fb6017879 100644 --- a/arch/arm64/src/common/arm64_fork_func.S +++ b/arch/arm64/src/common/arm64_fork_func.S @@ -41,46 +41,46 @@ ****************************************************************************/ /**************************************************************************** - * Name: fork + * Name: up_fork * * Description: - * The up_fork() function is the base of fork() function that provided in - * libc, and fork() is implemented as a wrapper of up_fork() function. - * The fork() function has the same effect as posix fork(), except that the - * behavior is undefined if the process created by fork() either modifies - * any data other than a variable of type pid_t used to store the return - * value from fork(), or returns from the function in which fork() was - * called, or calls any other function before successfully calling _exit() - * or one of the exec family of functions. + * The architecture-specific entry point of both of NuttX's cloning + * primitives. Both need exactly the same thing from assembly -- a + * snapshot of the caller's registers, stack pointer and return address -- + * and differ only in what the C code then does with it, so there is one + * entry point and the caller's x0 says which primitive was called. It is + * saved into the snapshot along with the other argument registers, which + * is harmless: x0-x18 are caller-saved and arm64_fork() does not + * propagate them to the child. * - * This thin layer implements fork by simply calling up_fork() with the - * fork() context as an argument. The overall sequence is: + * The overall sequence is: * - * 1) User code calls fork(). fork() collects context information and - * transfers control up up_fork(). - * 2) arm64_fork() and calls nxtask_setup_fork(). - * 3) nxtask_setup_fork() allocates and configures the child task's TCB. - * This consists of: + * 1) User code calls vfork() or fork(). Both are libc wrappers around + * this entry point. + * 2) The entry point collects the context and calls arm64_fork(). + * 3) arm64_fork() calls nxtask_setup_fork(), which allocates and + * configures the child task's TCB. This consists of: * - Allocation of the child task's TCB. * - Initialization of file descriptors and streams * - Configuration of environment variables - * - Allocate and initialize the stack + * - Establishing the child's address environment + * - Allocating the stack, or inheriting the parent's for fork() * - Setup the input parameters for the task. * - Initialization of the TCB (including call to up_initial_state()) - * 4) arm64_fork() provides any additional operating context. arm64_fork must: + * 4) arm64_fork() provides any additional operating context: * - Initialize special values in any CPU registers that were not * already configured by up_initial_state() + * - Relocate the copied stack, unless the child shares the parent's * 5) arm64_fork() then calls nxtask_start_fork() - * 6) nxtask_start_fork() then executes the child thread. + * 6) which executes the child thread. * * Input Parameters: - * None + * x0 - true for vfork(), false for fork() * * Returned Value: - * Upon successful completion, fork() returns 0 to the child process and - * returns the process ID of the child process to the parent process. - * Otherwise, -1 is returned to the parent, no child process is created, - * and errno is set to indicate the error. + * Upon successful completion, 0 is returned to the child and the process + * ID of the child is returned to the parent. Otherwise, -1 is returned to + * the parent, no child is created, and errno is set to indicate the error. * ****************************************************************************/ @@ -122,10 +122,13 @@ SECTION_FUNC(text, up_fork) ldp x0, x30, [sp], #16 #endif - /* Then, call arm64_fork(), passing it a pointer to the stack structure */ + /* Then, call arm64_fork(), passing it the vfork flag and a pointer to + * the stack structure. The flag comes back out of the snapshot: the + * sequence above clobbers x0, but it saved it first. + */ - mov x0, sp - mov x1, #0 + ldr x0, [sp, #8 * FORK_REG_X0] + mov x1, sp bl arm64_fork /* Release the stack data and return the value returned by arm64_fork */ diff --git a/arch/ceva/src/common/ceva_fork.c b/arch/ceva/src/common/ceva_fork.c index 2eca6db1b76..7cf6bd684ff 100644 --- a/arch/ceva/src/common/ceva_fork.c +++ b/arch/ceva/src/common/ceva_fork.c @@ -50,11 +50,16 @@ * called, or calls any other function before successfully calling _exit() * or one of the exec family of functions. * + * Those are vfork()'s semantics, and vfork() is all this architecture can + * provide: POSIX fork() needs an address environment to duplicate and + * CEVA has none. So up_fork()'s argument is always true here, and the + * entry point does not carry it through the context-saving trap. + * * The overall sequence is: * * 1) User code calls fork(). fork() collects context information and * transfers control up ceva_fork(). - * 2) ceva_fork()and calls nxtask_forksetup(). + * 2) ceva_fork() and calls nxtask_forksetup(). * 3) nxtask_setup_fork() allocates and configures the child task's TCB. * This consists of: * - Allocation of the child task's TCB. @@ -73,7 +78,7 @@ * nxtask_abort_fork() may be called if an error occurs between steps 3 & 6. * * Input Parameters: - * regs - Caller context information saved by fork() + * regs - Caller context information saved by up_fork() * * Return: * Upon successful completion, fork() returns 0 to the child process and @@ -97,9 +102,14 @@ pid_t ceva_fork(const uint32_t *regs) void *argv; int ret; + /* How large is the parent's stack argument area? */ + + argsize = (uintptr_t)parent->stack_base_ptr - + (uintptr_t)parent->stack_alloc_ptr; + /* Allocate and initialize a TCB for the child task. */ - child = nxtask_setup_fork(parent->start, &argsize); + child = nxtask_setup_fork(parent->start, true); if (!child) { serr("ERROR: nxtask_setup_fork failed\n"); @@ -204,7 +214,7 @@ pid_t ceva_fork(const uint32_t *regs) * will discard the TCB by calling nxtask_abort_fork(). */ - return nxtask_start_fork(child); + return nxtask_start_fork(child, true); #else /* CONFIG_SCHED_WAITPID */ return (pid_t)ERROR; #endif diff --git a/arch/mips/Kconfig b/arch/mips/Kconfig index 1d1f2a2965d..6efaa22ba23 100644 --- a/arch/mips/Kconfig +++ b/arch/mips/Kconfig @@ -56,7 +56,7 @@ endchoice config ARCH_MIPS32 bool default n - select ARCH_HAVE_FORK + select ARCH_HAVE_VFORK config ARCH_MIPS_M4K bool diff --git a/arch/mips/src/mips32/Kconfig b/arch/mips/src/mips32/Kconfig index 16fa3481654..c30962ed7b2 100644 --- a/arch/mips/src/mips32/Kconfig +++ b/arch/mips/src/mips32/Kconfig @@ -93,7 +93,7 @@ config MIPS32_TOOLCHAIN_MICROCHIP_XC32_LICENSED config MIPS32_FRAMEPOINTER bool "ABI Uses Frame Pointer" default n - depends on ARCH_HAVE_FORK + depends on ARCH_HAVE_VFORK ---help--- Register r30 may be a frame pointer in some ABIs. Or may just be saved register s8. It makes a difference for fork handling. diff --git a/arch/mips/src/mips32/fork.S b/arch/mips/src/mips32/fork.S index 790aa8cd43f..be2b263a335 100644 --- a/arch/mips/src/mips32/fork.S +++ b/arch/mips/src/mips32/fork.S @@ -47,20 +47,17 @@ * Name: up_fork * * Description: - * The up_fork() function is the base of fork() function that provided in - * libc, and fork() is implemented as a wrapper of up_fork() function. - * The fork() function has the same effect as posix fork(), except that the behavior is - * undefined if the process created by fork() either modifies any data other than - * a variable of type pid_t used to store the return value from fork(), or returns - * from the function in which fork() was called, or calls any other function before - * successfully calling _exit() or one of the exec family of functions. + * The architecture-specific entry point of both of NuttX's cloning + * primitives, vfork() and fork(). Both need exactly the same snapshot from + * assembly and differ only in what the C code then does with it, so there + * is one entry point and the caller's $a0 says which primitive was called. + * It is passed straight through to mips_fork(). * - * This thin layer implements fork by simply calling up_fork() with the fork() - * context as an argument. The overall sequence is: + * The overall sequence is: * - * 1) User code calls fork(). fork() collects context information and - * transfers control up up_fork(). - * 2) mips_fork() and calls nxtask_setup_fork(). + * 1) User code calls vfork() or fork(). Both are libc wrappers around + * this entry point, which collects context information and + * 2) calls mips_fork(), which calls nxtask_setup_fork(). * 3) nxtask_setup_fork() allocates and configures the child task's TCB. This * consists of: * - Allocation of the child task's TCB. @@ -76,7 +73,7 @@ * 6) nxtask_start_fork() then executes the child thread. * * Input Parameters: - * None + * $a0 - true for vfork(), false for fork() * * Returned Value: * Upon successful completion, fork() returns 0 to the child process and returns @@ -88,16 +85,19 @@ .text .align 2 - .globl up_fork - .type up_fork, function .set nomips16 #ifdef CONFIG_MIPS_MICROMIPS .set micromips #endif + + .globl up_fork + .type up_fork, function .ent up_fork up_fork: - /* Create a stack frame */ + /* Create a stack frame. $a0 holds the vfork flag and is not part of the + * snapshot, so it is still there when mips_fork() is called below. + */ move $t0, $sp /* Save the value of the stack on entry */ addiu $sp, $sp, -FORK_SIZEOF /* Allocate the structure on the stack */ @@ -130,9 +130,11 @@ up_fork: /* Floating point registers (not yet) */ - /* Then, call mips_fork(), passing it a pointer to the stack structure */ + /* Then, call mips_fork(), passing it a pointer to the stack structure. + * $a0 already holds the vfork flag. + */ - move $a0, $sp + move $a1, $sp jal mips_fork nop diff --git a/arch/mips/src/mips32/mips_fork.c b/arch/mips/src/mips32/mips_fork.c index 3d931d1da1d..6ae724a52f4 100644 --- a/arch/mips/src/mips32/mips_fork.c +++ b/arch/mips/src/mips32/mips_fork.c @@ -79,7 +79,8 @@ * and 6 * * Input Parameters: - * context - Caller context information saved by fork() + * vfork - true for vfork(), false for fork() + * context - Caller context information saved by up_fork() * * Returned Value: * Upon successful completion, fork() returns 0 to the child process and @@ -89,7 +90,7 @@ * ****************************************************************************/ -pid_t mips_fork(const struct fork_s *context) +pid_t mips_fork(bool vfork, const struct fork_s *context) { struct tcb_s *parent = this_task(); struct tcb_s *child; @@ -113,7 +114,7 @@ pid_t mips_fork(const struct fork_s *context) context->fp, context->sp, context->ra, context->gp); #else sinfo("fp:%08" PRIx32 " sp:%08" PRIx32 " ra:%08" PRIx32 "\n", - context->fp context->sp, context->ra); + context->fp, context->sp, context->ra); #endif #else sinfo("s5:%08" PRIx32 " s6:%08" PRIx32 " s7:%08" PRIx32 @@ -130,7 +131,7 @@ pid_t mips_fork(const struct fork_s *context) /* Allocate and initialize a TCB for the child task. */ - child = nxtask_setup_fork((start_t)context->ra); + child = nxtask_setup_fork((start_t)context->ra, vfork); if (!child) { sinfo("nxtask_setup_fork failed\n"); @@ -217,5 +218,5 @@ pid_t mips_fork(const struct fork_s *context) * will discard the TCB by calling nxtask_abort_fork(). */ - return nxtask_start_fork(child); + return nxtask_start_fork(child, vfork); } diff --git a/arch/risc-v/src/common/CMakeLists.txt b/arch/risc-v/src/common/CMakeLists.txt index 0b9d9a51f8c..e2b8f8bb902 100644 --- a/arch/risc-v/src/common/CMakeLists.txt +++ b/arch/risc-v/src/common/CMakeLists.txt @@ -86,7 +86,7 @@ if(CONFIG_STACK_COLORATION) list(APPEND SRCS riscv_checkstack.c) endif() -if(CONFIG_ARCH_HAVE_FORK) +if(CONFIG_ARCH_HAVE_VFORK OR CONFIG_ARCH_HAVE_FORK) list(APPEND SRCS fork.S riscv_fork.c) endif() diff --git a/arch/risc-v/src/common/Make.defs b/arch/risc-v/src/common/Make.defs index d98a828c508..4b46fc9ccad 100644 --- a/arch/risc-v/src/common/Make.defs +++ b/arch/risc-v/src/common/Make.defs @@ -86,7 +86,7 @@ ifeq ($(CONFIG_STACK_COLORATION),y) CMN_CSRCS += riscv_checkstack.c endif -ifeq ($(CONFIG_ARCH_HAVE_FORK),y) +ifneq ($(CONFIG_ARCH_HAVE_VFORK)$(CONFIG_ARCH_HAVE_FORK),) CMN_ASRCS += fork.S CMN_CSRCS += riscv_fork.c endif diff --git a/arch/risc-v/src/common/fork.S b/arch/risc-v/src/common/fork.S index 108ff00f80d..a516e57fd96 100644 --- a/arch/risc-v/src/common/fork.S +++ b/arch/risc-v/src/common/fork.S @@ -46,46 +46,36 @@ ****************************************************************************/ /**************************************************************************** - * Name: fork + * Name: up_fork * * Description: - * The up_fork() function is the base of fork() function that provided in - * libc, and fork() is implemented as a wrapper of up_fork() function. - * The fork() function has the same effect as posix fork(), except that the - * behavior is undefined if the process created by fork() either modifies - * any data other than a variable of type pid_t used to store the return - * value from fork(), or returns from the function in which fork() was - * called, or calls any other function before successfully calling _exit() - * or one of the exec family of functions. + * The architecture-specific entry point of both of NuttX's cloning + * primitives. Both need exactly the same thing from assembly -- a + * snapshot of the caller's callee-saved registers, stack pointer and + * return address -- and differ only in what the C code then does with it, + * so there is one entry point and the caller's a0 says which primitive was + * called. It is passed straight through to riscv_fork(). * - * This thin layer implements fork by simply calling up_fork() with the - * fork() context as an argument. The overall sequence is: + * The overall sequence is: * - * 1) User code calls fork(). fork() collects context information and - * transfers control up up_fork(). - * 2) riscv_fork() and calls nxtask_setup_fork(). - * 3) nxtask_setup_fork() allocates and configures the child task's TCB. - * This consists of: - * - Allocation of the child task's TCB. - * - Initialization of file descriptors and streams - * - Configuration of environment variables - * - Allocate and initialize the stack - * - Setup the input parameters for the task. - * - Initialization of the TCB (including call to up_initial_state()) - * 4) riscv_fork() provides any additional operating context. riscv_fork must: - * - Initialize special values in any CPU registers that were not - * already configured by up_initial_state() - * 5) riscv_fork() then calls nxtask_start_fork() - * 6) nxtask_start_fork() then executes the child thread. + * 1) User code calls vfork() or fork(). Both are libc wrappers around + * this entry point. + * 2) The entry point collects the context and calls riscv_fork(). + * 3) riscv_fork() calls nxtask_setup_fork(), which allocates and + * configures the child task's TCB. + * 4) riscv_fork() provides any additional operating context and relocates + * the copied stack. + * 5) riscv_fork() then calls nxtask_start_fork(), which for vfork() + * additionally suspends the caller. + * 6) which executes the child thread. * * Input Parameters: - * None + * a0 - true for vfork(), false for fork() * * Returned Value: - * Upon successful completion, fork() returns 0 to the child process and - * returns the process ID of the child process to the parent process. - * Otherwise, -1 is returned to the parent, no child process is created, - * and errno is set to indicate the error. + * Upon successful completion, 0 is returned to the child and the process + * ID of the child is returned to the parent. Otherwise, -1 is returned to + * the parent, no child is created, and errno is set to indicate the error. * ****************************************************************************/ @@ -94,7 +84,9 @@ up_fork: #ifdef CONFIG_LIB_SYSCALL - /* When coming via system call, everything is in place already */ + /* When coming via system call, everything is in place already: a0 already + * holds the vfork flag, and riscv_fork() takes its snapshot from the TCB. + */ tail riscv_fork #else @@ -129,8 +121,8 @@ up_fork: REGSTORE gp, FORK_GP_OFFSET(sp) #endif - addi a0, sp, FORK_SIZEOF - REGSTORE a0, FORK_SP_OFFSET(sp) /* original SP */ + addi a2, sp, FORK_SIZEOF + REGSTORE a2, FORK_SP_OFFSET(sp) /* original SP */ REGSTORE x1, FORK_RA_OFFSET(sp) /* return address */ /* Floating point registers */ @@ -150,9 +142,11 @@ up_fork: FSTORE fs11, FORK_FS11_OFFSET(sp) #endif - /* Then, call riscv_fork(), passing it a pointer to the stack frame */ + /* Then, call riscv_fork(). a0 still holds the vfork flag: nothing above + * touches it. + */ - mv a0, sp + mv a1, sp call riscv_fork /* Release the stack frame and return the value returned by riscv_fork */ diff --git a/arch/risc-v/src/common/riscv_fork.c b/arch/risc-v/src/common/riscv_fork.c index f25e5cf4e40..10757629785 100644 --- a/arch/risc-v/src/common/riscv_fork.c +++ b/arch/risc-v/src/common/riscv_fork.c @@ -41,8 +41,6 @@ #include "sched/sched.h" -#ifdef CONFIG_ARCH_HAVE_FORK - /**************************************************************************** * Pre-processor Definitions ****************************************************************************/ @@ -59,17 +57,15 @@ * Name: riscv_fork * * Description: - * The fork() function has the same effect as posix fork(), except that the - * behavior is undefined if the process created by fork() either modifies - * any data other than a variable of type pid_t used to store the return - * value from fork(), or returns from the function in which fork() was - * called, or calls any other function before successfully calling _exit() - * or one of the exec family of functions. + * The common RISC-V worker behind up_fork(). vfork() and fork() snapshot + * the caller's registers identically; `vfork' says which primitive was + * called, and is passed straight through to nxtask_setup_fork(), which is + * where the memory semantics are decided. * * The overall sequence is: * - * 1) User code calls fork(). fork() collects context information and - * transfers control up riscv_fork(). + * 1) User code calls vfork() or fork(). up_fork() collects context + * information and transfers control to riscv_fork(). * 2) riscv_fork() and calls nxtask_setup_fork(). * 3) nxtask_setup_fork() allocates and configures the child task's TCB. * This consists of: @@ -90,7 +86,8 @@ * and 6. * * Input Parameters: - * context - Caller context information saved by fork() + * vfork - true for vfork(), false for fork() + * context - Caller context information saved by up_fork() * * Returned Value: * Upon successful completion, fork() returns 0 to the child process and @@ -102,7 +99,7 @@ #ifdef CONFIG_LIB_SYSCALL -pid_t riscv_fork(const struct fork_s *context) +pid_t riscv_fork(bool vfork, const struct fork_s *context) { struct tcb_s *parent = this_task(); struct tcb_s *child; @@ -117,7 +114,7 @@ pid_t riscv_fork(const struct fork_s *context) /* Allocate and initialize a TCB for the child task. */ - child = nxtask_setup_fork((start_t)parent->xcp.sregs[REG_RA]); + child = nxtask_setup_fork((start_t)parent->xcp.sregs[REG_RA], vfork); if (!child) { sinfo("nxtask_setup_fork failed\n"); @@ -130,12 +127,26 @@ pid_t riscv_fork(const struct fork_s *context) DEBUGASSERT(stacktop > parent->xcp.sregs[REG_SP]); stackutil = stacktop - parent->xcp.sregs[REG_SP]; - /* Copy goes to child's user stack top */ + if (child->stack_base_ptr == parent->stack_base_ptr) + { + /* The child is running at the parent's stack addresses, inside its + * own duplicated address environment. There is nothing to relocate: + * every stack address the child inherits is still the address it + * names. + */ + + newsp = parent->xcp.sregs[REG_SP]; + } + else + { + /* Copy goes to child's user stack top */ - newtop = (uintptr_t)child->stack_base_ptr + child->adj_stack_size; - newsp = newtop - stackutil; + newtop = (uintptr_t)child->stack_base_ptr + child->adj_stack_size; + newsp = newtop - stackutil; - memcpy((void *)newsp, (const void *)parent->xcp.sregs[REG_SP], stackutil); + memcpy((void *)newsp, (const void *)parent->xcp.sregs[REG_SP], + stackutil); + } #ifdef CONFIG_SCHED_THREAD_LOCAL /* Save child's thread pointer */ @@ -184,12 +195,12 @@ pid_t riscv_fork(const struct fork_s *context) * will discard the TCB by calling nxtask_abort_fork(). */ - return nxtask_start_fork(child); + return nxtask_start_fork(child, vfork); } #else -pid_t riscv_fork(const struct fork_s *context) +pid_t riscv_fork(bool vfork, const struct fork_s *context) { struct tcb_s *parent = this_task(); struct tcb_s *child; @@ -215,7 +226,7 @@ pid_t riscv_fork(const struct fork_s *context) context->fp, context->sp, context->ra, context->gp); #else sinfo("fp:%" PRIxREG " sp:%" PRIxREG " ra:%" PRIxREG "\n", - context->fp context->sp, context->ra); + context->fp, context->sp, context->ra); #endif #else sinfo("s5:%" PRIxREG " s6:%" PRIxREG " s7:%" PRIxREG " s8:%" PRIxREG "\n", @@ -231,7 +242,7 @@ pid_t riscv_fork(const struct fork_s *context) /* Allocate and initialize a TCB for the child task. */ - child = nxtask_setup_fork((start_t)(uintptr_t)context->ra); + child = nxtask_setup_fork((start_t)(uintptr_t)context->ra, vfork); if (!child) { sinfo("nxtask_setup_fork failed\n"); @@ -252,52 +263,71 @@ pid_t riscv_fork(const struct fork_s *context) sinfo("Parent: stackutil:%" PRIxPTR "\n", stackutil); - /* Make some feeble effort to preserve the stack contents. This is - * feeble because the stack surely contains invalid pointers and other - * content that will not work in the child context. However, if the - * user follows all of the caveats of fork() usage, even this feeble - * effort is overkill. - */ + if (child->stack_base_ptr == parent->stack_base_ptr) + { + /* The child is running at the parent's stack addresses, inside its + * own duplicated address environment. There is nothing to relocate: + * every stack address the child inherits is still the address it + * names. + */ - newtop = (uintptr_t)child->stack_base_ptr + child->adj_stack_size; - newsp = newtop - stackutil; + newsp = (uintptr_t)context->sp; +#ifdef CONFIG_RISCV_FRAMEPOINTER + newfp = (uintptr_t)context->fp; +#endif + } + else + { + /* Make some feeble effort to preserve the stack contents. This is + * feeble because the stack surely contains invalid pointers and other + * content that will not work in the child context. However, if the + * user follows all of the caveats of vfork() usage, even this feeble + * effort is overkill. + * + * For a POSIX fork() child the stack contents are not merely a feeble + * effort: the child is entitled to use them, and it does. + */ - /* Set up frame for context and copy the initial context there */ + newtop = (uintptr_t)child->stack_base_ptr + child->adj_stack_size; + newsp = newtop - stackutil; - memcpy((void *)(newsp - XCPTCONTEXT_SIZE), - child->xcp.regs, XCPTCONTEXT_SIZE); + /* Set up frame for context and copy the initial context there */ - /* Copy the parent stack contents (overwrites child's SP and TP) */ + memcpy((void *)(newsp - XCPTCONTEXT_SIZE), + child->xcp.regs, XCPTCONTEXT_SIZE); - memcpy((void *)newsp, (const void *)(uintptr_t)context->sp, stackutil); + /* Copy the parent stack contents (overwrites child's SP and TP) */ - /* Set the new register restore area to the new stack top */ + memcpy((void *)newsp, (const void *)(uintptr_t)context->sp, stackutil); - child->xcp.regs = (void *)(newsp - XCPTCONTEXT_SIZE); + /* Set the new register restore area to the new stack top */ - /* Was there a frame pointer in place before? */ + child->xcp.regs = (void *)(newsp - XCPTCONTEXT_SIZE); -#ifdef CONFIG_RISCV_FRAMEPOINTER - if (context->fp >= context->sp && context->fp < stacktop) - { - uintptr_t frameutil = stacktop - context->fp; - newfp = newtop - frameutil; - } - else - { - newfp = context->fp; - } + /* Was there a frame pointer in place before? */ - sinfo("Old stack top:%" PRIxPTR " SP:%" PRIxREG " FP:%" PRIxREG "\n", - stacktop, context->sp, context->fp); - sinfo("New stack top:%" PRIxPTR " SP:%" PRIxPTR " FP:%" PRIxPTR "\n", - newtop, newsp, newfp); +#ifdef CONFIG_RISCV_FRAMEPOINTER + if (context->fp >= context->sp && context->fp < stacktop) + { + uintptr_t frameutil = stacktop - context->fp; + newfp = newtop - frameutil; + } + else + { + newfp = context->fp; + } + + sinfo("Old stack top:%" PRIxPTR " SP:%" PRIxREG " FP:%" PRIxREG "\n", + stacktop, context->sp, context->fp); + sinfo("New stack top:%" PRIxPTR " SP:%" PRIxPTR " FP:%" PRIxPTR "\n", + newtop, newsp, newfp); #else - sinfo("Old stack top:%" PRIxPTR " SP:%" PRIxREG "\n", - stacktop, context->sp); - sinfo("New stack top:%" PRIxPTR " SP:%" PRIxPTR "\n", - newtop, newsp); + sinfo("Old stack top:%" PRIxPTR " SP:%" PRIxREG "\n", + stacktop, context->sp); + sinfo("New stack top:%" PRIxPTR " SP:%" PRIxPTR "\n", + newtop, newsp); #endif + } /* Update the stack pointer, frame pointer, global pointer and saved * registers. When the child TCB was initialized, all of the values @@ -346,8 +376,7 @@ pid_t riscv_fork(const struct fork_s *context) * will discard the TCB by calling nxtask_abort_fork(). */ - return nxtask_start_fork(child); + return nxtask_start_fork(child, vfork); } #endif /* CONFIG_LIB_SYSCALL */ -#endif /* CONFIG_ARCH_HAVE_FORK */ diff --git a/arch/sim/src/Makefile b/arch/sim/src/Makefile index 280c3a8b050..cc6ad23c184 100644 --- a/arch/sim/src/Makefile +++ b/arch/sim/src/Makefile @@ -95,7 +95,7 @@ ifeq ($(CONFIG_SCHED_BACKTRACE),y) CSRCS += sim_backtrace.c endif -ifeq ($(CONFIG_ARCH_HAVE_FORK),y) +ifneq ($(CONFIG_ARCH_HAVE_VFORK)$(CONFIG_ARCH_HAVE_FORK),) CSRCS += sim_fork.c endif diff --git a/arch/sim/src/sim/CMakeLists.txt b/arch/sim/src/sim/CMakeLists.txt index 4adcc17ba76..cc5f4845e4e 100644 --- a/arch/sim/src/sim/CMakeLists.txt +++ b/arch/sim/src/sim/CMakeLists.txt @@ -82,7 +82,7 @@ if(CONFIG_SCHED_BACKTRACE) list(APPEND SRCS sim_backtrace.c) endif() -if(CONFIG_ARCH_HAVE_FORK) +if(CONFIG_ARCH_HAVE_VFORK OR CONFIG_ARCH_HAVE_FORK) list(APPEND SRCS sim_fork.c) endif() diff --git a/arch/sim/src/sim/sim_fork.c b/arch/sim/src/sim/sim_fork.c index 47ea655cbb6..72ff6c98a7e 100644 --- a/arch/sim/src/sim/sim_fork.c +++ b/arch/sim/src/sim/sim_fork.c @@ -48,17 +48,15 @@ * Name: sim_fork * * Description: - * The fork() function has the same effect as posix fork(), except that the - * behavior is undefined if the process created by fork() either modifies - * any data other than a variable of type pid_t used to store the return - * value from fork(), or returns from the function in which fork() was - * called, or calls any other function before successfully calling _exit() - * or one of the exec family of functions. + * The common simulator worker behind up_fork(). vfork() and fork() + * snapshot the caller's registers identically; `vfork' says which + * primitive was called, and is passed straight through to + * nxtask_setup_fork(), which is where the memory semantics are decided. * * The overall sequence is: * - * 1) User code calls fork(). fork() collects context information and - * transfers control up sim_fork(). + * 1) User code calls vfork() or fork(). up_fork() collects context + * information and transfers control to sim_fork(). * 2) sim_fork() and calls nxtask_setup_fork(). * 3) nxtask_setup_fork() allocates and configures the child task's TCB. * This consists of: @@ -77,6 +75,10 @@ * nxtask_abort_fork() may be called if an error occurs between steps 3 and * 6. * + * Input Parameters: + * vfork - true for vfork(), false for fork() + * context - Caller context information saved by up_fork() + * * Returned Value: * Upon successful completion, fork() returns 0 to the child process and * returns the process ID of the child process to the parent process. @@ -88,7 +90,7 @@ #ifdef CONFIG_SIM_ASAN nosanitize_address #endif -pid_t sim_fork(const xcpt_reg_t *context) +pid_t sim_fork(bool vfork, const xcpt_reg_t *context) { struct tcb_s *parent = this_task(); struct tcb_s *child; @@ -106,7 +108,7 @@ pid_t sim_fork(const xcpt_reg_t *context) /* Allocate and initialize a TCB for the child task. */ - child = nxtask_setup_fork((start_t)context[JB_PC]); + child = nxtask_setup_fork((start_t)context[JB_PC], vfork); if (!child) { serr("ERROR: nxtask_setup_fork failed\n"); @@ -175,5 +177,5 @@ pid_t sim_fork(const xcpt_reg_t *context) * will discard the TCB by calling nxtask_abort_fork(). */ - return nxtask_start_fork(child); + return nxtask_start_fork(child, vfork); } diff --git a/arch/sim/src/sim/sim_fork_arm.S b/arch/sim/src/sim/sim_fork_arm.S index ee39f54e003..8449c691bea 100644 --- a/arch/sim/src/sim/sim_fork_arm.S +++ b/arch/sim/src/sim/sim_fork_arm.S @@ -46,20 +46,20 @@ * Name: up_fork * * Description: - * The up_fork() function is the base of fork() function that provided in - * libc, and fork() is implemented as a wrapper of up_fork() function. - * The fork() function has the same effect as posix fork(), except that the behavior is - * undefined if the process created by fork() either modifies any data other than - * a variable of type pid_t used to store the return value from fork(), or returns - * from the function in which fork() was called, or calls any other function before - * successfully calling _exit() or one of the exec family of functions. + * The architecture-specific entry point of both of NuttX's cloning + * primitives, vfork() and fork(). The caller says which one it is, and + * the flag is passed straight through to sim_fork(). * - * This thin layer implements fork by simply calling up_fork() with the fork() - * context as an argument. The overall sequence is: + * On the simulator the caller's context is captured with setjmp() rather + * than by hand, and the child re-enters through longjmp() -- which is why + * the entry point tests setjmp()'s return value to tell which of the two + * returns it is on. * - * 1) User code calls fork(). fork() collects context information and - * transfers control up sim_fork(). - * 2) sim_fork() and calls nxtask_setup_fork(). + * The overall sequence is: + * + * 1) User code calls vfork() or fork(). Both are libc wrappers around + * this entry point, which collects context information and + * 2) calls sim_fork(), which calls nxtask_setup_fork(). * 3) nxtask_setup_fork() allocates and configures the child task's TCB. This * consists of: * - Allocation of the child task's TCB. @@ -75,7 +75,7 @@ * 6) nxtask_start_fork() then executes the child thread. * * Input Parameters: - * None + * r0 - true for vfork(), false for fork() * * Returned Value: * Upon successful completion, fork() returns 0 to the child process and returns @@ -86,18 +86,28 @@ ************************************************************************************/ .text + .globl up_fork .type up_fork, @function up_fork: + /* r4 is callee-saved, so it carries the vfork flag across setjmp() */ + + push {r4, lr} + mov r4, r0 + sub sp, sp, #XCPTCONTEXT_SIZE mov r0, sp bl setjmp subs r0, #1 - jz child + beq 1f + + mov r0, r4 + mov r1, sp bl sim_fork -child: +1: add sp, sp, #XCPTCONTEXT_SIZE - ret + pop {r4, lr} + bx lr .size up_fork, . - up_fork .end diff --git a/arch/sim/src/sim/sim_fork_arm64.S b/arch/sim/src/sim/sim_fork_arm64.S index 5d813822fa6..3a88939f6bf 100644 --- a/arch/sim/src/sim/sim_fork_arm64.S +++ b/arch/sim/src/sim/sim_fork_arm64.S @@ -55,21 +55,20 @@ * Name: up_fork * * Description: - * The up_fork() function is the base of fork() function that provided in - * libc, and fork() is implemented as a wrapper of up_fork() function. - * The fork() function has the same effect as posix fork(), except that the - * behavior is undefined if the process created by fork() either modifies - * any data other than a variable of type pid_t used to store the return - * value from fork(), or returns from the function in which fork() was - * called, or calls any other function before successfully calling _exit() - * or one of the exec family of functions. + * The architecture-specific entry point of both of NuttX's cloning + * primitives, vfork() and fork(). The caller says which one it is, and + * the flag is passed straight through to sim_fork(). * - * This thin layer implements fork by simply calling sim_fork() with the - * fork() context as an argument. The overall sequence is: + * On the simulator the caller's context is captured with setjmp() rather + * than by hand, and the child re-enters through longjmp() -- which is why + * the entry point tests setjmp()'s return value to tell which of the two + * returns it is on. * - * 1) User code calls fork(). fork() collects context information and - * transfers control up sim_fork(). - * 2) sim_fork() and calls nxtask_setup_fork(). + * The overall sequence is: + * + * 1) User code calls vfork() or fork(). Both are libc wrappers around + * this entry point, which collects context information and + * 2) calls sim_fork(), which calls nxtask_setup_fork(). * 3) nxtask_setup_fork() allocates and configures the child task's TCB. * This consists of: * - Allocation of the child task's TCB. @@ -85,7 +84,7 @@ * 6) nxtask_start_fork() then executes the child thread. * * Input Parameters: - * None + * x0 - true for vfork(), false for fork() * * Returned Value: * Upon successful completion, fork() returns 0 to the child process and @@ -96,27 +95,35 @@ ***************************************************************************/ .text - .globl SYMBOL(up_fork) .align 4 + .globl SYMBOL(up_fork) + SYMBOL(up_fork): stp x29, x30, [sp] /* save FP/LR register */ - sub sp, sp, #XCPTCONTEXT_SIZE /* area from stack for setjmp() */ - mov x0, sp /* pass stack area to setjmp() */ + /* Area from stack for setjmp(), plus a slot below it holding the vfork + * flag: setjmp() is allowed to clobber every argument register, so the + * flag cannot simply stay in one. + */ + + sub sp, sp, #XCPTCONTEXT_SIZE+16 + str x0, [sp] + + add x0, sp, #16 /* pass stack area to setjmp() */ bl SYMBOL(setjmp) /* save register for longjmp() */ subs x0, x0, #1 /* 0: parent / 1: child */ cbz x0, 1f /* child --> return */ - mov x0, sp /* pass stack area to sim_fork() */ + ldr x0, [sp] /* the vfork flag */ + add x1, sp, #16 /* pass stack area to sim_fork() */ bl SYMBOL(sim_fork) /* further process task creation */ 1: - add sp, sp, #XCPTCONTEXT_SIZE /* release area from stack */ + add sp, sp, #XCPTCONTEXT_SIZE+16 /* release area from stack */ ldp x29, x30, [sp] /* restore FP/LR register */ ret - .end diff --git a/arch/sim/src/sim/sim_fork_x86.S b/arch/sim/src/sim/sim_fork_x86.S index ec7486664c5..127dd6b92d7 100644 --- a/arch/sim/src/sim/sim_fork_x86.S +++ b/arch/sim/src/sim/sim_fork_x86.S @@ -54,20 +54,20 @@ * Name: up_fork * * Description: - * The up_fork() function is the base of fork() function that provided in - * libc, and fork() is implemented as a wrapper of up_fork() function. - * The fork() function has the same effect as posix fork(), except that the behavior is - * undefined if the process created by fork() either modifies any data other than - * a variable of type pid_t used to store the return value from fork(), or returns - * from the function in which fork() was called, or calls any other function before - * successfully calling _exit() or one of the exec family of functions. + * The architecture-specific entry point of both of NuttX's cloning + * primitives, vfork() and fork(). The caller says which one it is, and + * the flag is passed straight through to sim_fork(). * - * This thin layer implements fork by simply calling up_fork() with the fork() - * context as an argument. The overall sequence is: + * On the simulator the caller's context is captured with setjmp() rather + * than by hand, and the child re-enters through longjmp() -- which is why + * the entry point tests setjmp()'s return value to tell which of the two + * returns it is on. * - * 1) User code calls fork(). fork() collects context information and - * transfers control up up_fork(). - * 2) sim_fork() and calls nxtask_setup_fork(). + * The overall sequence is: + * + * 1) User code calls vfork() or fork(). Both are libc wrappers around + * this entry point, which collects context information and + * 2) calls sim_fork(), which calls nxtask_setup_fork(). * 3) nxtask_setup_fork() allocates and configures the child task's TCB. This * consists of: * - Allocation of the child task's TCB. @@ -83,7 +83,7 @@ * 6) nxtask_start_fork() then executes the child thread. * * Input Parameters: - * None + * arg0 - true for vfork(), false for fork() * * Returned Value: * Upon successful completion, fork() returns 0 to the child process and returns @@ -94,21 +94,35 @@ ************************************************************************************/ .text + .globl SYMBOL(up_fork) #ifdef __ELF__ .type SYMBOL(up_fork), @function #endif SYMBOL(up_fork): + /* %ebx is callee-saved, so it carries the vfork flag across setjmp() */ + + push %ebx + mov 8(%esp), %ebx + sub $XCPTCONTEXT_SIZE, %esp push %esp call SYMBOL(setjmp) sub $1, %eax - jz child + jz 1f + + /* sim_fork(vfork, context). The context pointer pushed for setjmp() + * is still in place, and is the second argument. + */ + + push %ebx call SYMBOL(sim_fork) -child: + add $4, %esp +1: add $XCPTCONTEXT_SIZE+4, %esp + pop %ebx ret #ifdef __ELF__ .size SYMBOL(up_fork), . - SYMBOL(up_fork) diff --git a/arch/sim/src/sim/sim_fork_x86_64.S b/arch/sim/src/sim/sim_fork_x86_64.S index 85b072acf86..4d1e6412ab5 100644 --- a/arch/sim/src/sim/sim_fork_x86_64.S +++ b/arch/sim/src/sim/sim_fork_x86_64.S @@ -54,20 +54,20 @@ * Name: up_fork * * Description: - * The up_fork() function is the base of fork() function that provided in - * libc, and fork() is implemented as a wrapper of up_fork() function. - * The fork() function has the same effect as posix fork(), except that the behavior is - * undefined if the process created by fork() either modifies any data other than - * a variable of type pid_t used to store the return value from fork(), or returns - * from the function in which fork() was called, or calls any other function before - * successfully calling _exit() or one of the exec family of functions. + * The architecture-specific entry point of both of NuttX's cloning + * primitives, vfork() and fork(). The caller says which one it is, and + * the flag is passed straight through to sim_fork(). * - * This thin layer implements fork by simply calling up_fork() with the fork() - * context as an argument. The overall sequence is: + * On the simulator the caller's context is captured with setjmp() rather + * than by hand, and the child re-enters through longjmp() -- which is why + * the entry point tests setjmp()'s return value to tell which of the two + * returns it is on. * - * 1) User code calls fork(). fork() collects context information and - * transfers control up up_fork(). - * 2) sim_fork() and calls nxtask_setup_fork(). + * The overall sequence is: + * + * 1) User code calls vfork() or fork(). Both are libc wrappers around + * this entry point, which collects context information and + * 2) calls sim_fork(), which calls nxtask_setup_fork(). * 3) nxtask_setup_fork() allocates and configures the child task's TCB. This * consists of: * - Allocation of the child task's TCB. @@ -83,7 +83,7 @@ * 6) nxtask_start_fork() then executes the child thread. * * Input Parameters: - * None + * arg0 - true for vfork(), false for fork() * * Returned Value: * Upon successful completion, fork() returns 0 to the child process and returns @@ -94,26 +94,41 @@ ************************************************************************************/ .text + .globl SYMBOL(up_fork) #ifdef __ELF__ .type SYMBOL(up_fork), @function #endif SYMBOL(up_fork): + push %rbx sub $XCPTCONTEXT_SIZE, %rsp + + /* %rbx is callee-saved, so it carries the vfork flag across setjmp() */ + #ifdef CONFIG_SIM_X8664_MICROSOFT + mov %rcx, %rbx mov %rsp, %rcx #else /* if defined(CONFIG_SIM_X8664_SYSTEMV) */ + mov %rdi, %rbx mov %rsp, %rdi #endif call SYMBOL(setjmp) sub $1, %eax - jz child + jz 1f +#ifdef CONFIG_SIM_X8664_MICROSOFT + mov %rbx, %rcx + mov %rsp, %rdx +#else /* if defined(CONFIG_SIM_X8664_SYSTEMV) */ + mov %rbx, %rdi + mov %rsp, %rsi +#endif call SYMBOL(sim_fork) -child: +1: add $XCPTCONTEXT_SIZE, %rsp + pop %rbx ret #ifdef __ELF__ .size SYMBOL(up_fork), . - SYMBOL(up_fork) diff --git a/arch/x86_64/src/common/CMakeLists.txt b/arch/x86_64/src/common/CMakeLists.txt index 841fa136799..de170a4d739 100644 --- a/arch/x86_64/src/common/CMakeLists.txt +++ b/arch/x86_64/src/common/CMakeLists.txt @@ -34,7 +34,7 @@ set(SRCS x86_64_tcbinfo.c x86_64_tlb.c) -if(CONFIG_ARCH_HAVE_FORK) +if(CONFIG_ARCH_HAVE_VFORK OR CONFIG_ARCH_HAVE_FORK) list(APPEND SRCS x86_64_fork.c fork.S) endif() diff --git a/arch/x86_64/src/common/Make.defs b/arch/x86_64/src/common/Make.defs index a5a21aff0a6..e72813edcda 100644 --- a/arch/x86_64/src/common/Make.defs +++ b/arch/x86_64/src/common/Make.defs @@ -29,7 +29,7 @@ CMN_CSRCS += x86_64_getintstack.c x86_64_initialize.c x86_64_nputs.c CMN_CSRCS += x86_64_modifyreg8.c x86_64_modifyreg16.c x86_64_modifyreg32.c CMN_CSRCS += x86_64_switchcontext.c x86_64_tlb.c -ifeq ($(CONFIG_ARCH_HAVE_FORK),y) +ifneq ($(CONFIG_ARCH_HAVE_VFORK)$(CONFIG_ARCH_HAVE_FORK),) CMN_CSRCS += x86_64_fork.c CMN_ASRCS += fork.S endif diff --git a/arch/x86_64/src/common/fork.S b/arch/x86_64/src/common/fork.S index 1621b560474..a1e5396d040 100644 --- a/arch/x86_64/src/common/fork.S +++ b/arch/x86_64/src/common/fork.S @@ -39,21 +39,18 @@ * Name: up_fork * * Description: - * The up_fork() function is the base of fork() function that provided in - * libc, and fork() is implemented as a wrapper of up_fork() function. - * The fork() function has the same effect as posix fork(), except that the - * behavior is undefined if the process created by fork() either modifies - * any data other than a variable of type pid_t used to store the return - * value from fork(), or returns from the function in which fork() was - * called, or calls any other function before successfully calling _exit() - * or one of the exec family of functions. + * The architecture-specific entry point of both of NuttX's cloning + * primitives, vfork() and fork(). Both need exactly the same thing from + * assembly -- a snapshot of the caller's registers, stack pointer and + * return address -- and differ only in what the C code then does with it, + * so there is one entry point and the caller's %rdi says which primitive + * was called. It is passed straight through to x86_64_fork(). * - * This thin layer implements fork by simply calling up_fork() with the - * fork() context as an argument. The overall sequence is: + * The overall sequence is: * - * 1) User code calls fork(). fork() collects context information and - * transfers control up up_fork(). - * 2) x86_64_fork() and calls nxtask_setup_fork(). + * 1) User code calls vfork() or fork(). Both are libc wrappers around + * this entry point, which collects context information and + * 2) calls x86_64_fork(), which calls nxtask_setup_fork(). * 3) nxtask_setup_fork() allocates and configures the child task's TCB. * This consists of: * - Allocation of the child task's TCB. @@ -69,7 +66,7 @@ * 6) nxtask_start_fork() then executes the child thread. * * Input Parameters: - * None + * %rdi - true for vfork(), false for fork() * * Returned Value: * Upon successful completion, fork() returns 0 to the child process and @@ -91,7 +88,7 @@ * | [r15] | i * | [r14] | n * | [r13] | g - * | [r12] | <- rsp before calling x86_64_fork, rdi = rsp + * | [r12] | <- rsp before calling x86_64_fork, rsi = rsp * | ......... | */ @@ -99,13 +96,17 @@ .type up_fork, @function up_fork: + /* %rdi still holds the vfork flag on entry and is left alone; %rdx is + * the scratch used to push %ss and %cs. + */ + movq %rsp, %rax addq $8, %rax - movq %ss, %rdi - pushq %rdi + movq %ss, %rdx + pushq %rdx pushfq - movq %cs, %rdi - pushq %rdi + movq %cs, %rdx + pushq %rdx /* push %rsp */ @@ -116,7 +117,7 @@ up_fork: pushq %r14 pushq %r13 pushq %r12 - movq %rsp, %rdi + movq %rsp, %rsi subq $8, %rsp diff --git a/arch/x86_64/src/common/x86_64_fork.c b/arch/x86_64/src/common/x86_64_fork.c index ea4d90ac044..05cd26016e8 100644 --- a/arch/x86_64/src/common/x86_64_fork.c +++ b/arch/x86_64/src/common/x86_64_fork.c @@ -79,7 +79,8 @@ * 6. * * Input Parameters: - * context - Caller context information saved by fork() + * vfork - true for vfork(), false for fork() + * context - Caller context information saved by up_fork() * * Returned Value: * Upon successful completion, fork() returns 0 to the child process and @@ -89,7 +90,7 @@ * ****************************************************************************/ -pid_t x86_64_fork(const struct fork_s *context) +pid_t x86_64_fork(bool vfork, const struct fork_s *context) { struct tcb_s *parent = this_task(); struct tcb_s *child; @@ -110,7 +111,7 @@ pid_t x86_64_fork(const struct fork_s *context) /* Allocate and initialize a TCB for the child task. */ - child = nxtask_setup_fork((start_t)context->rip); + child = nxtask_setup_fork((start_t)context->rip, vfork); if (!child) { serr("ERROR: nxtask_setup_fork failed\n"); @@ -133,44 +134,59 @@ pid_t x86_64_fork(const struct fork_s *context) sinfo("Parent: stackutil:%" PRIu64 "\n", stackutil); - /* Make some feeble effort to preserve the stack contents. This is - * feeble because the stack surely contains invalid pointers and other - * content that will not work in the child context. However, if the - * user follows all of the caveats of fork() usage, even this feeble - * effort is overkill. - */ - - newtop = (uint64_t)XCP_ALIGN_DOWN((uintptr_t)child->stack_base_ptr + - child->adj_stack_size - - XCPTCONTEXT_SIZE); - - newsp = newtop - stackutil; - - /* Move the register context (from parent) to newtop. */ + /* Move the register context (from parent) to the child. */ memcpy(child->xcp.regs, parent->xcp.regs, XCPTCONTEXT_SIZE); - memcpy((void *)newsp, (const void *)context->rsp, stackutil); - - /* Was there a frame pointer in place before? */ - - if (context->rbp >= context->rsp && context->rbp < stacktop) + if (child->stack_base_ptr == parent->stack_base_ptr) { - uint32_t frameutil = stacktop - context->rbp; - newfp = newtop - frameutil; + /* The child is running at the parent's stack addresses, inside its + * own duplicated address environment. There is nothing to relocate: + * every stack address the child inherits is still the address it + * names. + */ + + newsp = context->rsp; + newfp = context->rbp; } else { - newfp = context->rbp; + /* Make some feeble effort to preserve the stack contents. This is + * feeble because the stack surely contains invalid pointers and other + * content that will not work in the child context. However, if the + * user follows all of the caveats of vfork() usage, even this feeble + * effort is overkill. + * + * For a POSIX fork() child the stack contents are not merely a feeble + * effort: the child is entitled to use them, and it does. + */ + + newtop = (uint64_t)XCP_ALIGN_DOWN((uintptr_t)child->stack_base_ptr + + child->adj_stack_size - + XCPTCONTEXT_SIZE); + + newsp = newtop - stackutil; + + memcpy((void *)newsp, (const void *)context->rsp, stackutil); + + /* Was there a frame pointer in place before? */ + + if (context->rbp >= context->rsp && context->rbp < stacktop) + { + uint32_t frameutil = stacktop - context->rbp; + newfp = newtop - frameutil; + } + else + { + newfp = context->rbp; + } + + sinfo("Old stack top:%08" PRIx64 " RSP:%08" PRIx64 + " RBP:%08" PRIx64 "\n", stacktop, context->rsp, context->rbp); + sinfo("New stack top:%08" PRIx64 " RSP:%08" PRIx64 "\n", + newtop, newsp); } - /* We do not need to update the frame-pointer */ - - sinfo("Old stack top:%08" PRIx64 " RSP:%08" PRIx64 " RBP:%08" PRIx64 "\n", - stacktop, context->rsp, context->rbp); - sinfo("New stack top:%08" PRIx64 " RSP:%08" PRIx64 "\n", - newtop, newsp); - /* Update the stack pointer, frame pointer, and volatile registers. When * the child TCB was initialized, all of the values were set to zero. * up_initial_state() altered a few values, but the return value in RAX @@ -195,5 +211,5 @@ pid_t x86_64_fork(const struct fork_s *context) * will discard the TCB by calling nxtask_abort_fork(). */ - return nxtask_start_fork(child); + return nxtask_start_fork(child, vfork); } diff --git a/include/nuttx/addrenv.h b/include/nuttx/addrenv.h index 8e253c88eee..91c2eaa28d4 100644 --- a/include/nuttx/addrenv.h +++ b/include/nuttx/addrenv.h @@ -394,6 +394,31 @@ int addrenv_attach(FAR struct tcb_s *tcb, FAR struct addrenv_s *addrenv); int addrenv_join(FAR struct tcb_s *ptcb, FAR struct tcb_s *tcb); +/**************************************************************************** + * Name: addrenv_fork + * + * Description: + * Duplicate the parent's address environment for a POSIX fork() child and + * attach it: the child gets its own pages holding a copy of the parent's + * contents, mapped at the same virtual addresses. Contrast + * addrenv_join(), which gives the child the parent's memory. + * + * Input Parameters: + * ptcb - The tcb of the parent process. + * tcb - The tcb of the child process. + * + * Returned Value: + * This is a NuttX internal function so it follows the convention that + * 0 (OK) is returned on success and a negated errno is returned on + * failure. -ENOMEM is returned if there is not enough free memory to + * hold a copy of the parent. + * + ****************************************************************************/ + +#ifdef CONFIG_ARCH_HAVE_FORK +int addrenv_fork(FAR struct tcb_s *ptcb, FAR struct tcb_s *tcb); +#endif + /**************************************************************************** * Name: addrenv_leave * diff --git a/include/nuttx/arch.h b/include/nuttx/arch.h index e1a598c222b..91492fefe9c 100644 --- a/include/nuttx/arch.h +++ b/include/nuttx/arch.h @@ -253,8 +253,18 @@ extern initializer_t _einit[]; * Name: up_fork * * Description: - * The up_fork() function is the base of fork() function that provided in - * libc, and fork() is implemented as a wrapper of up_fork() function. + * Architecture-specific base of both cloning primitives. It snapshots the + * caller's registers and hands them to the common code, which builds the + * child from them; `vfork' says which primitive was called and so which + * memory semantics the child gets. + * + * Input Parameters: + * vfork - true for vfork(): the child shares the parent's memory and the + * parent is suspended until the child _exit()s or exec()s. + * false for POSIX fork(): the child receives its own copy of the + * parent's memory at the same virtual addresses and runs + * concurrently. Only available where CONFIG_ARCH_HAVE_FORK is + * selected. * * Returned Value: * Upon successful completion, up_fork() returns 0 to the child process @@ -264,7 +274,9 @@ extern initializer_t _einit[]; * ****************************************************************************/ -pid_t up_fork(void); +#if defined(CONFIG_ARCH_HAVE_VFORK) || defined(CONFIG_ARCH_HAVE_FORK) +pid_t up_fork(bool vfork); +#endif /**************************************************************************** * Name: up_initialize @@ -1327,6 +1339,35 @@ int up_addrenv_clone(FAR const arch_addrenv_t *src, FAR arch_addrenv_t *dest); #endif +/**************************************************************************** + * Name: up_addrenv_fork + * + * Description: + * Duplicate an address environment for POSIX fork(): allocate fresh + * pages for the destination, copy the source's contents into them, and map + * them at the same virtual addresses. Unlike up_addrenv_clone(), which + * copies only the representation and leaves both pointing at the same page + * tables, the result is independent of the source. + * + * Implemented only where CONFIG_ARCH_HAVE_FORK is selected. + * + * Input Parameters: + * src - The address environment to be duplicated. + * dest - The location to receive the duplicate. It is wiped by this + * function before anything is allocated into it. + * + * Returned Value: + * Zero (OK) on success; a negated errno value on failure. -ENOMEM is + * returned if there are not enough free pages to hold the copy, in which + * case nothing is left allocated. + * + ****************************************************************************/ + +#ifdef CONFIG_ARCH_HAVE_FORK +int up_addrenv_fork(FAR const arch_addrenv_t *src, + FAR arch_addrenv_t *dest); +#endif + /**************************************************************************** * Name: up_addrenv_attach * diff --git a/include/nuttx/sched.h b/include/nuttx/sched.h index 44addd4d57e..c2f6df13982 100644 --- a/include/nuttx/sched.h +++ b/include/nuttx/sched.h @@ -652,6 +652,14 @@ struct tcb_s /* after the frame has been */ /* removed from the stack. */ + /* vfork() Support ********************************************************/ + +#ifdef CONFIG_ARCH_HAVE_VFORK + FAR sem_t *vfork_rel; /* Non-NULL in a vfork() child: */ + /* the suspended parent to release */ + /* when this task is torn down. */ +#endif + /* External Module Support ************************************************/ #ifdef CONFIG_PIC @@ -1136,23 +1144,35 @@ void nxtask_startup(main_t entrypt, int argc, FAR char *argv[]); #endif /**************************************************************************** - * Internal fork support. The overall sequence is: - * - * 1) User code calls fork(). fork() is provided in architecture-specific - * code. - * 2) fork()and calls nxtask_setup_fork(). + * Internal support for the two cloning primitives, vfork() and fork(). The + * sequence below is common to both, and `vfork' says which one was called: + * + * vfork() the child shares the parent's memory and the parent is + * suspended until the child _exit()s or exec()s. + * fork() the child gets its own copy of the parent's memory at the same + * virtual addresses, and both run. + * + * 1) User code calls vfork() or fork(). Both are libc wrappers around + * up_fork(), which is provided in architecture-specific code. + * 2) The architecture-specific code snapshots the caller's registers and + * calls nxtask_setup_fork(). * 3) nxtask_setup_fork() allocates and configures the child task's TCB. * This consists of: * - Allocation of the child task's TCB. * - Initialization of file descriptors and streams * - Configuration of environment variables - * - Allocate and initialize the stack + * - Establishing the child's address environment: joined to the parent's + * for vfork(), duplicated from it for fork() + * - Allocating the stack, or inheriting the parent's for fork() * - Setup the input parameters for the task. * - Initialization of the TCB (including call to up_initial_state()) - * 4) fork() provides any additional operating context. fork must: + * 4) The architecture-specific code provides any additional operating + * context: * - Initialize special values in any CPU registers that were not * already configured by up_initial_state() - * 5) fork() then calls nxtask_start_fork() + * - Relocate the copied stack, unless the child shares the parent's + * 5) It then calls nxtask_start_fork(), which for vfork() additionally + * suspends the caller. * 6) nxtask_start_fork() then executes the child thread. * * nxtask_abort_fork() may be called if an error occurs between @@ -1160,8 +1180,8 @@ void nxtask_startup(main_t entrypt, int argc, FAR char *argv[]); * ****************************************************************************/ -FAR struct tcb_s *nxtask_setup_fork(start_t retaddr); -pid_t nxtask_start_fork(FAR struct tcb_s *child); +FAR struct tcb_s *nxtask_setup_fork(start_t retaddr, bool vfork); +pid_t nxtask_start_fork(FAR struct tcb_s *child, bool vfork); void nxtask_abort_fork(FAR struct tcb_s *child, int errcode); /**************************************************************************** diff --git a/include/sys/syscall_lookup.h b/include/sys/syscall_lookup.h index 595cded1ce2..242366c8fd6 100644 --- a/include/sys/syscall_lookup.h +++ b/include/sys/syscall_lookup.h @@ -116,8 +116,8 @@ SYSCALL_LOOKUP(nxsem_wait_slow, 1) /* The following can be individually enabled */ -#ifdef CONFIG_ARCH_HAVE_FORK - SYSCALL_LOOKUP(up_fork, 0) +#if defined(CONFIG_ARCH_HAVE_VFORK) || defined(CONFIG_ARCH_HAVE_FORK) + SYSCALL_LOOKUP(up_fork, 1) #endif #ifdef CONFIG_SCHED_WAITPID diff --git a/include/unistd.h b/include/unistd.h index e9d1686248c..885bbed12a5 100644 --- a/include/unistd.h +++ b/include/unistd.h @@ -347,8 +347,17 @@ extern "C" /* Task Control Interfaces */ +/* fork() is declared only where POSIX fork() semantics can be provided, so + * that calling it elsewhere is a build error rather than a silent change of + * meaning. + */ + +#ifdef CONFIG_ARCH_HAVE_FORK pid_t fork(void); +#endif +#ifdef CONFIG_ARCH_HAVE_VFORK pid_t vfork(void); +#endif pid_t getpid(void); pid_t getpgid(pid_t pid); pid_t getpgrp(void); diff --git a/libs/libbuiltin/libgcc/gcov.c b/libs/libbuiltin/libgcc/gcov.c index c73aa620b23..134b6aa79ac 100644 --- a/libs/libbuiltin/libgcc/gcov.c +++ b/libs/libbuiltin/libgcc/gcov.c @@ -468,10 +468,16 @@ void __gcov_execle(void) { } +/* GCC redirects fork() in instrumented code to __gcov_fork(), so this is + * reachable only where unistd.h declares fork() at all. + */ + +#ifdef CONFIG_ARCH_HAVE_FORK pid_t __gcov_fork(void) { return fork(); } +#endif void __gcov_dump(void) { diff --git a/libs/libc/libc.csv b/libs/libc/libc.csv index 4600f5329f9..2e2d13506cf 100644 --- a/libs/libc/libc.csv +++ b/libs/libc/libc.csv @@ -348,6 +348,7 @@ "usleep","unistd.h","","int","useconds_t" "vasprintf","stdio.h","","int","FAR char **","FAR const IPTR char *","va_list" "versionsort","dirent.h","","int","FAR const struct dirent **","FAR const struct dirent **" +"vfork","unistd.h","!defined(CONFIG_BUILD_KERNEL) && defined(CONFIG_ARCH_HAVE_VFORK)","pid_t" "vfprintf","stdio.h","defined(CONFIG_FILE_STREAM)","int","FAR FILE *","FAR const IPTR char *","va_list" "vprintf","stdio.h","","int","FAR const IPTR char *","va_list" "vscanf","stdio.h","defined(CONFIG_FILE_STREAM)","int","FAR const IPTR char *","va_list" diff --git a/libs/libc/unistd/CMakeLists.txt b/libs/libc/unistd/CMakeLists.txt index e57f69c07f1..834d61355ff 100644 --- a/libs/libc/unistd/CMakeLists.txt +++ b/libs/libc/unistd/CMakeLists.txt @@ -105,7 +105,7 @@ if(NOT CONFIG_DISABLE_MOUNTPOINTS) list(APPEND SRCS lib_truncate.c lib_posix_fallocate.c) endif() -if(CONFIG_ARCH_HAVE_FORK) +if(CONFIG_ARCH_HAVE_VFORK OR CONFIG_ARCH_HAVE_FORK) list(APPEND SRCS lib_fork.c) endif() diff --git a/libs/libc/unistd/Make.defs b/libs/libc/unistd/Make.defs index 65e1e7e5f04..d4fbc70879b 100644 --- a/libs/libc/unistd/Make.defs +++ b/libs/libc/unistd/Make.defs @@ -55,7 +55,7 @@ ifneq ($(CONFIG_DISABLE_MOUNTPOINTS),y) CSRCS += lib_truncate.c lib_posix_fallocate.c endif -ifeq ($(CONFIG_ARCH_HAVE_FORK),y) +ifneq ($(CONFIG_ARCH_HAVE_VFORK)$(CONFIG_ARCH_HAVE_FORK),) CSRCS += lib_fork.c endif diff --git a/libs/libc/unistd/lib_fork.c b/libs/libc/unistd/lib_fork.c index 53c2bfe0e9b..84813db3265 100644 --- a/libs/libc/unistd/lib_fork.c +++ b/libs/libc/unistd/lib_fork.c @@ -34,8 +34,6 @@ #include <errno.h> #include <nuttx/debug.h> -#if defined(CONFIG_ARCH_HAVE_FORK) - /**************************************************************************** * Private Functions ****************************************************************************/ @@ -137,27 +135,38 @@ static void atfork_parent(void) ****************************************************************************/ /**************************************************************************** - * Name: fork + * Name: vfork * * Description: - * The fork() function is a wrapper of up_fork() syscall + * The vfork() function is equivalent to fork(), except that the behavior + * is undefined if the process created by vfork() either modifies any data + * other than a variable of type pid_t used to store the return value from + * vfork(), or returns from the function in which vfork() was called, or + * calls any other function before successfully calling _exit() or one of + * the exec family of functions. + * + * The child shares the parent's memory and the parent is suspended until + * the child _exit()s or exec()s. The suspension lives in the kernel, so + * vfork() does not depend on CONFIG_SCHED_WAITPID. Wrapper of the + * up_fork() syscall. * * Returned Value: - * Upon successful completion, fork() returns 0 to the child process and + * Upon successful completion, vfork() returns 0 to the child process and * returns the process ID of the child process to the parent process. * Otherwise, -1 is returned to the parent, no child process is created, * and errno is set to indicate the error. * ****************************************************************************/ -pid_t fork(void) +#ifdef CONFIG_ARCH_HAVE_VFORK +pid_t vfork(void) { pid_t pid; #ifdef CONFIG_PTHREAD_ATFORK atfork_prepare(); #endif - pid = up_fork(); + pid = up_fork(true); #ifdef CONFIG_PTHREAD_ATFORK if (pid == 0) @@ -172,39 +181,38 @@ pid_t fork(void) return pid; } - -#if defined(CONFIG_SCHED_WAITPID) +#endif /* CONFIG_ARCH_HAVE_VFORK */ /**************************************************************************** - * Public Functions - ****************************************************************************/ - -/**************************************************************************** - * Name: vfork + * Name: fork * * Description: - * The vfork() function is implemented based on fork() function, on - * vfork(), the parent task need to wait until the child task is performing - * exec or running finished. + * POSIX fork(). The child receives its own copy of the parent's memory, + * at the same virtual addresses. It may modify anything, call anything, + * return from the function that called fork(), and it runs concurrently + * with the parent. None of vfork()'s restrictions apply. + * + * Provided only where CONFIG_ARCH_HAVE_FORK is selected; elsewhere fork() + * is not declared at all, so calling it is a build error. Wrapper of the + * up_fork() syscall. * * Returned Value: - * Upon successful completion, vfork() returns 0 to the child process and + * Upon successful completion, fork() returns 0 to the child process and * returns the process ID of the child process to the parent process. * Otherwise, -1 is returned to the parent, no child process is created, * and errno is set to indicate the error. * ****************************************************************************/ -pid_t vfork(void) +#ifdef CONFIG_ARCH_HAVE_FORK +pid_t fork(void) { - int status = 0; - int ret; pid_t pid; #ifdef CONFIG_PTHREAD_ATFORK atfork_prepare(); #endif - pid = up_fork(); + pid = up_fork(false); #ifdef CONFIG_PTHREAD_ATFORK if (pid == 0) @@ -217,22 +225,6 @@ pid_t vfork(void) } #endif - if (pid != 0) - { - /* we are in parent task, and we need to wait the child task - * until running finished or performing exec - */ - - ret = waitpid(pid, &status, WNOWAIT); - if (ret < 0) - { - serr("ERROR: waitpid failed: %d\n", get_errno()); - } - } - return pid; } - -#endif /* CONFIG_SCHED_WAITPID */ - #endif /* CONFIG_ARCH_HAVE_FORK */ diff --git a/sched/addrenv/addrenv.c b/sched/addrenv/addrenv.c index 9936a105c77..32e6cba247c 100644 --- a/sched/addrenv/addrenv.c +++ b/sched/addrenv/addrenv.c @@ -27,6 +27,7 @@ #include <nuttx/config.h> #include <assert.h> +#include <errno.h> #include <nuttx/debug.h> #include <nuttx/addrenv.h> @@ -292,6 +293,70 @@ int addrenv_join(FAR struct tcb_s *ptcb, FAR struct tcb_s *tcb) return OK; } +#ifdef CONFIG_ARCH_HAVE_FORK +/**************************************************************************** + * Name: addrenv_fork + * + * Description: + * Duplicate the parent process's address environment for a POSIX fork() + * child, and attach the duplicate to the child. + * + * This is the counterpart of addrenv_join(): where join gives the child + * the parent's memory, fork gives it a copy -- its own pages, holding a + * snapshot of the parent's contents, mapped at the same virtual addresses. + * Mapping at the same addresses is what lets the copy be exact: every + * pointer the parent held into its own memory remains valid in the child, + * including the pointers inside the copied heap's own metadata. + * + * The copy is eager -- there is no copy-on-write, because NuttX has no + * demand paging to build it on -- so forking a large process needs as much + * free memory as the process occupies, and fails with -ENOMEM if that is + * not available. That is the nature of the primitive on this class of + * system, not a defect of this implementation; spawn-heavy code should + * prefer posix_spawn() or vfork(). + * + * Input Parameters: + * ptcb - The tcb of the parent process + * tcb - The tcb of the child process + * + * Returned Value: + * This is a NuttX internal function so it follows the convention that + * 0 (OK) is returned on success and a negated errno is returned on + * failure. + * + ****************************************************************************/ + +int addrenv_fork(FAR struct tcb_s *ptcb, FAR struct tcb_s *tcb) +{ + FAR struct addrenv_s *addrenv; + int ret; + + DEBUGASSERT(ptcb->addrenv_own != NULL); + + addrenv = addrenv_allocate(); + if (addrenv == NULL) + { + return -ENOMEM; + } + + /* Duplicate the parent's regions into freshly allocated pages, mapped at + * the same virtual addresses. + */ + + ret = up_addrenv_fork(&ptcb->addrenv_own->addrenv, &addrenv->addrenv); + if (ret < 0) + { + berr("ERROR: up_addrenv_fork failed: %d\n", ret); + addrenv_drop(addrenv, false); + return ret; + } + + /* Hand the reference taken by addrenv_allocate() to the child */ + + return addrenv_attach(tcb, addrenv); +} +#endif /* CONFIG_ARCH_HAVE_FORK */ + /**************************************************************************** * Name: addrenv_leave * diff --git a/sched/sched/sched.h b/sched/sched/sched.h index 62ed2f72ecd..653d7ebaef6 100644 --- a/sched/sched/sched.h +++ b/sched/sched/sched.h @@ -321,6 +321,16 @@ void nxsched_remove_self(FAR struct tcb_s *rtrtcb); void nxsched_add_blocked(FAR struct tcb_s *btcb, tstate_t task_state); void nxsched_remove_blocked(FAR struct tcb_s *btcb); int nxsched_set_priority(FAR struct tcb_s *tcb, int sched_priority); + +/* Release the vfork() parent suspended on this child, if there is one. + * Called from nxsched_release_tcb(), the last point in the child's life -- + * by which time an exec()ing child has already handed its pid to the + * program it loaded. + */ + +#ifdef CONFIG_ARCH_HAVE_VFORK +void nxtask_resume_vfork(FAR struct tcb_s *child); +#endif #ifndef CONFIG_SMP bool nxsched_merge_pending(void); bool nxsched_reprioritize_rtr(FAR struct tcb_s *tcb, int priority); diff --git a/sched/sched/sched_releasetcb.c b/sched/sched/sched_releasetcb.c index 96b3e736e79..1f9d8fe1293 100644 --- a/sched/sched/sched_releasetcb.c +++ b/sched/sched/sched_releasetcb.c @@ -174,6 +174,15 @@ int nxsched_release_tcb(FAR struct tcb_s *tcb, uint8_t ttype) nxtask_joindestroy(tcb); #endif +#ifdef CONFIG_ARCH_HAVE_VFORK + /* Release a suspended vfork() parent here, the last point in the + * child's life: exec_swap() has already handed its pid to any + * program it loaded. + */ + + nxtask_resume_vfork(tcb); +#endif + /* And, finally, release the TCB itself */ if (tcb->flags & TCB_FLAG_FREE_TCB) diff --git a/sched/task/CMakeLists.txt b/sched/task/CMakeLists.txt index fdc19fdc589..7f6817e3dfc 100644 --- a/sched/task/CMakeLists.txt +++ b/sched/task/CMakeLists.txt @@ -46,7 +46,7 @@ if(CONFIG_SCHED_HAVE_PARENT) list(APPEND SRCS task_getppid.c task_reparent.c) endif() -if(CONFIG_ARCH_HAVE_FORK) +if(CONFIG_ARCH_HAVE_VFORK OR CONFIG_ARCH_HAVE_FORK) list(APPEND SRCS task_fork.c) endif() diff --git a/sched/task/Make.defs b/sched/task/Make.defs index 1fd24403b51..6c5857f6301 100644 --- a/sched/task/Make.defs +++ b/sched/task/Make.defs @@ -30,7 +30,7 @@ ifeq ($(CONFIG_SCHED_HAVE_PARENT),y) CSRCS += task_getppid.c task_reparent.c endif -ifeq ($(CONFIG_ARCH_HAVE_FORK),y) +ifneq ($(CONFIG_ARCH_HAVE_VFORK)$(CONFIG_ARCH_HAVE_FORK),) CSRCS += task_fork.c endif diff --git a/sched/task/task_exit.c b/sched/task/task_exit.c index e79c4f81250..892adfcfda9 100644 --- a/sched/task/task_exit.c +++ b/sched/task/task_exit.c @@ -157,5 +157,19 @@ int nxtask_exit(void) rtcb->lockcount--; + /* Publish anything woken while the TCB was being released. lockcount was + * raised directly rather than through sched_lock(), so the matching + * decrement above does not publish the way sched_unlock() would, and a + * vfork() parent released by nxsched_release_tcb() would be stranded -- + * in g_pendingtasks, or in g_readytorun on SMP. This mirrors what + * sched_unlock() does for each case. + */ + +#ifdef CONFIG_SMP + nxsched_deliver_task(this_cpu(), rtcb->cpu, SWITCH_HIGHER); +#else + nxsched_merge_pending(); +#endif + return ret; } diff --git a/sched/task/task_fork.c b/sched/task/task_fork.c index 5e2b1337442..ca7ff728636 100644 --- a/sched/task/task_fork.c +++ b/sched/task/task_fork.c @@ -34,7 +34,9 @@ #include <errno.h> #include <nuttx/debug.h> +#include <nuttx/kmalloc.h> #include <nuttx/queue.h> +#include <nuttx/semaphore.h> #include "sched/sched.h" #include "environ/environ.h" @@ -42,9 +44,101 @@ #include "task/task.h" #include "tls/tls.h" -/* fork() requires architecture-specific support as well as waipid(). */ +/**************************************************************************** + * Private Function Prototypes + ****************************************************************************/ -#ifdef CONFIG_ARCH_HAVE_FORK +#if defined(CONFIG_ARCH_ADDRENV) && defined(CONFIG_ARCH_HAVE_FORK) +static void fork_inherit_stack(FAR struct tcb_s *parent, + FAR struct tcb_s *child); +static void fork_inherit_tls(FAR struct tcb_s *child); +static void fork_restore_parent_env(void); +#endif + +/**************************************************************************** + * Private Functions + ****************************************************************************/ + +#if defined(CONFIG_ARCH_ADDRENV) && defined(CONFIG_ARCH_HAVE_FORK) +/**************************************************************************** + * Name: fork_inherit_stack + * + * Description: + * Give the fork() child the parent's stack at the parent's virtual + * address rather than a relocated copy. The child's address environment + * is a duplicate, so the parent's stack is already there -- same contents, + * same address, its own pages -- and nothing needs allocating or copying. + * + * A relocated stack would break plain C: a pointer to a local taken + * before the fork would name the parent's copy, not the child's live + * object. + * + * TCB_FLAG_FREE_STACK is left clear: the stack belongs to the duplicated + * image and is released with it, so up_release_stack() must not free it. + * + * Input Parameters: + * parent - The parent task's TCB + * child - The child task's TCB + * + ****************************************************************************/ + +static void fork_inherit_stack(FAR struct tcb_s *parent, + FAR struct tcb_s *child) +{ + child->stack_alloc_ptr = parent->stack_alloc_ptr; + child->stack_base_ptr = parent->stack_base_ptr; + child->adj_stack_size = parent->adj_stack_size; + child->flags &= ~TCB_FLAG_FREE_STACK; +} + +/**************************************************************************** + * Name: fork_inherit_tls + * + * Description: + * Retarget the thread-local storage the fork() child inherited. + * + * tls_dup_info() cannot be used: it carves a fresh TLS block off the + * stack, which on an inherited stack would carve a second one and shift + * stack_base_ptr away from the parent's. The child's copy is already in + * place, so only the fields naming the task itself need correcting. + * + * The write lands in user memory at an address the parent also occupies, + * so the child's address environment must be current for it -- otherwise + * the parent's own TLS is what gets modified. + * + * Input Parameters: + * child - The child task's TCB + * + * Returned Value: + * Zero (OK) on success; a negated errno value on failure. + * + ****************************************************************************/ + +static void fork_inherit_tls(FAR struct tcb_s *child) +{ + FAR struct tls_info_s *info = (FAR struct tls_info_s *) + child->stack_alloc_ptr; + + info->tl_task = child->group->tg_info; + info->tl_tid = child->pid; +} + +/**************************************************************************** + * Name: fork_restore_parent_env + * + * Description: + * Undo the addrenv_select() that nxtask_setup_fork() made on the child's + * behalf, putting the caller back in its own address environment. The + * environment to go back to does not have to be remembered: the caller is + * the parent, and what was current before was the parent's own. + * + ****************************************************************************/ + +static void fork_restore_parent_env(void) +{ + addrenv_restore(this_task()->addrenv_own); +} +#endif /* CONFIG_ARCH_ADDRENV && CONFIG_ARCH_HAVE_FORK */ /**************************************************************************** * Public Functions @@ -54,37 +148,22 @@ * Name: nxtask_setup_fork * * Description: - * The fork() function has the same effect as posix fork(), except that the - * behavior is undefined if the process created by fork() either modifies - * any data other than a variable of type pid_t used to store the return - * value from fork(), or returns from the function in which fork() was - * called, or calls any other function before successfully calling _exit() - * or one of the exec family of functions. - * - * This function provides one step in the overall fork() sequence: It - * Allocates and initializes the child task's TCB. The overall sequence - * is: - * - * 1) User code calls fork(). fork() is provided in - * architecture-specific code. - * 2) fork()and calls nxtask_setup_fork(). - * 3) nxtask_setup_fork() allocates and configures the child task's TCB. - * This consists of: - * - Allocation of the child task's TCB. - * - Initialization of file descriptors and streams - * - Configuration of environment variables - * - Allocate and initialize the stack - * - Setup the input parameters for the task. - * - Initialization of the TCB (including call to up_initial_state()) - * 4) up_fork() provides any additional operating context. up_fork must: - * - Initialize special values in any CPU registers that were not - * already configured by up_initial_state() - * 5) up_fork() then calls nxtask_start_fork() - * 6) nxtask_start_fork() then executes the child thread. + * Allocate and initialize the child task's TCB. This is one step in the + * sequence common to vfork() and fork(); see the comment above the + * prototype in include/nuttx/sched.h for the whole sequence and for what + * the two primitives mean. + * + * Exactly two things depend on `vfork': + * + * - the address environment: vfork() joins the parent's, fork() + * duplicates it. + * - the stack: a vfork() child gets its own, which the architecture code + * fills with a relocated copy; a fork() child inherits the parent's + * address (fork_inherit_stack()). * * Input Parameters: - * retaddr - Return address - * argsize - Location to return the argument size + * retaddr - Address at which the child resumes + * vfork - true for vfork(), false for fork() * * Returned Value: * Upon successful completion, nxtask_setup_fork() returns a pointer to @@ -93,7 +172,7 @@ * ****************************************************************************/ -FAR struct tcb_s *nxtask_setup_fork(start_t retaddr) +FAR struct tcb_s *nxtask_setup_fork(start_t retaddr, bool vfork) { FAR struct tcb_s *ptcb = this_task(); FAR struct tcb_s *parent; @@ -160,16 +239,80 @@ FAR struct tcb_s *nxtask_setup_fork(start_t retaddr) } #if defined(CONFIG_ARCH_ADDRENV) - /* Join the parent address environment */ - if (ttype != TCB_FLAG_TTYPE_KERNEL) { - ret = addrenv_join(parent, child); + if (vfork) + { + /* vfork(): join the parent address environment, exactly as + * pthread_create() does. The child shares .data, .bss and the + * heap. + */ + + ret = addrenv_join(parent, child); + } +#ifdef CONFIG_ARCH_HAVE_FORK + else + { + /* POSIX fork(): duplicate the parent's address environment now, + * before anything else is set up. The duplicate holds a copy of + * the parent's contents -- including its stack -- at the parent's + * virtual addresses, which is what lets the child go on to inherit + * the stack address rather than be given a relocated copy. See + * fork_inherit_stack(). + */ + + ret = addrenv_fork(parent, child); + if (ret >= 0) + { + /* Make the child's address environment current for the rest of + * the setup, and for the architecture code that runs after it. + * + * From here on, everything written on the child's behalf + * has to land in the child's image rather than the parent's, + * because + * the two occupy the same virtual addresses: its thread-local + * storage, and -- on architectures that keep the register save + * area on the user stack rather than on a kernel stack -- the + * register context the child is resumed from. Writing those + * under the parent's environment corrupts the parent and + * leaves the child reading whatever the snapshot happened to + * contain. + * + * Reads are unaffected: everything the setup reads from the + * parent -- environ, the argument vector -- is legible at the + * same address in the child, precisely because it is a copy. + * + * nxtask_start_fork() puts the parent's environment back. + */ + + FAR struct addrenv_s *oldenv; + + ret = addrenv_select(child->addrenv_own, &oldenv); + } + } +#else + /* An address environment without ARCH_HAVE_FORK -- a protected build + * over an MMU, for instance. There is an address environment to join, + * but no POSIX fork() to duplicate it for, so the branch above is not + * compiled and `vfork' is always true here. + */ + + DEBUGASSERT(vfork); +#endif + if (ret < 0) { goto errout_with_tcb; } } +#else + /* Without address environments there is only one address space, so + * everything except the stack is shared no matter which primitive was + * called. POSIX fork() cannot be provided at all, and CONFIG_ARCH_HAVE_ + * FORK is not selected, so `vfork' is always true here. + */ + + DEBUGASSERT(vfork); #endif /* Duplicate the parent tasks environment */ @@ -193,12 +336,27 @@ FAR struct tcb_s *nxtask_setup_fork(start_t retaddr) argv = nxsched_get_stackargs(parent); nxtask_setup_name(child, argv[0]); - /* Allocate the stack for the TCB */ + /* Allocate the stack for the TCB, or inherit the parent's */ + +#if defined(CONFIG_ARCH_ADDRENV) && defined(CONFIG_ARCH_HAVE_FORK) + if (!vfork && ttype != TCB_FLAG_TTYPE_KERNEL) + { + /* The child's copy of the parent's stack is already in place, at the + * parent's address, courtesy of the duplication above. + */ + + fork_inherit_stack(parent, child); + ret = OK; + } + else +#endif + { + stack_size = (uintptr_t)ptcb->stack_base_ptr - + (uintptr_t)ptcb->stack_alloc_ptr + ptcb->adj_stack_size; - stack_size = (uintptr_t)ptcb->stack_base_ptr - - (uintptr_t)ptcb->stack_alloc_ptr + ptcb->adj_stack_size; + ret = up_create_stack(child, stack_size, ttype); + } - ret = up_create_stack(child, stack_size, ttype); if (ret < OK) { goto errout_with_tcb; @@ -235,20 +393,35 @@ FAR struct tcb_s *nxtask_setup_fork(start_t retaddr) goto errout_with_tcb; } - /* Setup thread local storage */ - - ret = tls_dup_info(child, parent); - if (ret < OK) + /* Set up thread local storage and the argument vector. + * + * A fork() child that inherited its stack already has both, byte for + * byte, at the addresses the parent has them at -- they came across with + * the rest of the image. Re-creating them would carve fresh frames off a + * stack that already contains them, moving stack_base_ptr away from the + * parent's and undoing the inheritance. Only the TLS fields that name + * the task itself need correcting. + */ + +#if defined(CONFIG_ARCH_ADDRENV) && defined(CONFIG_ARCH_HAVE_FORK) + if (!vfork && ttype != TCB_FLAG_TTYPE_KERNEL) { - goto errout_with_tcb; + fork_inherit_tls(child); } - - /* Setup to pass parameters to the new task */ - - ret = nxtask_setup_stackargs(child, argv[0], &argv[1]); - if (ret < OK) + else +#endif { - goto errout_with_tcb; + ret = tls_dup_info(child, parent); + if (ret < OK) + { + goto errout_with_tcb; + } + + ret = nxtask_setup_stackargs(child, argv[0], &argv[1]); + if (ret < OK) + { + goto errout_with_tcb; + } } /* Now we have enough in place that we can join the group */ @@ -258,6 +431,18 @@ FAR struct tcb_s *nxtask_setup_fork(start_t retaddr) return child; errout_with_tcb: +#if defined(CONFIG_ARCH_ADDRENV) && defined(CONFIG_ARCH_HAVE_FORK) + /* Get back into the parent's address environment before unwinding. If the + * duplication above never happened this is the environment we are already + * in, and addrenv_restore() is then a no-op. + */ + + if (!vfork && ttype != TCB_FLAG_TTYPE_KERNEL) + { + fork_restore_parent_env(); + } +#endif + nxsched_release_tcb((FAR struct tcb_s *)child, ttype); errout: set_errno(-ret); @@ -268,65 +453,129 @@ errout: * Name: nxtask_start_fork * * Description: - * The fork() function has the same effect as fork(), except that the - * behavior is undefined if the process created by fork() either modifies - * any data other than a variable of type pid_t used to store the return - * value from fork(), or returns from the function in which fork() was - * called, or calls any other function before successfully calling _exit() - * or one of the exec family of functions. - * - * This function provides one step in the overall fork() sequence: It - * starts execution of the previously initialized TCB. The overall - * sequence is: - * - * 1) User code calls fork() - * 2) Architecture-specific code provides fork()and calls - * nxtask_setup_fork(). - * 3) nxtask_setup_fork() allocates and configures the child task's TCB. - * This consists of: - * - Allocation of the child task's TCB. - * - Initialization of file descriptors and streams - * - Configuration of environment variables - * - Allocate and initialize the stack - * - Setup the input parameters for the task. - * - Initialization of the TCB (including call to up_initial_state()) - * 4) fork() provides any additional operating context. fork must: - * - Initialize special values in any CPU registers that were not - * already configured by up_initial_state() - * 5) fork() then calls nxtask_start_fork() - * 6) nxtask_start_fork() then executes the child thread. + * The last step of both primitives: finish the child and run it. The + * architecture-specific code calls this once it has built the child's + * register context and stack. + * + * A vfork() additionally suspends the caller until the child calls _exit() + * or one of the exec family of functions. The suspension lives here, in + * the kernel primitive, rather than in a libc waitpid() as it once did. + * Two things follow from that. The parent is released when the child's + * TCB is torn down (see nxtask_resume_vfork()), which for an exec()ing + * child is immediately after exec_swap() has handed the child's pid to the + * program it loaded -- so the parent resumes at exec(), holding a pid that + * names the running program, as POSIX requires. And vfork() no longer + * depends on CONFIG_SCHED_WAITPID. * * Input Parameters: - * child - The tcb_s struct instance that created by - * nxtask_setup_fork() method - * wait_child - whether need to wait until the child is running finished + * child - The tcb_s struct instance created by nxtask_setup_fork() + * vfork - true for vfork(), false for fork() * * Returned Value: - * Upon successful completion, fork() returns 0 to the child process and - * returns the process ID of the child process to the parent process. - * Otherwise, -1 is returned to the parent, no child process is created, - * and errno is set to indicate the error. + * The process ID of the child, or ERROR on failure. * ****************************************************************************/ -pid_t nxtask_start_fork(FAR struct tcb_s *child) +pid_t nxtask_start_fork(FAR struct tcb_s *child, bool vfork) { +#ifdef CONFIG_ARCH_HAVE_VFORK + /* The rendezvous between the suspended parent and the child lives in this + * frame: the parent is blocked here for the whole lifetime of the child, + * so the storage is alive exactly as long as it is needed, and no + * allocation is required on a path that must not fail. + */ + + sem_t rel; + int ret; +#endif pid_t pid; - sinfo("Starting Child TCB=%p\n", child); + sinfo("Starting Child TCB=%p vfork=%d\n", child, vfork); DEBUGASSERT(child); +#if defined(CONFIG_ARCH_ADDRENV) && defined(CONFIG_ARCH_HAVE_FORK) + /* The architecture code has finished writing the child's image, so put the + * parent back in its own address environment. See nxtask_setup_fork(). + */ + + if (!vfork && + (child->flags & TCB_FLAG_TTYPE_MASK) != TCB_FLAG_TTYPE_KERNEL) + { + fork_restore_parent_env(); + } +#endif + /* Get the assigned pid before we start the task */ pid = child->pid; +#ifdef CONFIG_ARCH_HAVE_VFORK + if (vfork) + { + nxsem_init(&rel, 0, 0); + child->vfork_rel = &rel; + } +#endif + /* Activate the task */ nxtask_activate(child); +#ifdef CONFIG_ARCH_HAVE_VFORK + if (vfork) + { + /* Wait for the child to _exit() or exec(). This is not a cancellation + * point and must not be interrupted by a signal: the child may be + * running on our stack, so returning early would corrupt it. + */ + + do + { + ret = nxsem_wait_uninterruptible(&rel); + } + while (ret == -EINTR); + + nxsem_destroy(&rel); + } +#endif + return pid; } +#ifdef CONFIG_ARCH_HAVE_VFORK +/**************************************************************************** + * Name: nxtask_resume_vfork + * + * Description: + * Release the vfork() parent suspended on this child, if there is one. + * + * Called from nxsched_release_tcb(), the last point in the child's life, + * by which time an exec()ing child has already handed its pid to the + * program it loaded. nxtask_abort_fork() reaches it too, so a fork that + * fails after the rendezvous also releases the parent. + * + * Input Parameters: + * child - The TCB being torn down + * + * Returned Value: + * None + * + ****************************************************************************/ + +void nxtask_resume_vfork(FAR struct tcb_s *child) +{ + FAR sem_t *rel = child->vfork_rel; + + if (rel != NULL) + { + /* Clearing the pointer first is what makes this once-only. */ + + child->vfork_rel = NULL; + nxsem_post(rel); + } +} +#endif /* CONFIG_ARCH_HAVE_VFORK */ + /**************************************************************************** * Name: nxtask_abort_fork * @@ -340,6 +589,20 @@ pid_t nxtask_start_fork(FAR struct tcb_s *child) void nxtask_abort_fork(FAR struct tcb_s *child, int errcode) { +#if defined(CONFIG_ARCH_ADDRENV) && defined(CONFIG_ARCH_HAVE_FORK) + /* A child holding an address environment of its own, rather than a + * reference to the caller's, is a fork() child, and nxtask_setup_fork() + * left that environment selected. Get back into the parent's before + * unwinding. See nxtask_setup_fork(). + */ + + if (child->addrenv_own != NULL && + child->addrenv_own != this_task()->addrenv_own) + { + fork_restore_parent_env(); + } +#endif + /* The TCB was added to the active task list by nxtask_setup_scheduler() */ dq_rem((FAR dq_entry_t *)child, list_inactivetasks()); @@ -349,5 +612,3 @@ void nxtask_abort_fork(FAR struct tcb_s *child, int errcode) nxsched_release_tcb(child, child->flags & TCB_FLAG_TTYPE_MASK); set_errno(errcode); } - -#endif /* CONFIG_ARCH_HAVE_FORK */ diff --git a/syscall/syscall.csv b/syscall/syscall.csv index 4b7a69f9fc4..f5675cea0a0 100644 --- a/syscall/syscall.csv +++ b/syscall/syscall.csv @@ -208,7 +208,7 @@ "umount2","sys/mount.h","!defined(CONFIG_DISABLE_MOUNTPOINT)","int","FAR const char *","unsigned int" "unlink","unistd.h","!defined(CONFIG_DISABLE_MOUNTPOINT)","int","FAR const char *" "unsetenv","stdlib.h","!defined(CONFIG_DISABLE_ENVIRON)","int","FAR const char *" -"up_fork","nuttx/arch.h","defined(CONFIG_ARCH_HAVE_FORK)","pid_t" +"up_fork","nuttx/arch.h","defined(CONFIG_ARCH_HAVE_VFORK) || defined(CONFIG_ARCH_HAVE_FORK)","pid_t","bool" "utimens","sys/stat.h","","int","FAR const char *","const struct timespec [2]|FAR const struct timespec *" "wait","sys/wait.h","defined(CONFIG_SCHED_WAITPID) && defined(CONFIG_SCHED_HAVE_PARENT)","pid_t","FAR int *" "waitid","sys/wait.h","defined(CONFIG_SCHED_WAITPID) && defined(CONFIG_SCHED_HAVE_PARENT)","int","idtype_t","id_t"," FAR siginfo_t *","int"
