Re: POSIX environ variable
>I think this is already done for a few group common things. This is what task_info_s in TLS does, it is the "process common information" field. So putting env there would work for the whole process. The name "TLS" here is a bit confusing, obviously the env data is shared by the process and all its threads. -Ville From: Gregory Nutt Sent: Tuesday, November 5, 2024 3:35 PM To: [email protected] Subject: Re: POSIX environ variable On 11/5/2024 6:57 AM, Ville Juven wrote: > For direct user level access, one option is to move tg_envp to TLS / > task_info_s. This way the user could access the memory without boring a hole > into the kernel. The same environ is used by all threads in a task group/process. If one thread changes the environment, it effects all threads in group/process It could be possible to put the environment in the TLS of the main thread and other could the access indirectly through their TLS references. > In KERNEL mode this issue is very simple; every process has its own instance > of environ** anyway so there is no need to use TLS or anything. Yep. Lots of things are simpler in true kernel mode. Just like the designers of Unix intended! NuttX is more complex and requires some trade-offs. > However, in FLAT mode having a per-process instance of environ** is going to > be more difficult. You could -in theory- make a reference to the TLS area and > use it like that. This reference would have to be updated on every context > switch. Whether or not this is even possible to implement in a multicore > (SMP) system is beyond me, the reference would need to be duplicated on every > CPU and the user would need to be able to query which CPU it is running on. I think this is already done for a few group common things. But I don't remember. I haven't been that active for awhile (I got too old! 74 in three days).
Re: POSIX environ variable
On 11/5/2024 6:57 AM, Ville Juven wrote: For direct user level access, one option is to move tg_envp to TLS / task_info_s. This way the user could access the memory without boring a hole into the kernel. The same environ is used by all threads in a task group/process. If one thread changes the environment, it effects all threads in group/process It could be possible to put the environment in the TLS of the main thread and other could the access indirectly through their TLS references. In KERNEL mode this issue is very simple; every process has its own instance of environ** anyway so there is no need to use TLS or anything. Yep. Lots of things are simpler in true kernel mode. Just like the designers of Unix intended! NuttX is more complex and requires some trade-offs. However, in FLAT mode having a per-process instance of environ** is going to be more difficult. You could -in theory- make a reference to the TLS area and use it like that. This reference would have to be updated on every context switch. Whether or not this is even possible to implement in a multicore (SMP) system is beyond me, the reference would need to be duplicated on every CPU and the user would need to be able to query which CPU it is running on. I think this is already done for a few group common things. But I don't remember. I haven't been that active for awhile (I got too old! 74 in three days).
Re: POSIX environ variable
For direct user level access, one option is to move tg_envp to TLS / task_info_s. This way the user could access the memory without boring a hole into the kernel. In KERNEL mode this issue is very simple; every process has its own instance of environ** anyway so there is no need to use TLS or anything. However, in FLAT mode having a per-process instance of environ** is going to be more difficult. You could -in theory- make a reference to the TLS area and use it like that. This reference would have to be updated on every context switch. Whether or not this is even possible to implement in a multicore (SMP) system is beyond me, the reference would need to be duplicated on every CPU and the user would need to be able to query which CPU it is running on. You could combine Takashi's idea with TLS to achieve a "good enough" solution. -Ville From: Gregory Nutt Sent: Tuesday, November 5, 2024 2:39 PM To: [email protected] Subject: Re: POSIX environ variable On 11/5/2024 3:43 AM, Takashi Yamamoto wrote: > i guess it's possible to implement it similarly to what we do for errno. > > eg. > # define environ (*get_environ_ptr_ptr()) > > although it still isn't quite posix compatible, it might be good > enough for many applications. > d18be28c82a2a6d82115c8af19d/shell/ash.c#L10574 There would be a complexity in this: The environ is shared by all threads in a task group. Hence, the environ lies in the protected, kernel task group structure. That structure is not accessible to user code in PROTECTED or KERNEL build modes. So in order to do what you want, you would need to (1) redesign how common data is stored so that the environ is accessible from user space or (2) marshal the environ to other accessible memory when it is requested. Neither are simple and the latter would have synchronization issues. I think there is an open issue about the POSIX compatibility of the environment.
Re: POSIX environ variable
On 11/5/2024 3:43 AM, Takashi Yamamoto wrote: i guess it's possible to implement it similarly to what we do for errno. eg. # define environ (*get_environ_ptr_ptr()) although it still isn't quite posix compatible, it might be good enough for many applications. d18be28c82a2a6d82115c8af19d/shell/ash.c#L10574 There would be a complexity in this: The environ is shared by all threads in a task group. Hence, the environ lies in the protected, kernel task group structure. That structure is not accessible to user code in PROTECTED or KERNEL build modes. So in order to do what you want, you would need to (1) redesign how common data is stored so that the environ is accessible from user space or (2) marshal the environ to other accessible memory when it is requested. Neither are simple and the latter would have synchronization issues. I think there is an open issue about the POSIX compatibility of the environment.
Re: POSIX environ variable
i guess it's possible to implement it similarly to what we do for errno. eg. # define environ (*get_environ_ptr_ptr()) although it still isn't quite posix compatible, it might be good enough for many applications. whatever approach we take, it probably requires some extra validations in the kernel. (ie. untrust the pointer as it might be user-given now.) On Tue, Nov 5, 2024 at 5:22 PM Marco Casaroli wrote: > > Hello, > > I am sorry if this was already discussed before. I could not find > concrete information in the documentation or the mailing list history. > > According to POSIX specifications [1]: > > > The array is pointed to by the external variable environ, which is defined > > as: > > > > extern char **environ; > > But in NuttX, it is defined differently [2], when > CONFIG_DISABLE_ENVIRON is not set, like: > > > # define environ get_environ_ptr() > > Which calls a function [3] and gets its result. This gives problems if > the apps try to do things like: > > > environ = ...; > > Which is, in fact, fairly common [4, 5, 6]. > > So I have two questions: > > 1. What is the cleanest way to work around the environment update > these apps are trying to do? What are the changes I should make to the > app code so that it can correctly handle setting the environment? > > 2. Is there any way we can update NuttX to actually define environ > like POSIX specifies, so that we can handle these apps without > modifications? > > Thank you very much. > > BR > > // Marco Casaroli > > [1] > https://pubs.opengroup.org/onlinepubs/9699919799.2018edition/basedefs/V1_chap08.html > [2] > https://github.com/apache/nuttx/blob/0fad2ee73f5431cbe34cc02253909f290938cd12/include/stdlib.h#L70 > [3] > https://github.com/apache/nuttx/blob/0fad2ee73f5431cbe34cc02253909f290938cd12/sched/environ/env_getenvironptr.c#L57 > [4] > https://github.com/landley/toybox/blob/c47184b389d5bf97135afbf55aa4a41bc29812eb/lib/env.c#L29 > [5] > https://github.com/landley/toybox/blob/c47184b389d5bf97135afbf55aa4a41bc29812eb/toys/pending/sh.c#L1379 > [6] > https://github.com/mirror/busybox/blob/371fe9f71d445d18be28c82a2a6d82115c8af19d/shell/ash.c#L10574
