RE: system() and pthread_atfork()
> -Original Message- > From: Robert Elz > Sent: Tuesday, January 14, 2020 11:35 AM > To: Schwarz, Konrad (CT RDA IOT SES-DE) > Cc: nate.karst...@garmin.com; austin-group-l@opengroup.org > Subject: Re: system() and pthread_atfork() > | The point I was trying to make with the text you did not quote is that > | if the OP had been more judicious in closing sockets/file descriptors, > | he would not have run into the problem in the first place. > > The issue (as I understand it, I do not like the threading methedology, and > do not use it) is that in threaded > processes, with everything happening in parallel, and no one thread having > any real idea what any other thread > might be doing, there is no way to achieve the result you are expecting. > > That is, at the exact same instant one thread is doing fork() another > is doing open() or socket(). If it happens that the open() finishes one > zeta-second before the fork() starts then the fd from that open will be > inherited by the child of the fork, but > because the thread doing the open has not hat time to save the fd anywhere > yet, there is no way for the child > process (which only contains the thread which forked, not the others, > including not the one that did the open()) > to ever discover what that fd was, or what it connects to. But this is benign. Only one thread is running in the new process, that thread does not touch any sockets it knows nothing about, and it will soon exec(), and the close-on-exit flag will do its job. The mistake the OP was doing is to close the listening socket, which is bound to INADDR_ANY (and a fixed port), in response to an IP address change. When he does this and an unrelated fork() occurs, a race ensures: if the exec() does not happen soon enough, the parent fails to rebind to the socket, because it is still open in the child. Had he simply left the existing listening socket alone, everything would have worked. > > | This seriously undermines the case for an all new F_CLOFORK flag > > If it weren't for threading, I would not support it at all. In any > non-threaded context it is a stupid idea. But I can see the need for > it with our current threading methedology. The real problem is that > threads are a horrid misfeature. Unfortunately, lots of people seem > to like the evil things. No use case demonstrating need for this feature has been presented up to now.
Re: system() and pthread_atfork()
Date:Tue, 14 Jan 2020 08:35:14 + From:"Schwarz, Konrad" Message-ID: | Captain Obvious here: | A minimal quality of implementation attribute for DHCP daemons is for | these addresses to remain fixed for as long as possible. Yes, address stability is useful, as connections break when it is lost (given connections are identified by a tuple that includes the address). | Cf. https://kb.isc.org/docs/isc-dhcp-44-manual-pages-dhcpdleases, | which is designed to persist address assignments across restarts, | e.g., because the hosting server needs to reboot. It is a design goal. Some dhcp servers have no stable storage at all, and cannot meet that goal. | Or are you suggesting applications must be (re-)coded | such that they are resistant to dynamic changes of IP addresses? There is no way around that. Sometime addresses simply need to change. In the v4 world, the typical practice was to reconfigure the system, and then reboot, which forced all the applications to restart, with new addresses. Not exactlty an ideal environment. More recently, addresses can (even if they do not often) change while the system is running, so yes, applications need to be able to cope with that. It is only because in practice the addresses do not change very frequently - especially as most hosts live inside NAT enclaves, where use of private addr space means there is very little incentive to change, that there has been elatively little movement in this area. But consider a mobile device (not mobile-IP here, just a typical modern phone) that is using 4G comms as you move around, when you with who knows what addr from your phone company, when you arrive at my house, and connect to my WiFi, and are now using something on network 10.*.*.* ... the addresses the apps on the phone are seeing will have changed (unless the phone OS does a private internal NAT, and makes some internal address space into whatever it is given externally - which is just a method of coping with changing addresses, not of avoiding it from happening). eg: I know I need to, and so have a script to do it, restart my sendmail daemon whenever my laptop connects to a different access point, eg: when I move from my house to a hotspot provided by my phone ... because sendmail doesn't deal with this well at all. | The point I was trying to make with the text you did not quote is that | if the OP had been more judicious in closing sockets/file descriptors, | he would not have run into the problem in the first place. The issue (as I understand it, I do not like the threading methedology, and do not use it) is that in threaded processes, with everything happening in parallel, and no one thread having any real idea what any other thread might be doing, there is no way to achieve the result you are expecting. That is, at the exact same instant one thread is doing fork() another is doing open() or socket(). If it happens that the open() finishes one zeta-second before the fork() starts then the fd from that open will be inherited by the child of the fork, but because the thread doing the open has not hat time to save the fd anywhere yet, there is no way for the child process (which only contains the thread which forked, not the others, including not the one that did the open()) to ever discover what that fd was, or what it connects to. | This seriously undermines the case for an all new F_CLOFORK flag If it weren't for threading, I would not support it at all. In any non-threaded context it is a stupid idea. But I can see the need for it with our current threading methedology. The real problem is that threads are a horrid misfeature. Unfortunately, lots of people seem to like the evil things. kre
RE: system() and pthread_atfork()
> -Original Message- > From: Robert Elz > Sent: Monday, January 13, 2020 1:43 PM > To: Schwarz, Konrad (CT RDA IOT SES-DE) > Cc: nate.karst...@garmin.com; austin-group-l@opengroup.org > Subject: Re: system() and pthread_atfork() > > Date:Mon, 13 Jan 2020 10:13:04 + > From:"Schwarz, Konrad" <mailto:konrad.schw...@siemens.com> > Message-ID: > <mailto:a45b1767f1002449a37508c2cc6003d7172c9...@defthw99em4msx.ww902.siemens.net> > > | I actually feel this problem is out-of-scope for POSIX: compliant machines > | are not supposed to dynamically change their IP addresses at run-time. > > I have no idea what (if anything) POSIX says about IP networking > requirements, but I'd expect not much, but that > (if it were stated > somewhere) would be an error. > > [DHCP is allowed to dynamically change addresses] Captain Obvious here: A minimal quality of implementation attribute for DHCP daemons is for these addresses to remain fixed for as long as possible. Cf. https://kb.isc.org/docs/isc-dhcp-44-manual-pages-dhcpdleases, which is designed to persist address assignments across restarts, e.g., because the hosting server needs to reboot. Or are you suggesting applications must be (re-)coded such that they are resistant to dynamic changes of IP addresses? > All that said, I agree that anything related to this issue would be out of > scope for POSIX, but the more general > problem of threaded applications (which it must be, that's the only way that > the process can be simultaneously > closing & opening sockets, while also, unknown to itself, also forking) > and the interactions wrt fork & threads, is a POSIX issue. That the > actual problem is networking related is just a side issue, I believe. The point I was trying to make with the text you did not quote is that if the OP had been more judicious in closing sockets/file descriptors, he would not have run into the problem in the first place. This seriously undermines the case for an all new F_CLOFORK flag and associated paraphernalia. Indeed, except for Solaris, no implementation has ever implemented this, presumably because there is no real-world need for it. smime.p7s Description: S/MIME cryptographic signature
RE: system() and pthread_atfork()
I agree with Robert's statement -- this is more of a general problem of threaded applications and the interactions with fork & threads, which is a POSIX issue. The networking issue provides an example of one scenario that we saw a problem with. It could probably be made into a more general case: 1) Application acquires exclusive access to a system resource 2) Application forks 3) Application tries to release & re-acquire access to that resource; there is a race between the exec() in the child process (releasing the child's access) and the parent re-acquiring access Thanks, Nate -Original Message- From: Robert Elz Sent: Monday, January 13, 2020 6:43 AM To: Schwarz, Konrad Cc: Karstens, Nate ; austin-group-l@opengroup.org Subject: Re: system() and pthread_atfork() CAUTION - EXTERNAL EMAIL: Do not click any links or open any attachments unless you trust the sender and know the content is safe. Date:Mon, 13 Jan 2020 10:13:04 + From:"Schwarz, Konrad" Message-ID: | I actually feel this problem is out-of-scope for POSIX: compliant machines | are not supposed to dynamically change their IP addresses at run-time. I have no idea what (if anything) POSIX says about IP networking requirements, but I'd expect not much, but that (if it were stated somewhere) would be an error. Staticaally assigned IPv4 (or even IPv6) addresses do not tend to change much, but just about no-one uses those any more. If you obtain your address (v4 or v6) via DHCP, then when your host goes to renew the lease, the server can decline, and instruct you (your DHCP client) to request a new address, at which point it can give out whatever is appropriate, not necessarily even slightly related to what was there before. For v6 without DHCP, using stateless autoconfiguration, the router advertises a (or more than one) local network prefix, with a lifetime, and can alter that prefix whenever it is required. One of the real problems with networking in the 80's was dealing with client networks that needed to be renumbered - the workload for any reasonable sized network was prohibitive. One of v6's goals was to defeat that problem (it wasn't only about more addresses). In the meantime v4 has switched away from static configs (/etc/hosts or the equivalent) or from BOOTP for hosts without static storage, and to DHCP (where the 'D' really does mean dynamic) which some v6 sites also use (some odd notion of the network managers actually being able to control things that way, which is incorrect, but gives the net admins a warm fuzzy feeling, and it is what they have become used to with v4). All that said, I agree that anything related to this issue would be out of scope for POSIX, but the more general problem of threaded applications (which it must be, that's the only way that the process can be simultaneously closing & opening sockets, while also, unknown to itself, also forking) and the interactions wrt fork & threads, is a POSIX issue. That the actual problem is networking related is just a side issue, I believe. kre (IAB member during the period all this network transition started). CONFIDENTIALITY NOTICE: This email and any attachments are for the sole use of the intended recipient(s) and contain information that may be Garmin confidential and/or Garmin legally privileged. If you have received this email in error, please notify the sender by reply email and delete the message. Any disclosure, copying, distribution or use of this communication (including attachments) by someone other than the intended recipient is prohibited. Thank you.
Re: system() and pthread_atfork()
Date:Mon, 13 Jan 2020 10:13:04 + From:"Schwarz, Konrad" Message-ID: | I actually feel this problem is out-of-scope for POSIX: compliant machines | are not supposed to dynamically change their IP addresses at run-time. I have no idea what (if anything) POSIX says about IP networking requirements, but I'd expect not much, but that (if it were stated somewhere) would be an error. Staticaally assigned IPv4 (or even IPv6) addresses do not tend to change much, but just about no-one uses those any more. If you obtain your address (v4 or v6) via DHCP, then when your host goes to renew the lease, the server can decline, and instruct you (your DHCP client) to request a new address, at which point it can give out whatever is appropriate, not necessarily even slightly related to what was there before. For v6 without DHCP, using stateless autoconfiguration, the router advertises a (or more than one) local network prefix, with a lifetime, and can alter that prefix whenever it is required. One of the real problems with networking in the 80's was dealing with client networks that needed to be renumbered - the workload for any reasonable sized network was prohibitive. One of v6's goals was to defeat that problem (it wasn't only about more addresses). In the meantime v4 has switched away from static configs (/etc/hosts or the equivalent) or from BOOTP for hosts without static storage, and to DHCP (where the 'D' really does mean dynamic) which some v6 sites also use (some odd notion of the network managers actually being able to control things that way, which is incorrect, but gives the net admins a warm fuzzy feeling, and it is what they have become used to with v4). All that said, I agree that anything related to this issue would be out of scope for POSIX, but the more general problem of threaded applications (which it must be, that's the only way that the process can be simultaneously closing & opening sockets, while also, unknown to itself, also forking) and the interactions wrt fork & threads, is a POSIX issue. That the actual problem is networking related is just a side issue, I believe. kre (IAB member during the period all this network transition started).
RE: system() and pthread_atfork()
> -Original Message- > From: Karstens, Nate > Sent: Sunday, January 12, 2020 11:52 AM > To: 'austin-group-l@opengroup.org' > Subject: Re: system() and pthread_atfork() Going back to the original problem, > We are running Linux on an embedded system. The platform can > change the IP address either according to a proprietary negotiation scheme > or a manual setting. The application uses netlink to listen for IP address > changes; > when this occurs the application closes all of its sockets and re-opens them > using the new address. > > A problem can occur if the application is simultaneously fork/exec-ing a new > process. > The parent process attempts to bind a new socket to a port that it had > previously > bound to (before the IP address change), only to fail because the child > process > continues to hold a socket bound to that port. I actually feel this problem is out-of-scope for POSIX: compliant machines are not supposed to dynamically change their IP addresses at run-time. Even if we accept this premise, I'm not sure I understand the problem: suppose, on the bound socket, you used the specific IP address; then, when switching to the new address, you could simply create a new socket using the new specific address. Since port numbers are local to IP addresses, this should not create a conflict. (The process should close the old socket to conserve file descriptors). On the other hand, if you were using INADDR_ANY, why not simply leave the socket open? I would expect a machine that allows dynamic changes to the supported internet addresses to route new connection requests to pre-existing sockets bound to INADDR_ANY.
Re: system() and pthread_atfork()
I wanted to thank everyone for the discussion and various suggestions. There are two changes that I have created in the Austin Group defect tracker: 1) system(), popen(), etc. should be required to call pthread_atfork() handlers if their implementation is not atomic (see http://austingroupbugs.net/view.php?id=1317). 2) POSIX should specify a SOCK_CLOFORK / FD_CLOFORK socket option (see http://austingroupbugs.net/view.php?id=1318). In addition, it would have been preferable for the default behavior to close all file descriptors on a fork and have flags to override that behavior. I have also been in contact with the author of the original FD_CLOFORK patch for Linux and they are OK with me taking that over and updating for the latest kernel. Cheers, Nate -Original Message- From: Karstens, Nate Sent: Friday, December 20, 2019 4:40 PM To: austin-group-l@opengroup.org Subject: Re: system() and pthread_atfork() Thanks to everyone for the great responses! Below are some notes/discussion on the various replies I've seen: > If it's supposed to behave as if fork() has been called, this implicitly > includes calling atfork handlers when POSIX_C_SOURCE is defined... Regarding glibc, I'm not sure the POSIX_C_SOURCE macro would control that. Here is a reference to the latest definition of system() in the POSIX standard: https://pubs.opengroup.org/onlinepubs/9699919799/functions/system.html The glibc developers had a discussion on this topic here: https://sourceware.org/bugzilla/show_bug.cgi?id=17490 They seemed to go back and forth on this same issue, and one developer suggested communicating the ambiguity with the Austin Group. I'm not sure if this was ever done, but there still appears to be a discrepancy (in system(), popen() -- https://pubs.opengroup.org/onlinepubs/9699919799/functions/popen.html, and maybe others). > Couldn't that be addressed by SO_REUSEADDR? According to https://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_10_16, SO_REUSEADDR "indicates that the rules used in validating addresses supplied in a https://pubs.opengroup.org/onlinepubs/9699919799/functions/bind.html should allow reuse of local addresses". Based on this, it doesn't appear to be for the purposes of addressing this issue, and we're not excited about the other side effects as well. > The reason it likely that the implementer of system() and popen() get the > freedom to use a more efficient mechanism. That might be, but it would be good to document the tradeoffs associated with that efficiency. Even better would be to come up with a mechanism to avoid it entirely. If requiring pthread_atfork() handlers to execute is not an option, then maybe a file-descriptor/socket option that marks the socket as closing once the process forks (like SOCK_CLOEXEC / FD_CLOEXEC, but done at fork() instead of exec()). > The vfork() used before also did not call the pthread_atfork() handlers > either. https://pubs.opengroup.org/onlinepubs/009604599/functions/vfork.html comes with a lot of warnings, an explicit recommendation to use fork(), and a note that it may be withdrawn in the future. As such we weren't as worried about it, but maybe we should be... > If you have control over the callers of system() We don't have control over them, which makes this more disconcerting. Cheers, Nate CONFIDENTIALITY NOTICE: This email and any attachments are for the sole use of the intended recipient(s) and contain information that may be Garmin confidential and/or Garmin legally privileged. If you have received this email in error, please notify the sender by reply email and delete the message. Any disclosure, copying, distribution or use of this communication (including attachments) by someone other than the intended recipient is prohibited. Thank you.
RE: system() and pthread_atfork()
> -Original Message- > From: Matthew Dempsky > Sent: Friday, January 3, 2020 2:27 AM > To: Schwarz, Konrad (CT RDA IOT SES-DE) > Cc: Karstens, Nate ; austin-group-l@opengroup.org > Subject: Re: system() and pthread_atfork() > > On Thu, Jan 2, 2020 at 5:01 AM Schwarz, Konrad > <mailto:konrad.schw...@siemens.com> wrote: > > I think the right solution is for POSIX to require system() and popen() to > > call pthread_atfork() handlers. > > How would this work for systems where system() is implemented using > posix_spawn()? posix_spawn()'s RATIONALE > explicitly mentions that it can be used to implement system(), but also it's > meant to be implementable without > using fork() (and thus without fork handlers). > > It seems like the requirement should be more nuanced. E.g., that *if* > system() is implemented using fork(), then it must call at-fork handlers. I'm > not sure how to phrase that in > standardese though. In my “second attempt”, I wrote > I think the right solution is for POSIX to require system() and popen() to > call pthread_atfork() handlers, if they [i.e., system() and popen()] are not > atomic with regards to exec().
RE: system() and pthread_atfork()
> From: Schwarz, Konrad <mailto:konrad.schw...@siemens.com> > Sent: Thursday, January 2, 2020 5:00 AM > To: Karstens, Nate <mailto:nate.karst...@garmin.com>; > mailto:austin-group-l@opengroup.org > Subject: [EXTERNAL] RE: system() and pthread_atfork() > >> -Original Message- >> From: Karstens, Nate <mailto:nate.karst...@garmin.com> >> Sent: Thursday, December 19, 2019 12:26 AM >> To: mailto:austin-group-l@opengroup.org >> Subject: system() and pthread_atfork() >> >> The current definition of system() does not define if the pthread_atfork() >> handlers are called. We ran into a >> scenario where this caused a problem and wanted to share it with the mailing >> list to better understand why those >> handlers are not required and get some advice on how best to proceed. > > I think the right solution is for POSIX to require system() and popen() to > call pthread_atfork() handlers. > I haven't noticed any arguments against such a solution and it clearly fills > a need. > > I suggest you open a corresponding defect report on http://austingroupbugs.net Both system() and popen() will, nearly immediately, exec() completely new processes. Under many circumstances, the new processes will not interact with objects maintained by any registered pthread_atfork() handlers. Any work those handlers do will, in those cases, be completely wasted as a result of the exec(). That is one argument against your proposed solution. I doubt this is "undefined" by accident. In other words, I doubt this is a defect. If there's no discussion in rationale, I could see that being a defect, but defects in non-normative text are, by definition, not actually defects. The way to proceed, I believe, is for a portable application to provide its own implementations of system() and popen() which provide precisely the semantics required by that application. Open source implementations of both system() and popen() are readily available and can be altered to call your pthread_atfork() handlers as you see fit, without regard to what the platform's implementation might, or might not, do.
RE: system() and pthread_atfork()
Both system() and popen() will, nearly immediately, exec() completely new processes. Under many circumstances, the new processes will not interact with objects maintained by any registered pthread_atfork() handlers. Any work those handlers do will, in those cases, be completely wasted as a result of the exec(). That is one argument against your proposed solution. I doubt this is "undefined" by accident. In other words, I doubt this is a defect. If there's no discussion in rationale, I could see that being a defect, but defects in non-normative text are, by definition, not actually defects. The way to proceed, I believe, is for a portable application to provide its own implementations of system() and popen() which provide precisely the semantics required by that application. Open source implementations of both system() and popen() are readily available and can be altered to call your pthread_atfork() handlers as you see fit, without regard to what the platform's implementation might, or might not, do. From: Schwarz, Konrad Sent: Thursday, January 2, 2020 5:00 AM To: Karstens, Nate ; austin-group-l@opengroup.org Subject: [EXTERNAL] RE: system() and pthread_atfork() > -Original Message- > From: Karstens, Nate > mailto:nate.karst...@garmin.com>> > Sent: Thursday, December 19, 2019 12:26 AM > To: austin-group-l@opengroup.org<mailto:austin-group-l@opengroup.org> > Subject: system() and pthread_atfork() > > The current definition of system() does not define if the pthread_atfork() > handlers are called. We ran into a > scenario where this caused a problem and wanted to share it with the mailing > list to better understand why those > handlers are not required and get some advice on how best to proceed. I think the right solution is for POSIX to require system() and popen() to call pthread_atfork() handlers. I haven't noticed any arguments against such a solution and it clearly fills a need. I suggest you open a corresponding defect report on http://austingroupbugs.net<https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Faustingroupbugs.net%2Fview.php%3Fid%3D1306=02%7C01%7Cjason.zions%40microsoft.com%7Cdea2ea0f24054ec1fb6d08d78f84020c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637135669478802830=UteAhiMJC4LQ%2BFbZDG%2FZ%2FwWzf5miW8OhA8iguFMs7iA%3D=0>
RE: system() and pthread_atfork()
> -Original Message- > From: Karstens, Nate > Sent: Thursday, December 19, 2019 12:26 AM > To: austin-group-l@opengroup.org > Subject: system() and pthread_atfork() > > The current definition of system() does not define if the pthread_atfork() > handlers are called. We ran into a > scenario where this caused a problem and wanted to share it with the mailing > list to better understand why those > handlers are not required and get some advice on how best to proceed. I think the right solution is for POSIX to require system() and popen() to call pthread_atfork() handlers. I haven't noticed any arguments against such a solution and it clearly fills a need. I suggest you open a corresponding defect report on http://austingroupbugs.net<http://austingroupbugs.net/view.php?id=1306>
Re: system() and pthread_atfork()
On 12/20/19 2:40 PM, Karstens, Nate wrote: That might be, but it would be good to document the tradeoffs associated with that efficiency. Even better would be to come up with a mechanism to avoid it entirely. If requiring pthread_atfork() handlers to execute is not an option, then maybe a file-descriptor/socket option that marks the socket as closing once the process forks (like SOCK_CLOEXEC / FD_CLOEXEC, but done at fork() instead of exec()). Solaris has FD_CLOFORK et al. for that purpose: https://docs.oracle.com/cd/E88353_01/html/E37841/fcntl-2.html https://lwn.net/Articles/785430/ suggests AIX, BSD, & MacOS have also defined it, and though it's been proposed multiple times for Linux, never adopted there. -- -Alan Coopersmith- alan.coopersm...@oracle.com Oracle Solaris Engineering - https://blogs.oracle.com/alanc
Re: system() and pthread_atfork()
Thanks to everyone for the great responses! Below are some notes/discussion on the various replies I've seen: > If it's supposed to behave as if fork() has been called, this implicitly > includes calling atfork handlers when POSIX_C_SOURCE is defined... Regarding glibc, I'm not sure the POSIX_C_SOURCE macro would control that. Here is a reference to the latest definition of system() in the POSIX standard: https://pubs.opengroup.org/onlinepubs/9699919799/functions/system.html The glibc developers had a discussion on this topic here: https://sourceware.org/bugzilla/show_bug.cgi?id=17490 They seemed to go back and forth on this same issue, and one developer suggested communicating the ambiguity with the Austin Group. I'm not sure if this was ever done, but there still appears to be a discrepancy (in system(), popen() -- https://pubs.opengroup.org/onlinepubs/9699919799/functions/popen.html, and maybe others). > Couldn't that be addressed by SO_REUSEADDR? According to https://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_10_16, SO_REUSEADDR "indicates that the rules used in validating addresses supplied in a https://pubs.opengroup.org/onlinepubs/9699919799/functions/bind.html should allow reuse of local addresses". Based on this, it doesn't appear to be for the purposes of addressing this issue, and we're not excited about the other side effects as well. > The reason it likely that the implementer of system() and popen() get the > freedom to use a more efficient mechanism. That might be, but it would be good to document the tradeoffs associated with that efficiency. Even better would be to come up with a mechanism to avoid it entirely. If requiring pthread_atfork() handlers to execute is not an option, then maybe a file-descriptor/socket option that marks the socket as closing once the process forks (like SOCK_CLOEXEC / FD_CLOEXEC, but done at fork() instead of exec()). > The vfork() used before also did not call the pthread_atfork() handlers > either. https://pubs.opengroup.org/onlinepubs/009604599/functions/vfork.html comes with a lot of warnings, an explicit recommendation to use fork(), and a note that it may be withdrawn in the future. As such we weren't as worried about it, but maybe we should be... > If you have control over the callers of system() We don't have control over them, which makes this more disconcerting. Cheers, Nate CONFIDENTIALITY NOTICE: This email and any attachments are for the sole use of the intended recipient(s) and contain information that may be Garmin confidential and/or Garmin legally privileged. If you have received this email in error, please notify the sender by reply email and delete the message. Any disclosure, copying, distribution or use of this communication (including attachments) by someone other than the intended recipient is prohibited. Thank you.
Re: system() and pthread_atfork()
>This seems like a scenario where you would want system() to require >pthread_atfork() handlers to be called. Aside from that, it seems somewhat >contradictory that system() "shall behave as if a child process were created >using fork()..." but it is undefined if pthread_atfork() handlers are called . > The reason it likely that the implementer of system() and popen() get the freedom to use a more efficient mechanism. In the case of Solaris, we have been using posix_spawn(); originally implemented on top of vfork() & execve(), in Solaris 11.4 it is an actual system call. No call to fork() even under the covers. The vfork() used before also did not call the pthread_atfork() handlers either. If you have control over the callers of system(), you could replace those with posix_spawn() and have posix_spawn attributes in place to close the specific file descriptors. Casper
Re: system() and pthread_atfork()
Hello. Karstens, Nate wrote in : |The current definition of system() does not define if the pthread_atfork() \ ... No head to think about that but.. ... |a new process. The parent process attempts to bind a new socket to \ |a port that it had previously bound to (before the IP address change), \ |only to fail because the child process continues to hold a socket bound \ |to that port. | |The close-on-exec flag (SOCK_CLOEXEC / FD_CLOEXEC) seems like it would \ |help here, but does not because the child process has not called exec() \ |yet. Couldn't that be addressed by SO_REUSEADDR? ... |Any feedback would be much appreciated. ..following that star :) --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
system() and pthread_atfork()
Greetings, The current definition of system() does not define if the pthread_atfork() handlers are called. We ran into a scenario where this caused a problem and wanted to share it with the mailing list to better understand why those handlers are not required and get some advice on how best to proceed. We are running Linux on an embedded system. The platform can change the IP address either according to a proprietary negotiation scheme or a manual setting. The application uses netlink to listen for IP address changes; when this occurs the application closes all of its sockets and re-opens them using the new address. A problem can occur if the application is simultaneously fork/exec-ing a new process. The parent process attempts to bind a new socket to a port that it had previously bound to (before the IP address change), only to fail because the child process continues to hold a socket bound to that port. The close-on-exec flag (SOCK_CLOEXEC / FD_CLOEXEC) seems like it would help here, but does not because the child process has not called exec() yet. Our solution was to use a pthread_atfork() handler to lock a mutex and wait for the child process to close all of its sockets (as signaled through an eventfd) before the parent attempts to create them again. This doesn't work if the application uses system() anywhere (the version of glibc we use does not invoke pthread_atfork() handlers). We changed all calls to system() to instead use fork()/exec(), but are still concerned about third-party libraries. This seems like a scenario where you would want system() to require pthread_atfork() handlers to be called. Aside from that, it seems somewhat contradictory that system() "shall behave as if a child process were created using fork()..." but it is undefined if pthread_atfork() handlers are called. Any feedback would be much appreciated. Thanks, Nate CONFIDENTIALITY NOTICE: This email and any attachments are for the sole use of the intended recipient(s) and contain information that may be Garmin confidential and/or Garmin legally privileged. If you have received this email in error, please notify the sender by reply email and delete the message. Any disclosure, copying, distribution or use of this communication (including attachments) by someone other than the intended recipient is prohibited. Thank you.