RE: system() and pthread_atfork()

[Schwarz, Konrad]

> -Original Message-
> From: Robert Elz 
> Sent: Tuesday, January 14, 2020 11:35 AM
> To: Schwarz, Konrad (CT RDA IOT SES-DE) 
> Cc: nate.karst...@garmin.com; austin-group-l@opengroup.org
> Subject: Re: system() and pthread_atfork()


>   | The point I was trying to make with the text you did not quote is that
>   | if the OP had been more judicious in closing sockets/file descriptors,
>   | he would not have run into the problem in the first place.
> 
> The issue (as I understand it, I do not like the threading methedology, and 
> do not use it) is that in threaded
> processes, with everything happening in parallel, and no one thread having 
> any real idea what any other thread
> might be doing, there is no way to achieve the result you are expecting.
> 
> That is, at the exact same instant one thread is doing fork() another
> is doing open() or socket().   If it happens that the open() finishes one
> zeta-second before the fork() starts then the fd from that open will be 
> inherited by the child of the fork, but
> because the thread doing the open has not hat time to save the fd anywhere 
> yet, there is no way for the child
> process (which only contains the thread which forked, not the others, 
> including not the one that did the open())
> to ever discover what that fd was, or what it connects to.

But this is benign.  Only one thread is running in the new process,
that thread does not touch any sockets it knows nothing about,
and it will soon exec(), and the close-on-exit flag will do its job.

The mistake the OP was doing is to close the listening socket, which is bound 
to INADDR_ANY (and a fixed port),
in response to an IP address change.  When he does this and an unrelated fork() 
occurs,
a race ensures: if the exec() does not happen soon enough, the parent fails to 
rebind to the
socket, because it is still open in the child.

Had he simply left the existing listening socket alone, everything would have 
worked.

> 
>   | This seriously undermines the case for an all new F_CLOFORK flag
> 
> If it weren't for threading, I would not support it at all.   In any
> non-threaded context it is a stupid idea.   But I can see the need for
> it with our current threading methedology.   The real problem is that
> threads are a horrid misfeature.   Unfortunately, lots of people seem
> to like the evil things.

No use case demonstrating need for this feature has been presented up to now.

Re: system() and pthread_atfork()

[Robert Elz]

Date:Tue, 14 Jan 2020 08:35:14 +
From:"Schwarz, Konrad" 
Message-ID:  


  | Captain Obvious here:
  | A minimal quality of implementation attribute for DHCP daemons is for
  | these addresses to remain fixed for as long as possible.

Yes, address stability is useful, as connections break when it is
lost (given connections are identified by a tuple that includes the
address).

  | Cf. https://kb.isc.org/docs/isc-dhcp-44-manual-pages-dhcpdleases,
  | which is designed to persist address assignments across restarts,
  | e.g., because the hosting server needs to reboot.

It is a design goal.   Some dhcp servers have no stable storage at
all, and cannot meet that goal.

  | Or are you suggesting applications must be (re-)coded
  | such that they are resistant to dynamic changes of IP addresses?

There is no way around that.   Sometime addresses simply need to
change.  In the v4 world, the typical practice was to reconfigure
the system, and then reboot, which forced all the applications to
restart, with new addresses.  Not exactlty an ideal environment.

More recently, addresses can (even if they do not often) change
while the system is running, so yes, applications need to be able
to cope with that.

It is only because in practice the addresses do not change very
frequently - especially as most hosts live inside NAT enclaves,
where use of private addr space means there is very little incentive
to change, that there has been elatively little movement in this area.

But consider a mobile device (not mobile-IP here, just a typical
modern phone) that is using 4G comms as you move around, when you
with who knows what addr from your phone company, when you arrive
at my house, and connect to my WiFi, and are now using something
on network 10.*.*.* ... the addresses the apps on the phone are
seeing will have changed (unless the phone OS does a private internal
NAT, and makes some internal address space into whatever it is given
externally - which is just a method of coping with changing addresses,
not of avoiding it from happening).

eg: I know I need to, and so have a script to do it, restart
my sendmail daemon whenever my laptop connects to a different
access point, eg: when I move from my house to a hotspot provided
by my phone ... because sendmail doesn't deal with this well at all.

  | The point I was trying to make with the text you did not quote is that
  | if the OP had been more judicious in closing sockets/file descriptors,
  | he would not have run into the problem in the first place.

The issue (as I understand it, I do not like the threading methedology,
and do not use it) is that in threaded processes, with everything happening
in parallel, and no one thread having any real idea what any other
thread might be doing, there is no way to achieve the result you are
expecting.

That is, at the exact same instant one thread is doing fork() another
is doing open() or socket().   If it happens that the open() finishes one
zeta-second before the fork() starts then the fd from that open will
be inherited by the child of the fork, but because the thread doing the
open has not hat time to save the fd anywhere yet, there is no way for
the child process (which only contains the thread which forked, not the
others, including not the one that did the open()) to ever discover what
that fd was, or what it connects to.

  | This seriously undermines the case for an all new F_CLOFORK flag

If it weren't for threading, I would not support it at all.   In any
non-threaded context it is a stupid idea.   But I can see the need for
it with our current threading methedology.   The real problem is that
threads are a horrid misfeature.   Unfortunately, lots of people seem
to like the evil things.

kre

RE: system() and pthread_atfork()

[Schwarz, Konrad]

> -Original Message-
> From: Robert Elz 
> Sent: Monday, January 13, 2020 1:43 PM
> To: Schwarz, Konrad (CT RDA IOT SES-DE) 
> Cc: nate.karst...@garmin.com; austin-group-l@opengroup.org
> Subject: Re: system() and pthread_atfork()
> 
> Date:Mon, 13 Jan 2020 10:13:04 +
> From:"Schwarz, Konrad" <mailto:konrad.schw...@siemens.com>
> Message-ID:  
> <mailto:a45b1767f1002449a37508c2cc6003d7172c9...@defthw99em4msx.ww902.siemens.net>
> 
>   | I actually feel this problem is out-of-scope for POSIX: compliant machines
>   | are not supposed to dynamically change their IP addresses at run-time.
> 
> I have no idea what (if anything) POSIX says about IP networking 
> requirements, but I'd expect not much, but that
> (if it were stated
> somewhere) would be an error.
> 
> [DHCP is allowed to dynamically change addresses]

Captain Obvious here:
A minimal quality of implementation attribute for DHCP daemons is for
these addresses to remain fixed for as long as possible.
Cf. https://kb.isc.org/docs/isc-dhcp-44-manual-pages-dhcpdleases,
which is designed to persist address assignments across restarts,
e.g., because the hosting server needs to reboot.

Or are you suggesting applications must be (re-)coded
such that they are resistant to dynamic changes of IP addresses?
 
> All that said, I agree that anything related to this issue would be out of 
> scope for POSIX, but the more general
> problem of threaded applications (which it must be, that's the only way that 
> the process can be simultaneously
> closing & opening sockets, while also, unknown to itself, also forking)
> and the interactions wrt fork & threads, is a POSIX issue.   That the
> actual problem is networking related is just a side issue, I believe.

The point I was trying to make with the text you did not quote is that
if the OP had been more judicious in closing sockets/file descriptors,
he would not have run into the problem in the first place.

This seriously undermines the case for an all new F_CLOFORK flag and associated
paraphernalia.  Indeed, except for Solaris, no implementation has ever
implemented this, presumably because there is no real-world need for it.


smime.p7s
Description: S/MIME cryptographic signature

RE: system() and pthread_atfork()

[Karstens, Nate]

I agree with Robert's statement -- this is more of a general problem of 
threaded applications and the interactions with fork & threads, which is a 
POSIX issue. The networking issue provides an example of one scenario that we 
saw a problem with. It could probably be made into a more general case:

1) Application acquires exclusive access to a system resource
2) Application forks
3) Application tries to release & re-acquire access to that resource; there is 
a race between the exec() in the child process (releasing the child's access) 
and the parent re-acquiring access

Thanks,

Nate

-Original Message-
From: Robert Elz 
Sent: Monday, January 13, 2020 6:43 AM
To: Schwarz, Konrad 
Cc: Karstens, Nate ; austin-group-l@opengroup.org
Subject: Re: system() and pthread_atfork()

CAUTION - EXTERNAL EMAIL: Do not click any links or open any attachments unless 
you trust the sender and know the content is safe.


Date:Mon, 13 Jan 2020 10:13:04 +
From:"Schwarz, Konrad" 
Message-ID:  


  | I actually feel this problem is out-of-scope for POSIX: compliant machines
  | are not supposed to dynamically change their IP addresses at run-time.

I have no idea what (if anything) POSIX says about IP networking requirements, 
but I'd expect not much, but that (if it were stated
somewhere) would be an error.

Staticaally assigned IPv4 (or even IPv6) addresses do not tend to change much, 
but just about no-one uses those any more.

If you obtain your address (v4 or v6) via DHCP, then when your host goes to 
renew the lease, the server can decline, and instruct you (your DHCP client) to 
request a new address, at which point it can give out whatever is appropriate, 
not necessarily even slightly related to what was there before.

For v6 without DHCP, using stateless autoconfiguration, the router advertises a 
(or more than one) local network prefix, with a lifetime, and can alter that 
prefix whenever it is required.

One of the real problems with networking in the 80's was dealing with client 
networks that needed to be renumbered - the workload for any reasonable sized 
network was prohibitive.  One of v6's goals was to
defeat that problem (it wasn't only about more addresses).   In the
meantime v4 has switched away from static configs (/etc/hosts or the
equivalent) or from BOOTP for hosts without static storage, and to DHCP (where 
the 'D' really does mean dynamic) which some v6 sites also use (some odd notion 
of the network managers actually being able to control things that way, which 
is incorrect, but gives the net admins a warm fuzzy feeling, and it is what 
they have become used to with v4).

All that said, I agree that anything related to this issue would be out of 
scope for POSIX, but the more general problem of threaded applications (which 
it must be, that's the only way that the process can be simultaneously closing 
& opening sockets, while also, unknown to itself, also forking)
and the interactions wrt fork & threads, is a POSIX issue.   That the
actual problem is networking related is just a side issue, I believe.

kre   (IAB member during the period all this network transition started).




CONFIDENTIALITY NOTICE: This email and any attachments are for the sole use of 
the intended recipient(s) and contain information that may be Garmin 
confidential and/or Garmin legally privileged. If you have received this email 
in error, please notify the sender by reply email and delete the message. Any 
disclosure, copying, distribution or use of this communication (including 
attachments) by someone other than the intended recipient is prohibited. Thank 
you.

Re: system() and pthread_atfork()

[Robert Elz]

Date:Mon, 13 Jan 2020 10:13:04 +
From:"Schwarz, Konrad" 
Message-ID:  


  | I actually feel this problem is out-of-scope for POSIX: compliant machines
  | are not supposed to dynamically change their IP addresses at run-time.

I have no idea what (if anything) POSIX says about IP networking
requirements, but I'd expect not much, but that (if it were stated
somewhere) would be an error.

Staticaally assigned IPv4 (or even IPv6) addresses do not tend to
change much, but just about no-one uses those any more.

If you obtain your address (v4 or v6) via DHCP, then when your host
goes to renew the lease, the server can decline, and instruct you
(your DHCP client) to request a new address, at which point it can
give out whatever is appropriate, not necessarily even slightly
related to what was there before.

For v6 without DHCP, using stateless autoconfiguration, the router
advertises a (or more than one) local network prefix, with a lifetime,
and can alter that prefix whenever it is required.

One of the real problems with networking in the 80's was dealing with
client networks that needed to be renumbered - the workload for any
reasonable sized network was prohibitive.  One of v6's goals was to
defeat that problem (it wasn't only about more addresses).   In the
meantime v4 has switched away from static configs (/etc/hosts or the
equivalent) or from BOOTP for hosts without static storage, and to
DHCP (where the 'D' really does mean dynamic) which some v6 sites
also use (some odd notion of the network managers actually being able
to control things that way, which is incorrect, but gives the net admins
a warm fuzzy feeling, and it is what they have become used to with v4).

All that said, I agree that anything related to this issue would be out
of scope for POSIX, but the more general problem of threaded applications
(which it must be, that's the only way that the process can be simultaneously
closing & opening sockets, while also, unknown to itself, also forking)
and the interactions wrt fork & threads, is a POSIX issue.   That the
actual problem is networking related is just a side issue, I believe.

kre   (IAB member during the period all this network transition started).

RE: system() and pthread_atfork()

[Schwarz, Konrad]

> -Original Message-
> From: Karstens, Nate 
> Sent: Sunday, January 12, 2020 11:52 AM
> To: 'austin-group-l@opengroup.org' 
> Subject: Re: system() and pthread_atfork()

Going back to the original problem,

> We are running Linux on an embedded system. The platform can
> change the IP address either according to a proprietary negotiation scheme
> or a manual setting. The application uses netlink to listen for IP address 
> changes;
> when this occurs the application closes all of its sockets and re-opens them 
> using the new address.
>
> A problem can occur if the application is simultaneously fork/exec-ing a new 
> process.
> The parent process attempts to bind a new socket to a port that it had 
> previously
> bound to (before the IP address change), only to fail because the child 
> process
> continues to hold a socket bound to that port.

I actually feel this problem is out-of-scope for POSIX: compliant machines
are not supposed to dynamically change their IP addresses at run-time.

Even if we accept this premise, I'm not sure I understand the problem:
suppose, on the bound socket, you used the specific IP address;
then, when switching to the new address, you could simply create
a new socket using the new specific address.  Since port numbers
are local to IP addresses, this should not create
a conflict.  (The process should close the old socket to conserve file 
descriptors).

On the other hand, if you were using INADDR_ANY, why not simply
leave the socket open?  I would expect a machine that allows dynamic
changes to the supported internet addresses to route new
connection requests to pre-existing sockets bound to INADDR_ANY.

Re: system() and pthread_atfork()

[Karstens, Nate]

I wanted to thank everyone for the discussion and various suggestions. There 
are two changes that I have created in the Austin Group defect tracker:

1) system(), popen(), etc. should be required to call pthread_atfork() handlers 
if their implementation is not atomic (see 
http://austingroupbugs.net/view.php?id=1317).
2) POSIX should specify a SOCK_CLOFORK / FD_CLOFORK socket option (see 
http://austingroupbugs.net/view.php?id=1318). In addition, it would have been 
preferable for the default behavior to close all file descriptors on a fork and 
have flags to override that behavior.

I have also been in contact with the author of the original FD_CLOFORK patch 
for Linux and they are OK with me taking that over and updating for the latest 
kernel.

Cheers,

Nate

-Original Message-
From: Karstens, Nate
Sent: Friday, December 20, 2019 4:40 PM
To: austin-group-l@opengroup.org
Subject: Re: system() and pthread_atfork()

Thanks to everyone for the great responses! Below are some notes/discussion on 
the various replies I've seen:

> If it's supposed to behave as if fork() has been called, this implicitly 
> includes calling atfork handlers when POSIX_C_SOURCE is defined...

Regarding glibc, I'm not sure the POSIX_C_SOURCE macro would control that. Here 
is a reference to the latest definition of system() in the POSIX standard:

https://pubs.opengroup.org/onlinepubs/9699919799/functions/system.html

The glibc developers had a discussion on this topic here:

https://sourceware.org/bugzilla/show_bug.cgi?id=17490

They seemed to go back and forth on this same issue, and one developer 
suggested communicating the ambiguity with the Austin Group. I'm not sure if 
this was ever done, but there still appears to be a discrepancy (in system(), 
popen() -- 
https://pubs.opengroup.org/onlinepubs/9699919799/functions/popen.html, and 
maybe others).

> Couldn't that be addressed by SO_REUSEADDR?

According to 
https://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_10_16,
 SO_REUSEADDR "indicates that the rules used in validating addresses supplied 
in a https://pubs.opengroup.org/onlinepubs/9699919799/functions/bind.html 
should allow reuse of local addresses". Based on this, it doesn't appear to be 
for the purposes of addressing this issue, and we're not excited about the 
other side effects as well.

> The reason it likely that the implementer of system() and popen() get the 
> freedom to use a more efficient mechanism.

That might be, but it would be good to document the tradeoffs associated with 
that efficiency. Even better would be to come up with a mechanism to avoid it 
entirely. If requiring pthread_atfork() handlers to execute is not an option, 
then maybe a file-descriptor/socket option that marks the socket as closing 
once the process forks (like SOCK_CLOEXEC / FD_CLOEXEC, but done at fork() 
instead of exec()).

> The vfork() used before also did not call the pthread_atfork() handlers 
> either.

https://pubs.opengroup.org/onlinepubs/009604599/functions/vfork.html comes with 
a lot of warnings, an explicit recommendation to use fork(), and a note that it 
may be withdrawn in the future. As such we weren't as worried about it, but 
maybe we should be...

> If you have control over the callers of system()

We don't have control over them, which makes this more disconcerting.

Cheers,

Nate



CONFIDENTIALITY NOTICE: This email and any attachments are for the sole use of 
the intended recipient(s) and contain information that may be Garmin 
confidential and/or Garmin legally privileged. If you have received this email 
in error, please notify the sender by reply email and delete the message. Any 
disclosure, copying, distribution or use of this communication (including 
attachments) by someone other than the intended recipient is prohibited. Thank 
you.

RE: system() and pthread_atfork()

[Schwarz, Konrad]

> -Original Message-
> From: Matthew Dempsky 
> Sent: Friday, January 3, 2020 2:27 AM
> To: Schwarz, Konrad (CT RDA IOT SES-DE) 
> Cc: Karstens, Nate ; austin-group-l@opengroup.org
> Subject: Re: system() and pthread_atfork()
> 
> On Thu, Jan 2, 2020 at 5:01 AM Schwarz, Konrad 
> <mailto:konrad.schw...@siemens.com> wrote:
> > I think the right solution is for POSIX to require system() and popen() to 
> > call pthread_atfork() handlers.
> 
> How would this work for systems where system() is implemented using 
> posix_spawn()? posix_spawn()'s RATIONALE
> explicitly mentions that it can be used to implement system(), but also it's 
> meant to be implementable without
> using fork() (and thus without fork handlers).
> 
> It seems like the requirement should be more nuanced. E.g., that *if*
> system() is implemented using fork(), then it must call at-fork handlers. I'm 
> not sure how to phrase that in
> standardese though.

In my “second attempt”, I wrote

> I think the right solution is for POSIX to require system() and popen() to 
> call pthread_atfork() handlers, if they [i.e., system() and popen()] are not 
> atomic with regards to exec().

RE: system() and pthread_atfork()

[Jason Zions]

> From: Schwarz, Konrad <mailto:konrad.schw...@siemens.com> 
> Sent: Thursday, January 2, 2020 5:00 AM
> To: Karstens, Nate <mailto:nate.karst...@garmin.com>; 
> mailto:austin-group-l@opengroup.org
> Subject: [EXTERNAL] RE: system() and pthread_atfork()
> 
>> -Original Message-
>> From: Karstens, Nate <mailto:nate.karst...@garmin.com>
>> Sent: Thursday, December 19, 2019 12:26 AM
>> To: mailto:austin-group-l@opengroup.org
>> Subject: system() and pthread_atfork()
>> 
>> The current definition of system() does not define if the pthread_atfork() 
>> handlers are called. We ran into a
>> scenario where this caused a problem and wanted to share it with the mailing 
>> list to better understand why those
>> handlers are not required and get some advice on how best to proceed. 
>
> I think the right solution is for POSIX to require system() and popen() to 
> call pthread_atfork() handlers.
> I haven't noticed any arguments against such a solution and it clearly fills 
> a need.
>
> I suggest you open a corresponding defect report on http://austingroupbugs.net

Both system() and popen() will, nearly immediately, exec() completely new 
processes. Under many circumstances, the new processes will not interact with 
objects maintained by any registered pthread_atfork() handlers. Any work those 
handlers do will, in those cases, be completely wasted as a result of the 
exec(). That is one argument against your proposed solution.

I doubt this is "undefined" by accident. In other words, I doubt this is a 
defect. If there's no discussion in rationale, I could see that being a defect, 
but defects in non-normative text are, by definition, not actually defects.

The way to proceed, I believe, is for a portable application to provide its own 
implementations of system() and popen() which provide precisely the semantics 
required by that application. Open source implementations of both system() and 
popen() are readily available and can be altered to call your pthread_atfork() 
handlers as you see fit, without regard to what the platform's implementation 
might, or might not, do.

RE: system() and pthread_atfork()

[Jason Zions]

Both system() and popen() will, nearly immediately, exec() completely new 
processes. Under many circumstances, the new processes will not interact with 
objects maintained by any registered pthread_atfork() handlers. Any work those 
handlers do will, in those cases, be completely wasted as a result of the 
exec(). That is one argument against your proposed solution.

I doubt this is "undefined" by accident. In other words, I doubt this is a 
defect. If there's no discussion in rationale, I could see that being a defect, 
but defects in non-normative text are, by definition, not actually defects.

The way to proceed, I believe, is for a portable application to provide its own 
implementations of system() and popen() which provide precisely the semantics 
required by that application. Open source implementations of both system() and 
popen() are readily available and can be altered to call your pthread_atfork() 
handlers as you see fit, without regard to what the platform's implementation 
might, or might not, do.

From: Schwarz, Konrad 
Sent: Thursday, January 2, 2020 5:00 AM
To: Karstens, Nate ; austin-group-l@opengroup.org
Subject: [EXTERNAL] RE: system() and pthread_atfork()


> -Original Message-

> From: Karstens, Nate 
> mailto:nate.karst...@garmin.com>>

> Sent: Thursday, December 19, 2019 12:26 AM

> To: austin-group-l@opengroup.org<mailto:austin-group-l@opengroup.org>

> Subject: system() and pthread_atfork()

>

> The current definition of system() does not define if the pthread_atfork() 
> handlers are called. We ran into a

> scenario where this caused a problem and wanted to share it with the mailing 
> list to better understand why those

> handlers are not required and get some advice on how best to proceed.



I think the right solution is for POSIX to require system() and popen() to call 
pthread_atfork() handlers.

I haven't noticed any arguments against such a solution and it clearly fills a 
need.



I suggest you open a corresponding defect report on 
http://austingroupbugs.net<https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Faustingroupbugs.net%2Fview.php%3Fid%3D1306=02%7C01%7Cjason.zions%40microsoft.com%7Cdea2ea0f24054ec1fb6d08d78f84020c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637135669478802830=UteAhiMJC4LQ%2BFbZDG%2FZ%2FwWzf5miW8OhA8iguFMs7iA%3D=0>

RE: system() and pthread_atfork()

[Schwarz, Konrad]

> -Original Message-

> From: Karstens, Nate 

> Sent: Thursday, December 19, 2019 12:26 AM

> To: austin-group-l@opengroup.org

> Subject: system() and pthread_atfork()

>

> The current definition of system() does not define if the pthread_atfork() 
> handlers are called. We ran into a

> scenario where this caused a problem and wanted to share it with the mailing 
> list to better understand why those

> handlers are not required and get some advice on how best to proceed.



I think the right solution is for POSIX to require system() and popen() to call 
pthread_atfork() handlers.

I haven't noticed any arguments against such a solution and it clearly fills a 
need.



I suggest you open a corresponding defect report on 
http://austingroupbugs.net

Re: system() and pthread_atfork()

[Alan Coopersmith]

On 12/20/19 2:40 PM, Karstens, Nate wrote:

That might be, but it would be good to document the tradeoffs associated with 
that efficiency. Even better would be to come up with a mechanism to avoid it 
entirely. If requiring pthread_atfork() handlers to execute is not an option, 
then maybe a file-descriptor/socket option that marks the socket as closing 
once the process forks (like SOCK_CLOEXEC / FD_CLOEXEC, but done at fork() 
instead of exec()).


Solaris has FD_CLOFORK et al. for that purpose:
https://docs.oracle.com/cd/E88353_01/html/E37841/fcntl-2.html

https://lwn.net/Articles/785430/ suggests AIX, BSD, & MacOS have also defined
it, and though it's been proposed multiple times for Linux, never adopted there.

--
-Alan Coopersmith-   alan.coopersm...@oracle.com
 Oracle Solaris Engineering - https://blogs.oracle.com/alanc

Re: system() and pthread_atfork()

[Karstens, Nate]

Thanks to everyone for the great responses! Below are some notes/discussion on 
the various replies I've seen:

> If it's supposed to behave as if fork() has been called, this implicitly 
> includes calling atfork handlers when POSIX_C_SOURCE is defined...

Regarding glibc, I'm not sure the POSIX_C_SOURCE macro would control that. Here 
is a reference to the latest definition of system() in the POSIX standard:

https://pubs.opengroup.org/onlinepubs/9699919799/functions/system.html

The glibc developers had a discussion on this topic here:

https://sourceware.org/bugzilla/show_bug.cgi?id=17490

They seemed to go back and forth on this same issue, and one developer 
suggested communicating the ambiguity with the Austin Group. I'm not sure if 
this was ever done, but there still appears to be a discrepancy (in system(), 
popen() -- 
https://pubs.opengroup.org/onlinepubs/9699919799/functions/popen.html, and 
maybe others).

> Couldn't that be addressed by SO_REUSEADDR?

According to 
https://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_10_16,
 SO_REUSEADDR "indicates that the rules used in validating addresses supplied 
in a https://pubs.opengroup.org/onlinepubs/9699919799/functions/bind.html 
should allow reuse of local addresses". Based on this, it doesn't appear to be 
for the purposes of addressing this issue, and we're not excited about the 
other side effects as well.

> The reason it likely that the implementer of system() and popen() get the 
> freedom to use a more efficient mechanism.

That might be, but it would be good to document the tradeoffs associated with 
that efficiency. Even better would be to come up with a mechanism to avoid it 
entirely. If requiring pthread_atfork() handlers to execute is not an option, 
then maybe a file-descriptor/socket option that marks the socket as closing 
once the process forks (like SOCK_CLOEXEC / FD_CLOEXEC, but done at fork() 
instead of exec()).

> The vfork() used before also did not call the pthread_atfork() handlers 
> either.

https://pubs.opengroup.org/onlinepubs/009604599/functions/vfork.html comes with 
a lot of warnings, an explicit recommendation to use fork(), and a note that it 
may be withdrawn in the future. As such we weren't as worried about it, but 
maybe we should be...

> If you have control over the callers of system()

We don't have control over them, which makes this more disconcerting.

Cheers,

Nate



CONFIDENTIALITY NOTICE: This email and any attachments are for the sole use of 
the intended recipient(s) and contain information that may be Garmin 
confidential and/or Garmin legally privileged. If you have received this email 
in error, please notify the sender by reply email and delete the message. Any 
disclosure, copying, distribution or use of this communication (including 
attachments) by someone other than the intended recipient is prohibited. Thank 
you.

Re: system() and pthread_atfork()

[Casper . Dik]

>This seems like a scenario where you would want system() to require 
>pthread_atfork() handlers to be called. Aside from that, it seems somewhat 
>contradictory that system() "shall behave as if a child process were created 
>using fork()..." but it is undefined if pthread_atfork() handlers are called
.
>

The reason it likely that the implementer of system() and popen() get the 
freedom to use a more efficient mechanism.

In the case of Solaris, we have been using posix_spawn(); originally 
implemented on top of vfork() & execve(), in Solaris 11.4 it is an actual 
system call.  No call to fork() even under the covers.

The vfork() used before also did not call the pthread_atfork() handlers 
either.

If you have control over the callers of system(), you could replace those 
with posix_spawn() and have posix_spawn attributes in place to close the 
specific file descriptors.

Casper

Re: system() and pthread_atfork()

[Steffen Nurpmeso]

Hello.

Karstens, Nate wrote in :
 |The current definition of system() does not define if the pthread_atfork() \
 ...

No head to think about that but..

  ...
 |a new process. The parent process attempts to bind a new socket to \
 |a port that it had previously bound to (before the IP address change), \
 |only to fail because the child process continues to hold a socket bound \
 |to that port.
 |
 |The close-on-exec flag (SOCK_CLOEXEC / FD_CLOEXEC) seems like it would \
 |help here, but does not because the child process has not called exec() \
 |yet.

Couldn't that be addressed by SO_REUSEADDR?

  ...
 |Any feedback would be much appreciated.

..following that star :)

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)