Re: Fwd: Re: [1003.1(2016/18)/Issue7+TC2 0001436]: make: add "-j max_jobs" option to support simultaneous rule processing
Paul Smith wrote in : |On Sat, 2021-05-22 at 00:59 +0200, Steffen Nurpmeso via austin-group-l |at The Open Group wrote: |> Then stating something like "i am process X, and my parent is Y" |> etc. And the rest being up to the make(1) implementor as a quality |> of implementation (scheduling, fair even more so, seems to reside in |> the area of very complicated programming). I was surprised to see |> that rule content matters at all. | |The problem is that depending on the implementation, the parent make |might need to pass actual resources to the child make instance. Those |resources could negatively impact child processes that are NOT make. | In that case, make needs to know whether the thing it is invoking is |another make or not, so it knows whether to pass those resources. | |As an example, in GNU make today we use a simple pipe to implement this |feature, which means child makes need to have the open file descriptors |for the ends of the pipe provided to them when they are started. | |But, some other programs that are not make, might not work well if they |are invoked with extra file descriptors already open, and if they write |garbage into the jobserver pipe then it will break things. So in this |implementation make would want to close-on-exec those file descriptors |before invoking the sub-process, IF it knows the sub-process is not a |make instance. | |Of course, for an implementation which can assume POSIX most likely |this is NOT the method that would be chosen to implement the jobserver |as it leads to complications. GNU make, however, is part of a system |bootstrap toolchain (needed to build a compiler for an old system to |start to update it for example) so I try to keep it portable to very |old systems. | |Nevertheless, I've been considering switching GNU make's implementation |to a named pipe (mkfifo) or a POSIX semaphore. These implementations |would resolve all the above issues (and others, such as blocking/non- |blocking FDs etc.) since only a sub-make would access the named pipe. | | |Anyway, that's not really relevant but just FYI as to why (I suspect) |the standard was worded like this. Thanks for the explanation. --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: Fwd: Re: [1003.1(2016/18)/Issue7+TC2 0001436]: make: add "-j max_jobs" option to support simultaneous rule processing
On Sat, 2021-05-22 at 00:59 +0200, Steffen Nurpmeso via austin-group-l at The Open Group wrote: > Then stating something like "i am process X, and my parent is Y" > etc. And the rest being up to the make(1) implementor as a quality > of implementation (scheduling, fair even more so, seems to reside in > the area of very complicated programming). I was surprised to see > that rule content matters at all. The problem is that depending on the implementation, the parent make might need to pass actual resources to the child make instance. Those resources could negatively impact child processes that are NOT make. In that case, make needs to know whether the thing it is invoking is another make or not, so it knows whether to pass those resources. As an example, in GNU make today we use a simple pipe to implement this feature, which means child makes need to have the open file descriptors for the ends of the pipe provided to them when they are started. But, some other programs that are not make, might not work well if they are invoked with extra file descriptors already open, and if they write garbage into the jobserver pipe then it will break things. So in this implementation make would want to close-on-exec those file descriptors before invoking the sub-process, IF it knows the sub-process is not a make instance. Of course, for an implementation which can assume POSIX most likely this is NOT the method that would be chosen to implement the jobserver as it leads to complications. GNU make, however, is part of a system bootstrap toolchain (needed to build a compiler for an old system to start to update it for example) so I try to keep it portable to very old systems. Nevertheless, I've been considering switching GNU make's implementation to a named pipe (mkfifo) or a POSIX semaphore. These implementations would resolve all the above issues (and others, such as blocking/non- blocking FDs etc.) since only a sub-make would access the named pipe. Anyway, that's not really relevant but just FYI as to why (I suspect) the standard was worded like this.
Fwd: Re: [1003.1(2016/18)/Issue7+TC2 0001436]: make: add "-j max_jobs" option to support simultaneous rule processing
Hello. It is kind of redundant, but i think i forward the message once again, now that my subscription is reestablished. --- Forwarded from Steffen Nurpmeso --- Date: Fri, 21 May 2021 17:57:20 +0200 From: Steffen Nurpmeso Subject: Re: [1003.1(2016/18)/Issue7+TC2 0001436]: make: add "-j max_jobs" option to support simultaneous rule processing Austin Group Bug Tracker wrote in <79557df8f0e0d0b53548449c40247...@austingroupbugs.net>: |https://austingroupbugs.net/view.php?id=1436 ... |-- | (0005362) rhansen (manager) - 2021-05-20 17:08 | https://austingroupbugs.net/view.php?id=1436#c5362 |-- |We think we have achieved consensus on a rewrite of the description of the |-j option; see "attempt #3" on line 65 of |https://posix.rhansen.org/p/2021-05-20. Feedback would be appreciated. I am not a honourable make(1) programmer but since i opened the issue i want to state that i liked it when i read it last night. I was actually surprised to see the issue reopened as such, because my thinking would have been that a -j parallelized make(1) enters some kind of jobserver mode that becomes established via some environmental setting (or whatever the programmer chooses aka can easily be found by subprocesses), the existence of which is checked by make(1) at startup. Then stating something like "i am process X, and my parent is Y" etc. And the rest being up to the make(1) implementor as a quality of implementation (scheduling, fair even more so, seems to reside in the area of very complicated programming). I was surprised to see that rule content matters at all. A nice weekend i wish from Germany, --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: POSIX gettext() and uselocale()
On Tue, May 04, 2021 at 01:07:39AM +0200, Bruno Haible via austin-group-l at The Open Group wrote: > https://posix.rhansen.org/p/gettext_split > says (line 92): > "The returned string may be invalidated by a subsequent call to >bind_textdomain_codeset(), bindtextdomain(), setlocale(), >textdomain(), or uselocale()." > While in most programs setlocale(), textdomain(), bindtextdomain(), > bind_textdomain_codeset() are being called at the beginning of the > program execution, before any call to gettext(), the situation is > very different for uselocale(). > 1) uselocale() is meant to have effects ONLY on the thread in which it >is called. > 2) uselocale() is a helper function to implement *_l functions where >the POSIX standard does not specify them or the system does not have >them. >For example, when a program wants to have a function to parse >a number, recognizing only the ASCII digits and only '.' as decimal >separator, a reliable way to implement such a function is by calling >uselocale of the "C" locale, strtod(), and then uselocale() again >to switch the thread back to the previous locale. >If POSIX did not have uselocale(), it would need to provide many >more *_l functions. > If the gettext() result may be invalidated by a uselocale() call (in > any other thread!), this would mean that > ** Programs can use gettext() or uselocale() but not both. ** > and - more or less - > ** Multithreaded programs that use libraries (that may use uselocale()) > cannot use gettext(). ** > I think that specifying gettext() to be so restricted is not useful. > It would make more sense to allow concurrent uselocale() calls. > Proposed wording: > "The returned string may be invalidated by a subsequent call to >bind_textdomain_codeset(), bindtextdomain(), setlocale(), >or textdomain()." This may be a bit too weak. Now the implementation can never free a string that was returned by a gettext call on a thread with uselocale() active, while logically the string may be owned by the locale and could be freed if that locale is no longer set on any thread and freelocale() has been called on it as needed. -- Jilles Tjoelker
Fwd: Re: [1003.1(2016/18)/Issue7+TC2 0001436]: make: add "-j max_jobs" option to support simultaneous rule processing
--- Forwarded from Steffen Nurpmeso --- Date: Fri, 21 May 2021 17:57:20 +0200 From: Steffen Nurpmeso To: Austin Group Bug Tracker Subject: Re: [1003.1(2016/18)/Issue7+TC2 0001436]: make: add "-j max_jobs" option to support simultaneous rule processing Message-ID: <20210521155720.khyhn%stef...@sdaoden.eu> OpenPGP: id=EE19E1C1F2F7054F8D3954D8308964B51883A0DD; url=https://ftp.sdaoden.eu/steffen.asc; preference=signencrypt Austin Group Bug Tracker wrote in <79557df8f0e0d0b53548449c40247...@austingroupbugs.net>: |https://austingroupbugs.net/view.php?id=1436 ... |-- | (0005362) rhansen (manager) - 2021-05-20 17:08 | https://austingroupbugs.net/view.php?id=1436#c5362 |-- |We think we have achieved consensus on a rewrite of the description of the |-j option; see "attempt #3" on line 65 of |https://posix.rhansen.org/p/2021-05-20. Feedback would be appreciated. I am not a honourable make(1) programmer but since i opened the issue i want to state that i liked it when i read it last night. I was actually surprised to see the issue reopened as such, because my thinking would have been that a -j parallelized make(1) enters some kind of jobserver mode that becomes established via some environmental setting (or whatever the programmer chooses aka can easily be found by subprocesses), the existence of which is checked by make(1) at startup. Then stating something like "i am process X, and my parent is Y" etc. And the rest being up to the make(1) implementor as a quality of implementation (scheduling, fair even more so, seems to reside in the area of very complicated programming). I was surprised to see that rule content matters at all. A nice weekend i wish from Germany, --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) -- End forward <20210521155720.khyhn%stef...@sdaoden.eu> --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Fwd: [1003.1(2016/18)/Issue7+TC2 0001436]: make: add "-j max_jobs" option to support simultaneous rule processing
Forwarding (Steffen does not have his address subscribed, so it did not make it to the list) > Begin forwarded message: > > From: Steffen Nurpmeso > Subject: Fwd: Re: [1003.1(2016/18)/Issue7+TC2 0001436]: make: add "-j > max_jobs" option to support simultaneous rule processing > Date: 22 May 2021 at 15:09:21 BST > To: austin-group-l@opengroup.org > X-Diagnostic: Not on the accept list > > --- Forwarded from Steffen Nurpmeso > Date: Fri, 21 May 2021 17:57:20 +0200 > From: Steffen Nurpmeso > To: Austin Group Bug Tracker > Subject: Re: [1003.1(2016/18)/Issue7+TC2 0001436]: make: add "-j max_jobs" > option to support simultaneous rule processing > Message-ID: <20210521155720.khyhn%stef...@sdaoden.eu> > OpenPGP: id=EE19E1C1F2F7054F8D3954D8308964B51883A0DD; > url=https://ftp.sdaoden.eu/steffen.asc; preference=signencrypt > > Austin Group Bug Tracker wrote in > <79557df8f0e0d0b53548449c40247...@austingroupbugs.net>: > |https://austingroupbugs.net/view.php?id=1436 > ... > |-- > | (0005362) rhansen (manager) - 2021-05-20 17:08 > | https://austingroupbugs.net/view.php?id=1436#c5362 > |-- > |We think we have achieved consensus on a rewrite of the description of the > |-j option; see "attempt #3" on line 65 of > |https://posix.rhansen.org/p/2021-05-20. Feedback would be appreciated. > > I am not a honourable make(1) programmer but since i opened the > issue i want to state that i liked it when i read it last night. > I was actually surprised to see the issue reopened as such, > because my thinking would have been that a -j parallelized make(1) > enters some kind of jobserver mode that becomes established via > some environmental setting (or whatever the programmer chooses aka > can easily be found by subprocesses), the existence of which is > checked by make(1) at startup. Then stating something like "i am > process X, and my parent is Y" etc. And the rest being up to the > make(1) implementor as a quality of implementation (scheduling, > fair even more so, seems to reside in the area of very complicated > programming). I was surprised to see that rule content matters at > all. > > A nice weekend i wish from Germany, > > --steffen > | > |Der Kragenbaer,The moon bear, > |der holt sich munter he cheerfully and one by one > |einen nach dem anderen runter wa.ks himself off > |(By Robert Gernhardt) > > -- End forward <20210521155720.khyhn%stef...@sdaoden.eu> > > --steffen > | > |Der Kragenbaer,The moon bear, > |der holt sich munter he cheerfully and one by one > |einen nach dem anderen runter wa.ks himself off > |(By Robert Gernhardt) > Andrew Josey, The Open Group, ajo...@opengroup.org
