Re: [1003.1(2016/18)/Issue7+TC2 0001437]: make: (document .NOTPARALLEL and .WAIT special targets) in RATIONALE
On Tue, 2022-09-13 at 14:53 +0100, Geoff Clare via austin-group-l at The Open Group wrote: > If I'm honest those cases never occurred to me when we were working > on the wording. It seems unlikely a makefile author would > intentionally use .WAIT in those ways, but obviously they could arise > through expansion of an empty macro. > > I think we should tweak the text at the next opportunity to add > "(if any)" in the appropriate places. I'll make a note to submit > a bug against draft 3 to request that change. Thanks. For everyone's info, I implemented .WAIT in GNU make (pushed to Git yesterday) and it will be available in the next release. I did review Steffen's patch but decided against that approach; but the effort is much appreciated Steffen!
Re: [1003.1(2016/18)/Issue7+TC2 0001437]: make: (document .NOTPARALLEL and .WAIT special targets) in RATIONALE
On Thu, 2021-09-02 at 16:28 +, Austin Group Bug Tracker wrote: > On page 2975, insert before line > 98708:.NOTPARALLELThe application > shall ensure that this special target is specified without > prerequisites or commands. What exactly is required by an implementation to meet this "shall ensure" statement? Does this mean make implementations must generate a fatal error of some kind if there are prerequisites or commands to .NOTPARALLEL (and .POSIX: target is specified)? It's quite simple to imagine adding a capability that, for example, allows you to specify prerequisites of .NOTPARALLEL and have it do useful things. Requiring a fatal error if someone adds .POSIX: seems overly strict. > On page 2975, insert before line > 98742:.WAITThe application shall > ensure that this special target, if specified as a target, is > specified without prerequisites or commands. When .WAIT appears > as a target, it shall have no effect. When .WAIT appears in a > target rule as a prerequisite, it shall not itself be treated as a > prerequisite; however, make shall not recursively process the > prerequisites to the right of the .WAIT until the > prerequisites to the left of it have been brought up-to-date. > Implementations may also enforce the same ordering between the > affected prerequisites while processing other target rules that > have some or all of the same affected > prerequisites. For my clarification, what is the behavior for a rule like this: target: .WAIT prereq ; : Am I right in assuming that the above text means that this is a no-op and identical in behavior to: target: prereq ; : because there are no "prerequisites to the left of the .WAIT"? Similarly, I assume that this: target: prereq .WAIT ; : is also a no-op since there are no "prerequisites to the right of the .WAIT"...?
Re: [1003.1(2008)/Issue 7 0000514]: Enhance internal macros in make
On Fri, 2022-09-09 at 14:13 -0400, Paul Smith via austin-group-l at The Open Group wrote: > Ah OK. I don't have access to the text (as far as I know) so I > wasn't aware that it was defined that way. Sounds good. Oh I see the text in the minutes now. OK with me. Thanks.
Re: [1003.1(2008)/Issue 7 0000514]: Enhance internal macros in make
On Fri, 2022-09-09 at 17:13 +0100, Geoff Clare via austin-group-l at The Open Group wrote: > I assumed when you said "other internal macros" you weren't including > $+ because the addition for $+ is being described by reference to $^ > and so is subject to the same exception for .WAIT. Ah OK. I don't have access to the text (as far as I know) so I wasn't aware that it was defined that way. Sounds good.
Re: [1003.1(2008)/Issue 7 0000514]: Enhance internal macros in make
On Fri, 2022-09-09 at 16:02 +0100, Geoff Clare via austin-group-l at The Open Group wrote: > > > $^ > > > The $^ macro shall evaluate to the list of prerequisites > > > for the current target, with any duplicates (except > > > the first) removed. It shall be evaluated for both > > > target and inference rules. If the list of > > > prerequisites of the target contain any .WAIT special > > > targets, the results of expanding $^ are unspecified. > > > > Thanks, but what about the other internal macros that may refer to > > prerequisites? Don't we need to specify the .WAIT behavior for > > those > > as well? > > No need, because the description of .WAIT being added says "When > .WAIT appears in a target rule as a prerequisite, it shall not itself > be treated as a prerequisite". Are you saying that the implementations that keep .WAIT (or even change its name) in $^ have been discovered to NOT keep .WAIT in $+ and so the $+ macro doesn't need the above special exception saying that the results are unspecified, and the standard can just affirm that .WAIT is not present in $+? That seems exceedingly odd to me but ok!
Re: [1003.1(2008)/Issue 7 0000514]: Enhance internal macros in make
On Thu, 2022-09-08 at 17:50 -0400, Lawrence Velázquez via austin-group- l at The Open Group wrote: > On Thu, Sep 8, 2022, at 3:29 PM, Paul Smith via austin-group-l at The > Open Group wrote: > > I don't have visibility into the current proposed text for the > > POSIX make definition, but can someone clarify how the .WAIT > > prerequisites are to be treated when expanding internal macros like > > $^, $+, $?, and even $< (if, for some weird reason, someone used > > .WAIT as the first prerequisite)? > > > > Is it the case that they are required to ALWAYS be present? Or > > required to NEVER be present? Or is the value of internal macros > > not well-defined for prerequisite lists that contain .WAIT? > > The latest note on bug 514 begins, "On D2.1 page 2947 line 98895, > after applying bug 1520, change...". So presumably the intended > text is a synthesis of the results of 1520 [1] and 514 [2]: > > $^ > The $^ macro shall evaluate to the list of prerequisites > for the current target, with any duplicates (except > the first) removed. It shall be evaluated for both > target and inference rules. If the list of > prerequisites of the target contain any .WAIT special > targets, the results of expanding $^ are unspecified. > > [example omitted] Thanks, but what about the other internal macros that may refer to prerequisites? Don't we need to specify the .WAIT behavior for those as well? I would imagine, for example, the same rule as above would be in effect for both the "$+" and "$?" internal macros (I guess for "$?" it's sort of implied that .WAIT can never be considered newer than the target, otherwise the target would always be rebuilt--but maybe that should be made explicit, I dunno).
Re: [1003.1(2008)/Issue 7 0000514]: Enhance internal macros in make
On Fri, 2022-09-09 at 10:19 +0100, Geoff Clare via austin-group-l at The Open Group wrote: > It's intentional. I tried 'all: a b a c' with the GNU, Solaris 11.4 > and FreeBSD implementations and they all expanded $^ (or $> for > FreeBSD) to 'a b c'. Are you suggesting that there are circumstances > where one of these implementations might not preserve the order? No, but the GNU make documentation doesn't state this as a requirement. It says: > $^ > The names of all the prerequisites, with spaces between them. For > prerequisites which are archive members, only the named member is > used (see Archives). A target has only one prerequisite on each other > file it depends on, no matter how many times each file is listed as a > prerequisite. So if you list a prerequisite more than once for a > target, the value of $^ contains just one copy of the name. This list > does not contain any of the order-only prerequisites; for those see > the ‘$|’ variable, below. So if the ordering is required I should update this documentation.
Re: [1003.1(2008)/Issue 7 0000514]: Enhance internal macros in make
On Thu, 2022-09-08 at 20:58 +0200, Steffen Nurpmeso wrote: > |First, this text doesn't mention the .WAIT prerequisites that were > |added as optional features; do we need to add text for how these > are > > Optional? .WAIT:? Only to be victorious over Borisorious! I went back through that (long) issue in Mantis and recalled that indeed it was not optional. I don't have visibility into the current proposed text for the POSIX make definition, but can someone clarify how the .WAIT prerequisites are to be treated when expanding internal macros like $^, $+, $?, and even $< (if, for some weird reason, someone used .WAIT as the first prerequisite)? Is it the case that they are required to ALWAYS be present? Or required to NEVER be present? Or is the value of internal macros not well-defined for prerequisite lists that contain .WAIT?
Re: [1003.1(2008)/Issue 7 0000514]: Enhance internal macros in make
On Thu, 2022-09-08 at 15:53 +, Austin Group Bug Tracker via austin- group-l at The Open Group wrote: > (0005962) geoffclare (manager) - 2022-09-08 15:53 > https://austingroupbugs.net/view.php?id=514#c5962 > - > - > On D2.1 page 2947 line 98895, after applying bug 1520, > change:The $^ macro shall evaluate to the list of > prerequisites for the current target.to:The > $^ macro shall evaluate to the list of prerequisites for the current > target, with any duplicates (except the first) removed. > On D2.1 page 2947 after line 98895 add:$+The > $+ macro shall be equivalent to $^, except that duplicates shall not > be removed; all prerequisites shall appear in the order they were > listed in the makefile This was closed before I had a chance to comment on the wording but a few things: First, this text doesn't mention the .WAIT prerequisites that were added as optional features; do we need to add text for how these are included (or not) in $^? Or, are they just included like any other prerequisite? Or did .WAIT not get included? I can't remember. This was the subject of https://austingroupbugs.net/view.php?id=1520 which itself doesn't mention the removal of duplicate entries, as the above text does. Second, the wording of the $^ implies that when duplicates are seen the it's always the first instance which is preserved and subsequent instances are omitted, which means order is preserved. Is that requirement intentional? I think I'm confused :).
Re: [Issue 8 drafts 0001505]: Make doesn't seem to specify unset macro expansion behaviour
I'm not sure whether continuing to go back and forth on this is fruitful. I certainly understand the issue under discussion but I absolutely do not agree that it's the place of the POSIX standard to try to remediate it by allowing implementations to fail. As the only (as far as I'm aware) representative of make implementers in this conversation, I'm opposed to the change in wording as currently described in #1505 and it should be rejected. It appears that the underlying objection is to the whole idea of importing environment variables as make variables. This proposal is a back-door way to make such usage untenable in portable makefiles, without actually changing the standard to disallow it. The arguments for this change boil down to "we don't think people should do this because it's not reliable" and the arguments against it boil down to "every implementation does this and most every makefile relies on it working this way". Both can be true, but the latter must outweigh the former when deciding what behavior to standardize. If makefile authors want to ensure that their makefiles are not subject to this kind of unreliability they are free to write their makefiles to avoid it. Whether an implementation assists them in that or not is a QoI issue. On Mon, 2021-12-20 at 12:41 +, Geoff Clare via austin-group-l at The Open Group wrote: > > Also, as far as I can tell there's nothing in the standard that > > says that make can't show a warning message even as default > > behavior, if some implementation wanted to do that. > > Then you didn't look far enough. Such as message is not allowed to > standard error because STDERR says the usual "The standard error > shall be used only for diagnostic messages" (which require non-zero > exit). And it's not allowed to standard output because there is > nothing in the STDOUT section that explicitly allows it, and 1.4 > Utility Description Defaults says that STDOUT "completely describes > the standard output of the utility". Fine, then I would be OK with a change to the standard that allowed diagnostics for this situation to be printed to stdout or stderr as a warning, but requiring that make expand the variable to the empty string and continuing to process the makefile without failing. Just to note, I'm not aware of any implementation of make that strictly obeys these rules regarding stdout and stderr. They all either may generate extra output to stdout, or may generate output to stderr while still exiting with a success/0 exit code, or both.
Re: [Issue 8 drafts 0001505]: Make doesn't seem to specify unset macro expansion behaviour
On Fri, 2021-12-17 at 21:34 +0100, Quentin Rameau via austin-group-l at The Open Group wrote: > Why not allowing in the end both behaviours? I feel like we've gone over the problem with this. While the current standard may not guarantee that unset variables expand to the empty string, virtually all makefiles--including intending-to-be conforming makefiles--assume that it does. If you allow a conforming make implementation to fail if it expands an unset variable, you're effectively saying that the only way to have a makefile that will work on all POSIX systems (which is, after all, the entire point of writing a makefile to conform to standard) is to write makefiles that never reference any unset variables. If you write a makefile which does reference an unset variable then your perfectly conforming makefile can fail with a perfectly conforming implementation of make which is clearly a bad situation for the standard. We have two choices: we can add in the text that every makefile author and implementer has assumed was there since the standard was created and require unset variables to expand to an empty string, retroactively blessing all otherwise-conforming makefiles as actually conforming. Or we can allow conforming make implementations to fail, consigning almost every makefile currently existing as allowed to fail on a conforming system and likely rendering the standard irrelevant, at least in this particular way, as authors ignore it. I don't see any way in which the latter is a preferred option. > The standard could state that the result of trying to expand an unset > macro is implementation-defined, this way implementations that want > to expand thoses as an empty string can, and also provide an option > to error out on such behaviour (-u for example, a bit like sh set > -u), for helping the people who want so making their makefiles safer. The standard needs to require that in a conforming make implementation unset variables will expand to the empty string, full stop, for the reasons I've outlined above. There should be no "implementation defined" behavior here. Beyond that, other proposals, such as a new make option that would enable a warning or even an error if unset variables are expanded, would be OK with me (although I don't see why the standard needs to get out in front of implementations on what is basically a quality-of- implementation issue).
Re: [Issue 8 drafts 0001505]: Make doesn't seem to specify unset macro expansion behaviour
On Fri, 2021-12-17 at 21:16 +0100, Quentin Rameau via austin-group-l at The Open Group wrote: > I'm not sure about what standard you're talking, the behaviour isn't > specified in the different make implementation manuals I could read. GNU make has an section that discusses empty variables versus undefined variables, which explains that both expand to the empty string.
Re: [Issue 8 drafts 0001505]: Make doesn't seem to specify unset macro expansion behaviour
On Fri, 2021-12-17 at 21:04 +0100, Quentin Rameau via austin-group-l at The Open Group wrote: > Again, relying on a variable being unset was never 100% safe. This same argument could be made about the shell. Why don't we, while we're at it, make a change to POSIX allowing implementations of sh to either expand an unset variable to the empty string, like every shell does today, or else throw an error and fail, at the discretion of the implementor? Both implementations are equally acceptable and both would be considered standards-conforming. Clearly the latter behavior would result in fewer accidental errors, and clearly it doesn't force any existing shell to change its behavior, so why not do it?
Re: [Issue 8 drafts 0001505]: Make doesn't seem to specify unset macro expansion behaviour
On Fri, 2021-12-17 at 19:52 +0100, Quentin Rameau via austin-group-l at The Open Group wrote: > > On Fri, 2021-12-17 at 18:19 +0100, Quentin Rameau via austin-group- > > l atThe Open Group wrote: > > > Then one can just call make VERBOSE=set, or VERBOSE=set make -e > > I already explained in a previous post why 'make -e' is not > > acceptable. It's a much, much more dangerous than allowing unset > > variables. > > That was just one example out of two, then you can simply use make > VERBOSE=set and be done with it :) There are a number of types of build environments where it's straightforward to add a new environment variable to the build's configuration, but it's not so simple (or even not possible) to add a new command line argument to the invocation of make, which might be buried inside lots of scripting or invoked by other tools. In short, people have evolved their systems and processes to utilize the features provided to them. If you remove those features you'll break things. We should not break things.
Re: [Issue 8 drafts 0001505]: Make doesn't seem to specify unset macro expansion behaviour
On Fri, 2021-12-17 at 18:19 +0100, Quentin Rameau via austin-group-l at The Open Group wrote: > Then one can just call make VERBOSE=set, or VERBOSE=set make -e I already explained in a previous post why 'make -e' is not acceptable. It's a much, much more dangerous than allowing unset variables. If it were possible to do something like "make -e VERBOSE" so that the override only impacted that one variable I guess it would be a way to go, but -e forces ALL environment variables to take precedence over ALL makefile-set variables.
Re: [Issue 8 drafts 0001505]: Make doesn't seem to specify unset macro expansion behaviour
On Fri, 2021-12-17 at 10:11 +, Geoff Clare via austin-group-l at The Open Group wrote: > Paul Smith wrote, on 16 Dec 2021: > > I'm somewhat nonplussed by this conversation. > > > > We have a POSIX tool, that's been in use for 45 years. All known > > implementations of that tool provide a specific behavior. No known > > implementations of that tool provide some different behavior. Users > > of the tool have written hundreds of thousands of files that expect > > and rely on the provided behavior and feel that it's valuable. > > They rely not only on the behaviour of make but also on macros being > unset by default. They are wrong to rely on the latter, because it > can easily not be the case depending on circumstances. They are not wrong, because that's exactly the behavior they want: they want the variable to expand to empty when it's not set, and expand to something else when it is set. And they're also not wrong because up until now (with the advent of ?=) there has been no other way for them to get the behavior they want in POSIX standard makefiles. It may be that they were relying on behavior that is not explicitly stated. But I can't understand why the right answer to that problem is not to standardize the behavior that everyone gets today and expects to get, and instead allow behaviors which would break all those assumptions. Whether we wish that some other behavior had been chosen 45 years ago, is irrelevant. POSIX standardizes what is, not what might have been. > I would not be surprised if there have been many cases over those 45 > years where this very problem has arisen and the makefile author has > simply fixed it either by changing the macro name or adding an > assignment. Users would tend to think "that's how make works, so > it's my problem and anyway it's simple to fix" rather than trying to > get make changed so as to prevent the same problem arising in future. make cannot ever change in this way, practically, because that would break almost every makefile in existence. There are many annoying things required by the POSIX standard because "that's how it's always worked" and since all implementations have that behavior, it's added to the standard. Such is life. > > Which by the way, no implementation will ever do > > because it would break the previously-mentioned hundreds of > > thousands of files. > > If you are so sure that no implementation will ever do that, then I > don't see that you have any reason to complain about the proposed > change to the standard. I'm not speaking as the maintainer of GNU make here. This change makes no difference to me as the maintainer because GNU make will certainly never change this behavior. I'm speaking on behalf of all the users who maintain makefiles. This has a big impact on them, unless they simply want to say "we won't bother to care about ensuring that our makefiles are POSIX compliant" which benefits no one, most especially POSX itself. > The more I think about it, the more I am convinced that an error > is the right thing for make to do, and I would urge you and the > other implementors to provide a means (e.g. a command line option) > to turn it on, and have manuals encourage its use. That is a QOI issue, not a standards issue. GNU make has had an option that will warn when expanding unset variables for almost 30 years. This is one reason I'm sure that throwing an error is a complete non-starter. If we wanted to add such an option to the standard, or even an option that forced an error, not just a warning, in this situation, I would be OK with that. Also, as far as I can tell there's nothing in the standard that says that make can't show a warning message even as default behavior, if some implementation wanted to do that. But, it must absolutely not be the case that the standard allows implementations to throw an error when expanding an unset variable as the default behavior.
Re: [Issue 8 drafts 0001505]: Make doesn't seem to specify unset macro expansion behaviour
I'm somewhat nonplussed by this conversation. The goal of POSIX (as I understand it) is to standardize existing behaviors of tools and libraries so that USERS of these tools and libraries can feel confident that when they move to another conforming platform, everything will still work the same way. We have a POSIX tool, that's been in use for 45 years. All known implementations of that tool provide a specific behavior. No known implementations of that tool provide some different behavior. Users of the tool have written hundreds of thousands of files that expect and rely on the provided behavior and feel that it's valuable. This is the quintessentially perfect situation for standardizing the in-use behavior. There's no role for POSIX, IMO, to say that this behavior can no longer be relied on and some future implementation may instead substitute another behavior. Which by the way, no implementation will ever do because it would break the previously-mentioned hundreds of thousands of files. On Thu, 2021-12-16 at 10:46 +, Geoff Clare via austin-group-l at The Open Group wrote: > Quentin Rameau wrote, on 16 Dec 2021: > > That would indeed be the correct way to do it, you found it > > yourself so anybody else can do it to! And so? Does the fact that it CAN be done mean it's OK to FORCE everyone to do it? > > As a note, it's unneeded to use the ?= operator, classical = > > operator is enough, macros specified on the command line overtake > > the one defined in make files, and environment ones do that with -e > > option if that's what the user wants. > > Classical = operator is exactly what it needed to avoid relying on > any given macro expanding to an empty string. Classical = is not sufficient because it overrides environment variables. Build systems want to allow environment variable settings to impact their behaviors. Whether we think it's the right thing or a bad idea or whatever is irrelevant. They want to do it, implementations let them do it, and it's not the place of the standard to tell them they can't. Also, using -e is a complete non-starter. I personally think that if our actual goal is to improve the safety and reliability of make-based build systems the very first thing we should do is remove -e. Using "make -e" is virtually guaranteed to break your build almost before it starts.
Re: [Issue 8 drafts 0001505]: Make doesn't seem to specify unset macro expansion behaviour
On Tue, 2021-12-14 at 10:15 +, Geoff Clare via austin-group-l at The Open Group wrote: > Effectively, if a makefile expects an unset macro to expand to an > empty string, this is a bug waiting to be triggered. Can you clarify the reason for this position? I don't agree with it. Is there any implementation of make anywhere that would treat this as a bug? There are hundreds of thousands of makefiles out there that rely on this behavior and there has never been a single report (that I'm aware of) that any of them have ever failed due to this. Let me reiterate my example (which is used in every single automake generated makefile, plus many many others) from the comment I added to the issue: Makefiles contain this: $(VERBOSE).SILENT: Now when you run "make", the .SILENT: feature is enabled. If you run "make VERBOSE=1" (or any other value) then the .SILENT: feature is disabled. Up until now this has been perfectly correct. Of course it would be possible to require all these makefiles to be rewritten as: VERBOSE ?= $(VERBOSE).SILENT: (leaving aside that ?= is not part of the currently published version of the standard) but I see no reason to make the original behavior non- conforming, which immediately changes all these otherwise-correct makefiles to be potentially wrong.
Re: [Issue 8 drafts 0001505]: Make doesn't seem to specify unset macro expansion behaviour
Has anyone considered my comment on: https://austingroupbugs.net/view.php?id=1505 ? I'm unhappy with the resolution of this issue, to allow a conforming make to throw an error when expanding an unset variable. No implementation I'm aware of does that or ever did that, and by allowing it we're suddenly changing tons of makefiles that would otherwise work properly to be indeterminate since it's allowed for a conforming implementation to throw such an error. What's the point of allowing this? Why shouldn't we just state that make implementations must expand unset variables to the empty string, which is what all implementations (that I'm aware of) do anyway?
Re: [Issue 8 drafts 0001471]: Add an orthogonal interface for immediate macro expansion definitions to make
On Thu, 2021-09-09 at 10:07 +0100, Geoff Clare via austin-group-l at The Open Group wrote: > You took a risk when you added ::= to gmake while it was only an > ccepted proposal, not part of an approved revision to the standard. > And gmake users who make use of it in the expectation that is will > be required by POSIX are taking a similar risk. This is a fine position when POSIX is following it's normal charter which is to standardize existing behaviors that have been proven to be useful or at least widely used. It's a problem when POSIX is inventing new syntax, like it is here. Especially if issues are only published every 10 years or more. That position suggests that implementations will not make new features available until AFTER they have been standardized by POSIX, else risk introducing backward-incompatible behavior to their users. That makes the standard poorer; I'm sure we're all aware of the dangers of standardizing before implementing. In any event, I don't see why this particular feature should be different than any other: if GNU make had added ::= on its own, not due to some putative change in POSIX, and it had 13 years of use in the wild under its belt, I doubt there would be much consideration of adding ::= to the standard in an incompatible way.
Re: [Issue 8 drafts 0001471]: Add an orthogonal interface for immediate macro expansion definitions to make
On Wed, 2021-09-08 at 17:13 +0200, sch...@schily.net wrote: > That as introduced by accident, because I did not realize at that > time that gmake used an icompatible implementation that differs from > smake and BSD make. I don't see how it can be possible that there was any confusion about this. That was the entire thrust of the decision to create a new ::= operator rather than using :=, because := was not compatible between GNU make and BSD make. If the implementations were compatible we would not have needed a different operator at all! > There is plenty of := usage in BSD makefiles and :::= just > standardzizes what BSD make and smake implemented via := before. No doubt. But, that is not what I asked. I asked for examples, or explanations of situations, where using the POSIX ::= operator as currently defined isn't sufficient, and the different behavior of the :::= operator is required instead.
Re: [Issue 8 drafts 0001471]: Add an orthogonal interface for immediate macro expansion definitions to make
On Wed, 2021-09-08 at 11:10 +0200, Joerg Schilling via austin-group-l at The Open Group wrote: > The :::= operator has been implemented in two independent make > programs before it was standardized. > > The :::= operator was introduced to smake and SunPro Make in March. I can't consider this a real precedent. Discovering ::= didn't work as expected then inventing a new operator and filing a request to standardize it almost the same day, and leveraging one person's maintenance of two different make sources to create "independent make programs", is just gaming the system. Everyone is explaining why it's OK to add :::= but I still have not seen any examples of the UTILITY of :::=. That's what I'm asking for. If there's real precedent for it then surely someone can show me some makefiles somewhere that use :::= (or even BSD make := since, comments about prior implementation notwithstanding, I would guess there isn't a single real makefile out there today that uses :::=) in a way that can't be implemented with ::= instead. I'm not just trying to pee in the pool here. I legitimately need some clarity on where :::= is useful. Without that I can't write documentation for it and without docs it can't be added to GNU make.
Re: [Issue 8 drafts 0001471]: Add an orthogonal interface for immediate macro expansion definitions to make
On Wed, 2021-09-08 at 09:29 +0100, Geoff Clare via austin-group-l at The Open Group wrote: > Personally I see no reason to prefer one over the other. This leads > to three choices for what goes in Issue 8: > > 1. Just an operator that works like gmake := > 2. Just an operator that works like BSD make := > 3. Two operators, one like gmake := and one like BSD make := > > We have decided to go with option 3. This is not the right way to look at it. If there truly is NO REASON to choose one over the other then it's wrong to standardize both because now you are just inventing two ways to do things which have no practical difference between them. If it was the case that lots of makefiles already used two DIFFERENT operators that did things differently, then there would be a good argument for adding both, so that lots of EXISTING makefiles could be made portable with no work on their part. That's a good goal. But unfortunately we do not have that happy situation here: here we have lots of makefiles that use the SAME operator that works differently, and no makefiles that implement :::= and so no makefiles will become portable by making this operator standard. Thus the only reason we should standardize both is if there IS some reason to choose the behavior of :::= over ::=. If there's no reason to choose one over the other, and makefiles have to change their operator to become standard anyway, then they can just change their operator to ::=. > > But here we're inventing an entirely new operator that NO VERSION > > of make currently implements (yes, I understand that BSD make has a > > different operator that works in this same way but that's not the > > same thing: no existing makefile today contains the :::= operator > > so every makefile that wants to use it will have to be changed, and > > in a way which is not backward-compatible with older versions of > > make). > > At the time that we accepted a proposal to add ::= no version of make > implemented it. So your argument applies equally to that addition. No, that's not right. In issue 7 there is no way to have any sort of immediate expansion in standard make. That's clearly something that users wanted (for the record note that I was not the one who wanted this standardized: I didn't propose it or push for it in any way; it was users who wanted this). The ::= operator added immediate expansion. That's certainly a useful addition, and worthy of creating a new operator for. The only legitimate (IMO) reason to add ANOTHER operator :::= is, as I've been trying to understand, there's some characteristic of that behavior that people would find useful enough to change their makefiles to use it, that they can't get by changing their makefiles to use ::= which is already accepted. > Just because the proposal for ::= was applied to an earlier Issue 8 > draft than :::= doesn't mean it has any claim to be treated > differently as part of the overall changes from Issue 7 to Issue 8. If we wanted to have that discussion we should have had it back before ::= was accepted. At this point, ::= DOES have a claim to be treated differently because there IS ample implementation precedent for it: as a result of the previous decision back in 2011, GNU make has been providing the ::= operator now for almost 8 years (released in GNU make 4.0 in October 2013). It can't be changed now. We could remove ::= from the standard and instead add :::= but that seems useless to me: there are real makefiles out there using ::= which would be made NON-PORTABLE by removing ::= from the standard.
Re: [Issue 8 drafts 0001471]: Add an orthogonal interface for immediate macro expansion definitions to make
On Tue, 2021-09-07 at 18:30 -0700, Nick Stoughton wrote: > The problem we were trying to address with this change is that > bsd make (bmake) and GNU make both have a := operator, but they > behave differently. We originally added ::= to match the gmake > behavior. The idea with :::= is to match the bmake behavior. OK, but that doesn't really answer my question. "We added this version so now we have to add that version too or else it's not fair" doesn't seem like a good reason to invent brand new syntax in the POSIX spec. There must be some reason for the BSD model to be added; there must be some important or useful feature that it enables, that is impossible, or difficult, or at least annoying to achieve in other ways. If this was a feature that already existed (that is, the operator :::= was already implemented and used by makefiles) then there's an argument for adding it to the standard just to make it official: standardizing existing practice has value. But here we're inventing an entirely new operator that NO VERSION of make currently implements (yes, I understand that BSD make has a different operator that works in this same way but that's not the same thing: no existing makefile today contains the :::= operator so every makefile that wants to use it will have to be changed, and in a way which is not backward-compatible with older versions of make). People who are writing BSD makefiles already have this feature: they can just use BSD's := operator. The only makefiles that would use :::= are makefiles that want to be portable, and that need this particular capability. So what I'm asking is, what does this operator enable, beyond what we have with "=" and "::=", that is helpful in writing portable makefiles? In what situations would a makefile need to use :::=? Can someone provide some use cases? > So, extracting from the etherpad of 2021-08-12: > > Options: > > > > > > 1. Add :::= with behavior of BSD := with $$ always preserved > > (.MAKE.SAVE_DOLLARS either doesn't affect :::= or is an extension) > > > > > > 2. Add :::= with behavior of BSD := with $$ never preserved > > (.MAKE.SAVE_DOLLARS is an extension, nominally required to be false > > by default if implemented so existing scripts don't break if > > implementation applies it to additional assignment types other than > > :::=, imo) > > > > > > 3. Add :::= with behavior of BSD := with .MAKE.SAVE_DOLLARS, > > default true > > > > > > 4. Add :::= with behavior of BSD := with .MAKE.SAVE_DOLLARS, > > default false > > > > > > 5. Don't add :::= > > I believe we opted for #4. Based on my reading of the text actually appearing the in current draft, it appears that the option chosen was #1 above, not #4: the text says that "$$" is preserved and there's no mention of .MAKE.SAVE_DOLLARS anywhere. If not I'm even more confused.
Re: [Issue 8 drafts 0001471]: Add an orthogonal interface for immediate macro expansion definitions to make
On Thu, 2021-08-12 at 15:19 +, Austin Group Bug Tracker wrote: > After page 2894 line 97263 add: > > The following form defines a delayed-expansion macro > (replacing any previous definition of the macro named by > string1): > > string1 :::= [string2] > > by immediately expanding macros in string2, if any, before > assigning the value. > > On page 2895 line 97310, change: > > The macro $$ shall be replaced by the single > character > '$'. > > to: > > The string "$$" shall be replaced by the single > character > '$', except during the immediate expansion performed for the :::= > operator, > where it shall be left unmodified. So I took a look at implementing this in GNU make since it seems to have been adopted. I just can't understand the point of this change. Can someone provide me with some example(s) of this :::= operator showing how it might be used and what problems it is trying to solve? The idea behind this completely escapes me so I'm wondering if I'm somehow misinterpreting the text above defining how :::= is supposed to work. I can sort of understand why someone might want to immediately expand a variable but then treat the expansion as delayed-expansion (although as I pointed out in the issue comments, you can already do this without this new operator you just have to use two variables, and I haven't had anyone ask me for this capability). But what is the point of having an immediately expanded variable where variables are expanded but "$$" is preserved? This I can't really fathom. Also, it's not clear to me whether preserving the '$$' only applies to the unexpanded string, or also applies to the expansion of variables contained in the string. For example, what would be the expected result of this: foo = one$$two bar :::= $(foo) three$$four all:;@echo '$(bar)' Is the internal value of the "bar" variable 'one$$two three$$four', so that the result of running make is: one$two three$four or is the internal value of the "bar" variable 'one$two three$$four', so that the result of running make is: onewo three$four
Re: [Issue 8 drafts 0001471]: Add an orthogonal interface for immediate macro expansion definitions to make
On Tue, 2021-09-07 at 15:06 +0100, Geoff Clare wrote: > Already reported in https://austingroupbugs.net/view.php?id=1513 Oh thanks! I must have missed that in my inbox.
Re: [Issue 8 drafts 0001471]: Add an orthogonal interface for immediate macro expansion definitions to make
On Thu, 2021-08-12 at 15:19 +, Austin Group Bug Tracker wrote: > A NOTE has been added to this issue. > == > https://austingroupbugs.net/view.php?id=1471 > == Reviewing a draft of the standard I think there may be some places that should reference :::=, that don't. In particular in the paragraph starting at line 98687 (on my draft) which starts: > In all cases the value of string1 is defined as all characters from > the first non- character to the last non- character, > inclusive the other assignment operators are listed but :::= is omitted.
Re: [1003.1(2016/18)/Issue7+TC2 0001437]: make: (document .NOTPARALLEL and .WAIT special targets) in RATIONALE
On Sat, 2021-08-28 at 02:38 +0200, Steffen Nurpmeso wrote: > So to get to the gory details was a bit more complicated than i first > thought, but i can offer a public domain patch for GNU make (git > [012918bf11]) Hi Steffen. Thanks for working on these changes. Please start a discussion of this on the bug-m...@gnu.org mailing list; that's where all patches and discussion of new features should happen, so that all users of and contributors to GNU make can participate. I haven't looked at this in detail so I can't comment on the approach taken here.
Re: [1003.1(2016/18)/Issue7+TC2 0001437]: make: (document .NOTPARALLEL and .WAIT special targets) in RATIONALE
On Sat, 2021-08-21 at 02:24 +0200, Steffen Nurpmeso via austin-group-l at The Open Group wrote: > Words like quick-dirty and hack i think are really displaced for > the nice and elegant solution that .WAIT: is. > > I remember doing GNU make files and how i had to mess with stamp > files in order to get at least a somehow reliable ordering. > It may be some time but still. Without knowing what you were trying to do all I can say is that the entire PURPOSE of make is to impose a reliable ordering on the build. I mean, you can just say: foo: bar and now you have a reliable ordering that bar will be completed before foo can be started, regardless of how high your -j value is or where else these targets appear in the makefile. If that's not sufficient for some particular use-case (and it might not be, as I discussed in my comments) then yes, more effort may be needed. > .WAIT: is in use in the BSD world in their holistic makefile > system for decades, and it enables people to write natural > looking makefiles that can be understood at a glance, even after > long work hours, and even after coming back to some project that > has not been looked at in a long time. Yes, I already said they're easy to read. But, they're easy to misuse and misunderstand. Maybe that's a tradeoff that works fine in specific situations (for example, BSD makefiles typically use .WAIT inside lists of singleton "top level" targets like subdirectories so they don't run into these issues because the targets only appear in only one target's prerequisite list) but "easy to read and hard to use reliably" does not make a great candidate for POSIX, at least IMO. > Consider for example > > openssl/Makefile:SUBDIR= lib .WAIT bin Why not just say: bin: lib ? > openssl/lib/Makefile:SUBDIR+= .WAITlibssl # depends > on libcrypto Why not just say: libssl: libcrypto ? and now you don't even need the comment: it's self-documenting. Also this form doesn't depend on the order that the SUBDIR variable was created with: the .WAIT version doesn't actually say "you can only build libssl after libcrypto", it says "everything before this point in the SUBDIRS variable must be completed before libssl (and anything that appears in SUBDIRS after libssl) can be started" which is a much stronger, in fact probably stronger than necessary, statement. In my opinion GNU make's solution: bin: | lib libssl: | libcrypto is simply better. It describes precisely what the relationship is and it is built into the dependency graph itself so takes effect wherever these targets are used. > Today it is only because of GNU make luckily supporting > .NOTPARALLEL: that portable makefiles which can make use of -j > can be written at all, _if_ following Wheeler's "everything can > be done with an indirection", as shown in the introductional > comment. I find it quite unbelievable that the ONLY way to write a portable makefile using -j is to use .NOTPARALLEL in GNU make. I certainly would not be surprised to learn that this is a preferred shortcut by folks who want to convert an existing BSD makefile that uses .WAIT to be portable to GNU make, without actually changing the makefile to use portable methods. But, that is not the same thing. > It seems to me GNU make could easily implement .WAIT:, even > though with slightly different semantics, by creating a "pipe" > rule of all prerequisites yet encountered on a line when > a .WAIT is seen? Well, everyone is welcome to their own opinion on the amount of effort needed for this work: I've given mine. I'll just point out, again, that the "create an internal rule" idea is not something new which allows me to have a "eureka" moment where I suddenly see a way to implement this feature in a few lines of code. I mentioned this as a possible way forward in my comments in Dec 2020, two days after the issue was filed. If we really wanted to implement this the easiest possible way, it would be simpler to implement like this: whenever a .WAIT target is seen, GNU make pauses ALL new jobs and waits for all existing jobs to complete before it proceeds. That would reduce the value of -j when you use .WAIT but it would meet the requirements of the wording AFAICT.
Re: Fwd: Re: [1003.1(2016/18)/Issue7+TC2 0001436]: make: add "-j max_jobs" option to support simultaneous rule processing
On Sat, 2021-05-22 at 00:59 +0200, Steffen Nurpmeso via austin-group-l at The Open Group wrote: > Then stating something like "i am process X, and my parent is Y" > etc. And the rest being up to the make(1) implementor as a quality > of implementation (scheduling, fair even more so, seems to reside in > the area of very complicated programming). I was surprised to see > that rule content matters at all. The problem is that depending on the implementation, the parent make might need to pass actual resources to the child make instance. Those resources could negatively impact child processes that are NOT make. In that case, make needs to know whether the thing it is invoking is another make or not, so it knows whether to pass those resources. As an example, in GNU make today we use a simple pipe to implement this feature, which means child makes need to have the open file descriptors for the ends of the pipe provided to them when they are started. But, some other programs that are not make, might not work well if they are invoked with extra file descriptors already open, and if they write garbage into the jobserver pipe then it will break things. So in this implementation make would want to close-on-exec those file descriptors before invoking the sub-process, IF it knows the sub-process is not a make instance. Of course, for an implementation which can assume POSIX most likely this is NOT the method that would be chosen to implement the jobserver as it leads to complications. GNU make, however, is part of a system bootstrap toolchain (needed to build a compiler for an old system to start to update it for example) so I try to keep it portable to very old systems. Nevertheless, I've been considering switching GNU make's implementation to a named pipe (mkfifo) or a POSIX semaphore. These implementations would resolve all the above issues (and others, such as blocking/non- blocking FDs etc.) since only a sub-make would access the named pipe. Anyway, that's not really relevant but just FYI as to why (I suspect) the standard was worded like this.
Re: [1003.1(2016/18)/Issue7+TC2 0001436]: make: add "-j max_jobs" option to support simultaneous rule processing
On Tue, 2021-04-27 at 10:30 -0700, Nick Stoughton wrote: > In short ... you should still be able to add new comments against > this issue, and it appears that you have succeeded in doing this! > Thanks for your review... Thanks for the explanation Nick. The terms are a little intimidating to those who aren't familiar, but, as we all know, naming is hard. I shall save your email for future reference. Cheers!
Re: [1003.1(2016/18)/Issue7+TC2 0001436]: make: add "-j max_jobs" option to support simultaneous rule processing
On Thu, 2021-04-22 at 15:36 +, Austin Group Bug Tracker via austin- group-l at The Open Group wrote: > The following issue has been RESOLVED. > = > https://austingroupbugs.net/view.php?id=1436 > = > Reported By:steffen > Assigned To: > = > Project:1003.1(2016/18)/Issue7+TC2 > Issue ID: 1436 > Category: Shell and Utilities > Type: Enhancement Request > Severity: Editorial > Priority: normal > Status: Resolved Just to point out: this issue was prematurely marked RESOLVED and "Accepted as Marked" before comments could be provided on issues in the text that need to be addressed before it should be added to the standard. I'm not sure what the proper lifecycle is for an issue is, but I don't think adding proposed text into the issue, and at the same time marking the issue RESOLVED with no time allowed for comments first, is correct. But, maybe I'm just misunderstanding what the RESOLVED status actually means. Cheers!
Re: [Shell Command Language][shortcomings of command utlity][improving robustness of POSIX shells]
On Sat, 2021-04-10 at 22:12 +0200, Joerg Schilling via austin-group-l at The Open Group wrote: > where ??? what is that? I don't know if this is what Jan was thinking of, but "where" exists on Windows and is equivalent to "which" etc. there.
Re: [1003.1(2008)/Issue 7 0000513]: Add pattern rules (metarules) to make
On Fri, 2020-12-04 at 02:44 +0700, Robert Elz via austin-group-l at The Open Group wrote: > | The fact that bmake prints a warning if you try to "overwrite" a rule > | is another hint that my proposal to first require an explicit removal > | with the same pattern and no command before permitting to redefine a > | rule is the better way for a potential future. I have no problem with POSIX requiring this, since it is compatible with how GNU make works, as long as POSIX does not specify a behavior (such as a warning) if a pattern is redefined _without_ first being removed.
Re: make pattern rules (was: Re: [Issue 8 drafts 0001325]: Allow make to remake an included file)
On Mon, 2020-11-09 at 14:16 +0100, Joerg Schilling wrote: > Paul Smith via austin-group-l at The Open Group < > austin-group-l@opengroup.org> wrote: > My idea is to start with a subset of the features of the original > implementation and to add some commonly *agreed* new features. Sure. > > I don't understand the distinction you're trying to draw between > > "disabled" and "deleted". Regardless, if you define a pattern rule > > with no commands then the previous pattern is gone and cannot be > > "re- > > enabled", so I think "deleted" is the correct term. > > Well, the fact that gmake still lists a "deleted" rule with gmake -p > is confusing users. I don't agree: it tells users that someone deleted that rule which I think is a useful thing to know. It's clear from the fact that no commands are shown, that this pattern cannot be used. > > Yes, that could be done. > > > > It's different than MAKEFLAGS += -r, however, because the latter is > > guaranteed to only delete the _built-in_ rules. This is almost > > always what the user wants. > > As mentioned before: internal rules need to be parsed before reading > the users Makefile, so a line in the form > > MAKEFLAGS += -r > > in the users Makefile is inefficient unless you ignore the rules for > the precedence of definitions that are in POSIX. I'm afraid I don't see the issue. It's not hard to keep a flag in any default rule specifying that it was created as a default, and either delete them all or just ignore them when searching rules, if they are disabled. Or, depending on the implementation, you can keep the default rules in a separate list and not search that list if the default rules are disabled. There are multiple implementation possibilities, which are not inefficient. > > Having "%:%" delete ALL the existing pattern rules is problematic > > because it relies on a specific ordering of appearance in > > makefiles. I'm sure such a feature would lead to a lot of head- > > scratching followed by cursing once a stray "%:%" pattern was > > discovered down in the bowels of the makefile system. > > Given that pattern rules _are_ based on the concept of a specific > ordering, this is a perfect enhancement. I disagree. The thing that users want is to remove the built-in rules. I see no reason why we should force them to add this special line as the first thing in their makefile to be sure they don't accidentally remove their own pattern rules. We don't require people to set .SUFFIXES: as the first thing in their makefile, before they define any suffix rules, for example. > > I would prefer to use a special target which forced all built-in / > > system rules (only) to be dropped, something like: > > > > .NOBUILTINS: > > This may be a useful potential enhancement, but this would still > differ from > > $make -r Since we are making it up there's no reason that it needs to differ from "make -r": we can just say that it doesn't :). > since .NOBUILTINS: would only affect rules from the internal > makefile, but not disable macro defitions. "make -r" does not disable macro definitions either. The standard says: > -r > Clear the suffix list and do not use the built-in rules. That does not mention macros anywhere, and in fact GNU make does not remove default macro definitions when given -r. This is the expected behavior because often makefiles will use the default macro definitions when defining their own rules, even if they are invoked with "make -r".
make pattern rules (was: Re: [Issue 8 drafts 0001325]: Allow make to remake an included file)
Unless we are proposing pattern rules for inclusion in POSIX, which I personally would be in favor of since they are much more useful than suffix rules, perhaps we should move the discussion to a different list? On Fri, 2020-11-06 at 13:25 +0100, Joerg Schilling wrote: > > I'm not sure where the idea that pattern rules are immutable comes > > from: in GNU make any pattern rule can be overwritten at any time, > > including the default pattern rules. > > This is how pattern rules have been defined when they appeared in > SunPro Make in January 1986. Well, it's not how pattern rules in GNU make work. > The fact that gmake overwrites existing pattern rules is not > sufficient (as it does not permit to remove them) That is not the case. You can absolutely remove them. > and it is a non-compliance related to the original definition so > this modification from gmake causes unpredictable behavior. There is non-compliance with the POSIX standard. There are differences of implementation in areas that are not covered by the standard. There is no such thing as non-compliance with areas that are not covered by the standard. > Well after some searching (documentation could be better), I > discovered that gmake seems to disable (not to delete) pattern rules > if a rule with the same pattern but no command is defined. This is > not compliant to the definition of how pattern rules work. I don't understand the distinction you're trying to draw between "disabled" and "deleted". Regardless, if you define a pattern rule with no commands then the previous pattern is gone and cannot be "re- enabled", so I think "deleted" is the correct term. I think the docs are pretty clear: https://www.gnu.org/software/make/manual/html_node/Canceling-Rules.html > This is non-compliant behavior Pattern rules are not part of the POSIX standard, so "non-compliant behavior" is not an appropriate term. I'm happy to discuss incompatibilities between SunPro make/smake and GNU make as long as the discussion proceeds from a position that they are two co-equal implementations each with their own long history and (certainly in the case of GNU make) vast user bases that rely on the existing behaviors. However, I don't have the energy for discussions that begin with the premise that everything that exists in SunPro make/smake is canonical and any behavior in GNU make that differs from that is "non-compliant". > The series: > > %.o: %.c > echo some command > > %.o: %.c > > could be defined to require to completely remove the first definition > from the chain of pattern rules. That is already the way GNU make works. Although I don't know what you mean by the "first definition"; there can only be one instance of a given pattern rule (note, not one instance of a given pattern _target_; obviously many pattern rules may be defined with the same target). > As a result, the series: > > %.o: %.c > echo some command > > %.o: %.c > > %.o: %.c > echo some new command > > could be defined as to first define a pattern rule, then to > completely remove it from the chain of pattern rules and to finaly > add a new rule with a different command to the current end of the > pattern rule chain, without being completely incompatible to the > existing pattern rule specs. There is no single "spec" for pattern rules, there is only how SunPro make handles pattern rules and how GNU make handles pattern rules. In GNU make there's no need to delete the pattern rule before you define a new one. You just define the new one and it replaces the existing one. I don't see the point in requiring people to delete the pattern rule first. What does that buy you? > To fix this problem, the nonsense pattern with no command definition: > > %: % > > could be defined as the method to completely remove all existing > pattern rules from the current chain of pattern rules. Yes, that could be done. It's different than MAKEFLAGS += -r, however, because the latter is guaranteed to only delete the _built-in_ rules. This is almost always what the user wants. Having "%:%" delete ALL the existing pattern rules is problematic because it relies on a specific ordering of appearance in makefiles. I'm sure such a feature would lead to a lot of head-scratching followed by cursing once a stray "%:%" pattern was discovered down in the bowels of the makefile system. Possibly even accidentally as the result of a variable expanding to empty unexpectedly... A number of the enhancements to GNU make are intended to reduce the reliance on order of definition as much as possible. Users do want and rely on these enhancements. I would prefer to use a special target which forced all built-in / system rules (only) to be dropped, something like: .NOBUILTINS: This would provide the identical behavior as "make -r", but without modifying MAKEFLAGS. This seems like something that could easily be added to the standard as well, without even considering pattern
Re: [Issue 8 drafts 0001325]: Allow make to remake an included file
On Thu, 2020-11-05 at 15:42 +, Geoff Clare via austin-group-l at The Open Group wrote: > > I agree we can't over-simplify but I don't see a problem with the > > specific case you mention. > > If the first target_name operand is the name of an include file that > needs to be created, your statement "All include file regeneration is > complete before make attempts to bring [up-to-date] the > first target_name operand" creates a paradox. I don't really see a paradox here, sorry. Maybe I'm just not good enough at detailed reading. All include file generation is performed, then make attempts to bring the operands up to date. Why is there a paradox if an operand is also an include file? But I don't see a problem per se with the current wording so I'm happy to let this thread die as an academic discussion. Thanks Geoff!
Re: [Issue 8 drafts 0001325]: Allow make to remake an included file
On Thu, 2020-11-05 at 19:36 +0100, Joerg Schilling wrote:
> .SUFFIXES:
> .SUFFIXES: .o .c
> .c.o:
> echo foo
>
> In other words, a users makefile has full control over the "builtin"
> default rules even without a need to use "make -r".
Sure, it's easy to redefine a suffix rule. The issue in question was
that Geoff wanted to not use the .c.o rule at all. He wanted to use
other rules to build .o files in other ways, but the .c.o rule was
interfering with it. There's no way to DELETE that .c.o rule other
than -r.
However as I mentioned in my previous message you can still do it
without requiring -r, if you're very careful to order your suffixes
correctly so that the built-in .c.o rule is not considered.
> > In my opinion the -r option is wrong.
>
> Well GNU make makes the mistake to define pattern matching implicit
> rules in its default rules and this causes the following problems:
This is irrelevant to the topic at hand.
In fact, GNU make allows users to disable built-in rules completely
WITHOUT having to require users to add -r to the make command line,
because it allows users to add this to their makefile:
MAKEFLAGS += -r
and it does the right thing. Of course this is problematic as well
because that is now passed to sub-make invocations which may not be
what you want. However, in practice it is rarely a problem.
In fact, most of the built-in rules in GNU make ARE defined as suffix
rules; see this from default.c:
".c.o",
"$(COMPILE.c) $(OUTPUT_OPTION) $<",
".cc.o",
"$(COMPILE.cc) $(OUTPUT_OPTION) $<",
".C.o",
"$(COMPILE.C) $(OUTPUT_OPTION) $<",
".cpp.o",
etc.
However, GNU make also provides a number of built-in rules that cannot
be expressed as suffix rules, such as these:
/* RCS. */
{ "%", "%,v",
"$(CHECKOUT,v)" },
{ "%", "RCS/%,v",
"$(CHECKOUT,v)" },
{ "%", "RCS/%",
"$(CHECKOUT,v)" },
> - pattern matching rules have precedence over suffix rules and may
> themselves not be overwritten
>
> - pattern matching rules match in the order of their apperance to the
> parser. The first pattern rule that matches a specific pattern is
> used. Later specifying a replacement rule for a specific pattern is
> impossible.
>
> - The built in default rules must to be "read" by make before the
> users makefiles are parsed in order to allow them (the suffix
> rules) to be overwritten. Any pattern matching rule that appears
> in the builtin default rules thus would have precedence over
> anything else and cannot be overwritten.
Maybe these are problems in SunOS make / smake, but they are certainly
not problems in GNU make.
I'm not sure where the idea that pattern rules are immutable comes
from: in GNU make any pattern rule can be overwritten at any time,
including the default pattern rules.
Also, pattern rules can be completely canceled without overwriting
them, including as well the default pattern rules. It's just annoying
to have to do it by hand, and make sure you get them all, so something
like MAKEFLAGS += -r is much more robust.
Also, user-defined pattern rules are always added to be searched before
built-in pattern rules so the first one will always be a user-defined
pattern, if such exists.
And finally, it's not actually true that the first pattern rule that
matches is used. That used to be the case but quite a while ago the
pattern matcher was changed so that the rule with the shortest matching
stem is chosen first. If multiple pattern rules have stems with the
same length then the first one is chosen.
This algorithm change is a little controversial and I'm not sure about
it, but it seems to be acceptable in practice and it does avoid
worrying about the order in which patterns are defined.
Re: [Issue 8 drafts 0001325]: Allow make to remake an included file
On Thu, 2020-11-05 at 09:25 +, Geoff Clare via austin-group-l at The Open Group wrote: > > .SUFFIXES: > > .SUFFIXES: .c .i .o > > > That doesn't make any difference, since .c and .o are both in the > specified suffixes, so that brings the default .c.o rule into play. Sorry, I didn't read your example closely enough. Not only do you have to remove pre-existing suffixes, but you have to be very careful to order the suffixes correctly when you redefine them. I find the text in the standard somewhat confusing (particularly how when first introducing inference rules they are shown as ".s1.s2" but then a few paragraphs later this is changed to ".s2.s1"; if you miss that switch then the rest of the text doesn't make any sense). The order in which suffix rules are found depends on the order in which the suffixes appear in .SUFFIXES. To make your example work you need the order to be this: .SUFFIXES: .SUFFIXES: .o .i .c rather than ".c .i .o"... in particular you want the .c suffix to be after the .i suffix so that rules with .i are preferred.
Re: [Issue 8 drafts 0001325]: Allow make to remake an included file
On Thu, 2020-11-05 at 09:57 +, Geoff Clare via austin-group-l at The Open Group wrote: > > > The aim here was to describe the cut-off-point where all include > > > file generation has been completed and after which the new > > > contents of the files is used. This cut-off-point needs to be > > > before make starts the "real work", i.e. starts the work to bring > > > the first target operand, or the first target make encounters if > > > there are no operands, up-to-date. > > Can we just say that? All include file regeneration is complete > > before make attempts to bring the first target_name operand, or the > > first target make encounters if no target is specified? > > If we simplify it too much, it won't be correct. For example, > consider the case where the first target_name operand is the name of > an include file that needs to be created. I agree we can't over-simplify but I don't see a problem with the specific case you mention. For GNU make, at any rate, nothing special will be done if a target_name operand is also an include file. The include file will be rebuilt (if needed) as usual, then make will attempt to build the target_name operand and realize it's already up to date, and go on to the next one (or finish).
Re: [Issue 8 drafts 0001325]: Allow make to remake an included file
On Thu, 2020-11-05 at 09:25 +, Geoff Clare via austin-group-l at The Open Group wrote: > That doesn't make any difference, since .c and .o are both in the > specified suffixes, so that brings the default .c.o rule into play. Hm. Right, I forgot that clearing the suffixes doesn't also delete the rules. It's annoying that there's no POSIX way to delete the default rules from within the makefile. In my opinion the -r option is wrong. Whether or not the default rules should be used is a function of the _makefile_, not a function of the _user_ of the makefile (the person who runs make). If the makefile is written so that it doesn't want or need the default rules then the makefile should be able to delete them. It shouldn't be up to the invoker of make to do that. And if the makefile is written so that it _does_ need the default rules, it shouldn't be up to the user to delete them with -r. Of course that's a completely separate topic :).
Re: [Issue 8 drafts 0001325]: Allow make to remake an included file
On Wed, 2020-11-04 at 18:05 +0100, Joerg Schilling via austin-group-l at The Open Group wrote: > I am not sure whether I missed something, but I cannot see a method > that may work in a portable way and unless I missed something, I see > no new idea that makes things really better than what we are > currently discussing. In the paper I discuss the problems with the generated output and the advantages to avoiding it. > The problem I see is that the advanced method available with SunPro > Make does not seem to be supported by GNU make and the options > mentioned in the paper from above seem to be GCC specific and still > require a separate compiler call. They are technically not GCC-specific: clang also provides these options. But that support is not necessary: in the paper I go over a few alternatives to use if your compiler isn't capable of generating dependencies as a side-effect of compilation. Whatever process you use to generate the .d files is simply added to the compiler invocation rule. Yes, it means you may need to run the compiler twice but it still works and you have to run the compiler twice when using generated include files anyway. The built-in compiler support is simply an efficiency improvement, it's not a fundamental requirement.
Re: [Issue 8 drafts 0001325]: Allow make to remake an included file
On Wed, 2020-11-04 at 16:47 +, Geoff Clare via austin-group-l at The Open Group wrote: > Having read this, I'm now wondering why we are bothering to add > requirements for generating include files, if it is no longer the > recommended way of doing things. There are other uses for auto-generating included files: it does give you a bit of "meta-programming" capability and some flexibility to auto-generate rules that plain makefiles cannot. Implicit rules, and particularly POSIX suffix rules, are really not powerful enough to provide all the different types of templating you might want. However I don't know if this is sufficient justification for a change to POSIX. I wrote a post about this too: http://make.mad-scientist.net/constructed-include-files/ Looks like this post needs to be updated to note you don't need to use -include anymore as well :) > > This post obviously makes use of GNU make-specific features but I > > don't believe there's anything there that couldn't be also > > implemented in standard make. > > To improve efficiency it relies on the compiler having an option to > request creation of a dependency list at the same time as creating > the .o file. However, something similar is possible with standard c99 > by creating a .i file and then creating the .o and .d from that. Yes. But in any event, even if you do the simple thing and just run it twice you're no worse off since regenerated include files require running the compiler twice as well. > Here's a modified version of the proposed example that uses this new > technique. Note that you have to use "make -r" otherwise it uses the > default .c.o rule instead! The rules for a.o and b.o seem to be > needed by SunPro make but not by GNU make. > > .POSIX: > .SUFFIXES: .c .i .o To avoid using make -r you can clear the suffixes before defining new ones: .SUFFIXES: .SUFFIXES: .c .i .o
Re: [Issue 8 drafts 0001325]: Allow make to remake an included file
On Wed, 2020-11-04 at 10:48 +, Geoff Clare via austin-group-l at The Open Group wrote: > > - > > (0005094) psmith (developer) - 2020-11-03 15:23 > > https://austingroupbugs.net/view.php?id=1325#c5094 > > > > Regarding the change to page 2891 lines 97033-97035: > > It's there because GNU make uses re-execution. The re-executed make > doesn't know what targets the previous invocation updated, so it > doesn't know to treat them as already up-to-date. I understand. I need to stop thinking of the re-exec'd make and simply consider the entirety of the process. The real process GNU make undertakes is something like: 1. Read all makefiles. Don't error if they do not exist. 2. Consider all makefiles as goal targets and try to rebuild them. 3. If any rebuild, clear out all knowledge of all parsed makefiles and go to step 1 (GNU make does this via re-exec but that's not required of course). 4. If any required makefiles are missing, fail. 5. Build the normal goal targets. So, it's never the case that by the time you get to step #5, any makefile will be reconsidered: at that point all makefiles will be known to be up to date. > The aim here was to describe the cut-off-point where all include file > generation has been completed and after which the new contents of the > files is used. This cut-off-point needs to be before make starts the > "real work", i.e. starts the work to bring the first target operand, > or the first target make encounters if there are no operands, up-to- > date. Can we just say that? All include file regeneration is complete before make attempts to bring the first target_name operand, or the first target make encounters if no target is specified? > > In the portability text (second bullet), a portable workaround to > > using a macro defined in one included file in the include line of > > another, is to put the second include _inside_ the first included > > file instead of in the outer including makefile. > > Okay, that sounds like it is worth adding. Maybe something like: > > Include lines and rules for creating dynamic include files do not > depend on the contents of any earlier dynamic include file. For > example, defining a macro in a dynamic include file and using > that > macro in a later include line should be avoided (unless the later > include line is itself inside the dynamic include file). OK. > If adding "ddir" to the search path is achieved by means of an > extension (such as -I), then the requirement in the standard does not > apply to "ddir"; it only applies to the default search path. > > It is only the default search path that I see as a problem. If a > user explicitly wants a directory searched, then that implies they > know what's in that directory and will choose include file names with > that in mind. If there's a clash which causes their makefile to > behave differently than intended, that's the user's fault. Yes, exactly. > Sounds like we are in agreement with how GNU make should handle this > issue. It can prioritise specified search paths over remaking, but > not the default search path. I believe the currently proposed text > allows that, because specifying a search path involves using an > extension. OK, I'm OK with making this change to GNU make to meet the standard if needed. Thanks!
Re: [Issue 8 drafts 0001325]: Allow make to remake an included file
On Wed, 2020-11-04 at 12:01 +, Geoff Clare via austin-group-l at The Open Group wrote: > I haven't been able to come up with a common initial statement that > is preferable to just describing the two methods separately. So > unless someone else wants to give it a go, I suggest that I should > restructure that part along these lines: > > The make utility shall use one of the following two methods > to attempt to create the file or bring it up-to-date: > > 1. The "immediate remaking" method > > If make uses this method, any target rules or inference > rules for the pathname that were parsed before the include line > was parsed shall be used to attempt to create the file or to > bring it up-to-date before opening the file. > > 2. The "delayed remaking" method > > If make uses this method, no attempt shall be made to > create the file or bring it up-to-date until after the > makefile(s) have been read. During processing of the include > line, make shall read the current contents of the file, > if it exists, or treat it as an empty file if it does not exist. > Once the makefile(s) have been read, make shall use any > applicable target rule or inference rule for the pathname, > regardless of whether it is parsed before or after the include > line, when creating the file or bringing it up-to-date. This seems OK to me. I like the idea of using two separate sections; the process seems different enough that it's confusing to shoe-horn it into a single one.
Re: make(1) parallelization, but especially .WAITing
On Tue, 2020-11-03 at 14:08 +, shwaresyst wrote: > I agree that's the probable intent, but like other undefined things, > what isn't precluded is a spot where a conformance distinction can't > be drawn. There how the identifier ends isn't specified, it's left > implied implementors will only use after the prefix that is > specified. What I'm saying is that I don't agree that it's undefined. "Names consisting of" cannot be interpreted to mean the same thing as "names beginning with". I don't agree that there's any ambiguity here. Cheers!
Re: make(1) parallelization, but especially .WAITing
On Mon, 2020-11-02 at 15:44 +, shwaresyst via austin-group-l at The Open Group wrote: > With that phrasing is also reserved, since it > is not " followed ONLY by uppercase". Using ".NO_parallel" > would be similarly conforming, it could be argued. I don't agree. By saying "names consisting of" the standard requires that the entire name must consist of those characters, not just the first part of the name. > (The last sentence before the "Macros" heading says "Targets with > names consisting of a leading followed by one or more > uppercase letters are reserved for implementation extensions."
Re: Austin Group teleconference +1 888 974 9888 PIN 618 156 403
On Thu, 2020-10-29 at 12:54 -0400, Single UNIX Specification via austin-group-l at The Open Group wrote: > Start time:Today 11:00 AM Apologies that I won't be able to attend today's meeting and I won't be able to give Geoff's updated text for issue #1325 the attention it deserves before then. It's especially galling given the effort I know he put into it!! I will make a concerted effort to provide comments well in advance of Thursday's meeting.
Re: make(1) parallelization, but especially .WAITing
On Wed, 2020-10-28 at 13:57 +0100, Joerg Schilling via austin-group-l at The Open Group wrote: > Yes, the speudo target .WAIT has been introduced into SunPro Make > with Solaris 2 and it pemits to get fine grained control to parallel > execution > > If GNU make did implement it as well, this would be a real benefit > for controllability and portablity. I have no objection to having this feature implemented in GNU make; however, it is not a simple enhancement and so would require some significant effort. > Since .NOTPARALLEL seems to just overwrite the -j# command line > option from GNU make by -j1, this does not seem to be hard to > implement in SunPro Make. That's not really what it does. In fact, it doesn't change the command line options at all; that would not be good. Setting .NOTPARALLEL in the current instance of make doesn't impact any sub-makes that are invoked; they will still be run in parallel unless they also define .NOTPARALLEL in their makefile. Basically setting .NOTPARALLEL is checked in one place in the GNU make code: if (job_slots == 1 || not_parallel) /* Since there is only one job slot, make things run linearly. So it does not change the configuration of make, it only adds an extra check for .NOTPARALLEL alongside the normal check for number of jobs.
Re: make(1) parallelization, but especially .WAITing
On Tue, 2020-10-27 at 20:48 +0100, Steffen Nurpmeso via austin-group-l at The Open Group wrote: > Ah. No, this is conscious, the evaluation has to be done by the > shell. OK. In any event, prefixing the command line that invokes the sub-make with "+" should solve the problem.
Re: make(1) parallelization, but especially .WAITing
On Tue, 2020-10-27 at 20:48 +0100, Steffen Nurpmeso wrote: > |As for the idea of standardizing parallel builds, I'm all for it but it > |will be a long process to work through the different implementations to > |arrive at an acceptable common subset. > > No, sorry, that is a misunderstanding. Like i have written, .WAIT is > for now all i would like to see a word about. I see. Well, I'm not sure how feasible it is to add something like .WAIT to the POSIX standard, in the absence of any other discussion of parallel builds. Also as you're aware, GNU make does not currently support .WAIT anyway. Doing that right is not a trivial thing. On the other hand, supporting .NOTPARALLEL: is straightforward for any version of make, and could easily be added to the standard without much other discussion. Cheers!
Re: make(1) parallelization, but especially .WAITing
On Tue, 2020-10-27 at 19:12 +0100, Steffen Nurpmeso via austin-group-l
at The Open Group wrote:
> build:
> @$(_prestop); LC_ALL=C $${MAKE} -f mk-config.mk $${MAKEFLAGS}
> all
>
> to specify the configure-time flags to the actually working make
> instance, and that then says
>
> make[1]: warning: -j4 forced in submake: resetting jobserver mode.
I'm sorry but I was not able to understand the bulk of your email :(
However, for the above specifically I can solve your problem. I do
urge you to take these sorts of implementation detail questions to an
appropriate mailing list such as help-m...@gnu.org.
Using ${MAKE} in your recipe instead of $${MAKE} would solve your
problem of course.
If for some reason you are forced to use $${MAKE}, you can add the
POSIX-standard (and probably even required for proper behavior in your
situation but without more information we can't be sure) "+" prefix
character to the recipe:
build:
+@$(_prestop); LC_ALL=C $${MAKE} -f mk-config.mk $${MAKEFLAGS}
all
This will keep GNU make from resetting jobserver mode.
Also, I should warn you that using ${MAKEFLAGS} in the command line
like that is not supported behavior. According to POSIX MAKEFLAGS is
intended to be parsed as an _environment variable_ which is formatted
in an implementation-defined way allowing a parent make to communicate
to its sub-make. The formatting of MAKEFLAGS is specifically _NOT_
required to be valid command line option syntax.
It should not be required to add it to the command line anyway; it
should be sufficient to use:
build:
+@$(_prestop); LC_ALL=C $${MAKE} -f mk-config.mk all
As for the idea of standardizing parallel builds, I'm all for it but it
will be a long process to work through the different implementations to
arrive at an acceptable common subset.
Re: Pseudoterminal terminology in POSIX
On Wed, 2020-08-05 at 23:38 +0700, Robert Elz wrote: > | do you consider the pseudoterminal as providing to the terminal, or the > | terminal as providing to the pseudoterminal. > > How did anyone ever get to a question like that? In the part of my message you elided I was arguing that using the names "provider" and "consumer" doesn't avoid confusion. I guess you proved my point.
Re: Pseudoterminal terminology in POSIX
On Wed, 2020-08-05 at 08:00 -0700, Donn Terry via austin-group-l at The Open Group wrote: > The suggestions here so far are cumbersome and tend to be ambiguous. > The old m-word and sl-word, and also "client" and "server" could > potentially be interpreted backwards from the conventional intent. > (You can think about it as the sl-word/client actually being in > control: telling the m-word/server what it's supposed to be doing, > e.g. "execute this command line".) > > How about "provider" and "consumer"? "Pseudoterminal provider" and > "...consumer" seem (at least to me) to be unambiguous in terms of the > reversal above, (reasonably) clear in meaning, and politically > neutral. Have the other discussions not shown here considered this? To me even "provider" / "consumer" still has this issue: do you consider the pseudoterminal as providing to the terminal, or the terminal as providing to the pseudoterminal. Both seem legitimate enough interpretations to create confusion. To remove ambiguity perhaps we need to think about the attributes that are unique to each element of the pair and use that in the term, for example "backend" / "frontend". This would have to be introduced, something like "a pseudoterminal device pair consists of a backend terminal device and a frontend pseudoterminal device".
