Error in processing of :: targets
Gnumake maintainers, I've run into a functional difference between the standard unix make and GNUmake (3.80) involving how the :: rules are processed. In the standard unix make (Solaris), the :: targets ( i.e., all::) are process in the order they are found in the Makefile serially, regardless of whether parallelism is used (i.e., -jN, where N is 1). In GNUmake (3.80), the when -j1 is used, the "::" targets are executed serially. When -jN is used the "::" targets are executed in parallel. Consider the following -- from an IMAKE generated Makefile: all:: all.prologue all:: all.serial all.parallel all:: ; @echo $@; sleep 1; echo [EMAIL PROTECTED] all:: all.epilogue all.epilogue all.prologue:: ; @echo $@; sleep 2; echo [EMAIL PROTECTED] all.serial:: ; @echo [EMAIL PROTECTED]; sleep 1; echo [EMAIL PROTECTED] all.serial:: ; @echo [EMAIL PROTECTED]; sleep 1; echo [EMAIL PROTECTED] all.parallel: aaa bbb ccc aaa bbb ccc: ; @echo $@; sleep 1; echo [EMAIL PROTECTED] In a non-parallel make (i.e. -j1) in both Unix and Gnu make, the following is the result: all.prologue all.prologue-done all.serial-1 all.serial-1-done all.serial-2 all.serial-2-done aaa aaa-done bbb bbb-done ccc ccc-done all all-done all.epilogue all.epilogue-done When run in parallel using Unix make (i.e., -j12), the following occurs: all.prologue all.prologue-done all.serial-1 aaa bbb ccc all.serial-1-done aaa-done bbb-done ccc-done all.serial-2 all.serial-2-done all all-done all-epilogue all-epilogue-done When run in parallel using GNUmake (i.e., -j12), the following occurs: all.prologue all.serial-1 aaa bbb ccc all.epilogue all.prologue-done all.serial-1-done aaa-done all.serial-2 all.epilogue-done bbb-done ccc-done all.serial-2-done all all-done It appears that all the :: targets that exist in the file are started at the same time and not executed sequentially as is done for standard Unix make. The only exception to this is the all.serial target which has multiple command, which are executed in order... As a result of this I cannot port our IMAKE based make utility to use GNUmake. The intent of the above targets is as follows: + all.prologue -- things which should be performed before any other all* targets are made For example, creating, or removing a directory. + all.epilogue -- things which should occur after all* targets have completed -- for example sending email. + a.serial -- rules that MUST execute serially + a.parallel -- rules that may execute in parallel The following defines the order of execution: all: all.prologue all: all.serial all.parallel [ multiple all, all-serial, and all.parallel targets] all: all.epilogue I'm interested in your views on this problem. I think it's a bug because GNUmake is clearly out of sync with standard make on the order in which :: targets are evaluated in a parallel build environment. Reid Madsen Principal Engineer - Architecture Group Tektronix, Inc. 214-912-5152 ___ Bug-make mailing list Bug-make@gnu.org http://lists.gnu.org/mailman/listinfo/bug-make
Re: Error in processing of :: targets
[EMAIL PROTECTED] wrote: Gnumake maintainers, I've run into a functional difference between the standard unix make and GNUmake (3.80) involving how the :: rules are processed. In the standard unix make (Solaris), the :: targets ( i.e., all::) are process in the order they are found in the Makefile serially, regardless of whether parallelism is used (i.e., -jN, where N is 1). That's funny, my standard Unix (Solaris) make doesn't have a -j switch, and doesn't support parallelism. In fact, -j is a GNU-specific extension, and it Works As Designed. In GNUmake (3.80), the when -j1 is used, the :: targets are executed serially. When -jN is used the :: targets are executed in parallel. Consider the following -- from an IMAKE generated Makefile: This, like most of the dreck that Imake produces, is bogus. http://lists.gnu.org/archive/html/bug-make/2001-09/msg00013.html http://lists.gnu.org/archive/html/bug-make/2004-09/msg1.html all:: all.prologue all:: all.serial all.parallel all:: ; @echo $@; sleep 1; echo [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] all:: all.epilogue all.epilogue all.prologue:: ; @echo $@; sleep 2; echo [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] all.serial:: ; @echo [EMAIL PROTECTED] mailto:[EMAIL PROTECTED]; sleep 1; echo [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] all.serial:: ; @echo [EMAIL PROTECTED] mailto:[EMAIL PROTECTED]; sleep 1; echo [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] all.parallel: aaa bbb ccc aaa bbb ccc: ; @echo $@; sleep 1; echo [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] -- -- Howard Chu Chief Architect, Symas Corp. Director, Highland Sun http://www.symas.com http://highlandsun.com/hyc Symas: Premier OpenSource Development and Support ___ Bug-make mailing list Bug-make@gnu.org http://lists.gnu.org/mailman/listinfo/bug-make
RE: Error in processing of :: targets
Paul, I guess I should provide an example for my previous statement: all:: first @echo one all:: second @echo two first: @echo first second: @echo second In GNUmake 3.75, this produced: first one second two In GNUmake 3.80, this produced: first second one two On the surface, this might not appear to be a big deal -- unless the order of execution between first second is important. In the larger picture, we like to intermix serialism and parallelism in the same makefile. In 3.80, if you choose parallelism, serialism is impossible. However, in 3.75, this worked just fine. Why did the functionality change? Reid -Original Message- From: Paul Smith [mailto:[EMAIL PROTECTED] On Behalf Of Paul D. Smith Sent: Tuesday, June 14, 2005 7:48 AM To: Madsen, Reid Cc: bug-make@gnu.org Subject: Re: Error in processing of :: targets %% [EMAIL PROTECTED] writes: rm I've run into a functional difference between the standard unix rm make and GNUmake (3.80) involving how the :: rules are processed. By standard unix make you seem to mean SystemV make. Note that the only official Standard-with-a-capital-S (recognized by the FSF and GNU project) for make is the POSIX standard. The POSIX standard doesn't address this issue (it doesn't address parallelism at all). SystemV make is, at best, a de facto standard, and there are plenty of other (incompatible) versions of make out there. In fact, GNU make is not intended to be a drop-in replacement for SystemV make and there are _several_ points at which the behavior of the two versions diverge. rm In the standard unix make (Solaris), the :: targets ( i.e., all::) rm are process in the order they are found in the Makefile serially, rm regardless of whether parallelism is used (i.e., -jN, where N is rm 1). rm In GNUmake (3.80), the when -j1 is used, the :: targets are rm executed serially. When -jN is used the :: targets are executed rm in parallel. GNU make's behavior here is exactly in keeping with the definition of double-colon rules in the GNU make manual, which states: When a target appears in multiple rules, all the rules must be the same type: all ordinary, or all double-colon. If they are double-colon, each of them is independent of the others. Each double-colon rule's commands are executed if the target is older than any prerequisites of that rule. If there are no prerequisites for that rule, its commands are always executed (even if the target already exists). This can result in executing none, any, or all of the double-colon rules. Double-colon rules with the same target are in fact completely separate from one another. Each double-colon rule is processed individually, just as rules with different targets are processed. If you think a different set of behavior is desirable, the best thing is probably to file an enhancement request on the Savannah (https://savannah.gnu.org/) project page for GNU make and someone will look at it. Cheers! -- --- Paul D. Smith [EMAIL PROTECTED] Find some GNU make tips at: http://www.gnu.org http://make.paulandlesley.org Please remain calm...I may be mad, but I am a professional. --Mad Scientist ___ Bug-make mailing list Bug-make@gnu.org http://lists.gnu.org/mailman/listinfo/bug-make
RE: Error in processing of :: targets
Howard, I thought I was using Sun's make, but in fact I was using GNUmake V3.75. And in that version this works perfectly. Meaning that the commands for all.prologue are executed to completion before aaa, bbb, and ccc, are build, and the commands for all.epilogue are executed AFTER aaa, bbb, and ccc are built. So you think it is bogus. Then how would you implement this makefile (which worked fine in 3.75 GNUmake) to achieve the described ordering? Specifically the commands of all.prologue must execute BEFORE aaa, bbb, or ccc. all:: all.prologue all:: aaa bbb ccc all:: all.epilogue all.prologue:: ; @echo prologue all.epilogue:: ; @echo epilogue aaa bbb ccc: ; @echo $@ Sure, you could add: aaa: all.prologue bbb: all.prologue ccc: all.prologue and all.epilogue:: aaa bbb ccc But assuming that every rule writer -- in a complex build system -- will remember to do this is an unsafe assumption. And it makes no difference whether this Makefile was generated by IMAKE or by hand -- the problem is the same. In version 3.75, each :: rule and its dependencies were treated separately. Meaning that for each :: rule, the dependencies for that rule were evaulated, and then the commands for that rule were run. Dependencies for subequent instances of the same target were deferred until that :: rule was evaulated. If multiple dependencies were specified, then they were built in parallel when the :: rule they were mentioned in was evaluated. In version 3.80, the dependencies for all :: rules are evaluated in parallel before any commands for the :: target are executed. This is a significant change from V3.75. Bottom line, GNUmake V3.75 provided a very useful mechanism to establish ordering of :: rules in a parallel environment. Thus functionality has been lost in later versions. Reid Madsen -Original Message- From: Howard Chu [mailto:[EMAIL PROTECTED] Sent: Tuesday, June 14, 2005 8:21 AM To: Madsen, Reid Cc: bug-make@gnu.org Subject: Re: Error in processing of :: targets [EMAIL PROTECTED] wrote: Gnumake maintainers, I've run into a functional difference between the standard unix make and GNUmake (3.80) involving how the :: rules are processed. In the standard unix make (Solaris), the :: targets ( i.e., all::) are process in the order they are found in the Makefile serially, regardless of whether parallelism is used (i.e., -jN, where N is 1). That's funny, my standard Unix (Solaris) make doesn't have a -j switch, and doesn't support parallelism. In fact, -j is a GNU-specific extension, and it Works As Designed. In GNUmake (3.80), the when -j1 is used, the :: targets are executed serially. When -jN is used the :: targets are executed in parallel. Consider the following -- from an IMAKE generated Makefile: This, like most of the dreck that Imake produces, is bogus. http://lists.gnu.org/archive/html/bug-make/2001-09/msg00013.html http://lists.gnu.org/archive/html/bug-make/2004-09/msg1.html all:: all.prologue all:: all.serial all.parallel all:: ; @echo $@; sleep 1; echo [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] all:: all.epilogue all.epilogue all.prologue:: ; @echo $@; sleep 2; echo [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] all.serial:: ; @echo [EMAIL PROTECTED] mailto:[EMAIL PROTECTED]; sleep 1; echo [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] all.serial:: ; @echo [EMAIL PROTECTED] mailto:[EMAIL PROTECTED]; sleep 1; echo [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] all.parallel: aaa bbb ccc aaa bbb ccc: ; @echo $@; sleep 1; echo [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] -- -- Howard Chu Chief Architect, Symas Corp. Director, Highland Sun http://www.symas.com http://highlandsun.com/hyc Symas: Premier OpenSource Development and Support ___ Bug-make mailing list Bug-make@gnu.org http://lists.gnu.org/mailman/listinfo/bug-make
RE: Error in processing of :: targets
%% [EMAIL PROTECTED] writes: rm all:: first rm @echo one rm all:: second rm @echo two rm first: rm @echo first rm second: rm @echo second rm In GNUmake 3.75, this produced: rm first rm one rm second rm two rm In GNUmake 3.80, this produced: rm first rm second rm one rm two I was looking through the NEWS file for GNU make and found this: Version 3.63 [...] * Multiple double-colon rules for the same target will no longer have their commands run simultaneously under -j, as this could result in the two commands trying to change the file at the same time and interfering with one another. So, it looks like you're right: double-colon rules should be invoked serially. I guess the docs weren't changed to make this clear when the code was updated. Obviously something changed between 3.75 and now to break that: I definitely didn't change it on purpose. Please file a bug at Savannah against GNU make and I'll look into it at some point. -- --- Paul D. Smith [EMAIL PROTECTED] Find some GNU make tips at: http://www.gnu.org http://make.paulandlesley.org Please remain calm...I may be mad, but I am a professional. --Mad Scientist ___ Bug-make mailing list Bug-make@gnu.org http://lists.gnu.org/mailman/listinfo/bug-make
