Re: parallelism: job slots vs. levels
%% Dan Jacobson <[EMAIL PROTECTED]> writes: dj> Anyways, dj> $ make x& make y& wait dj> cannot always be rewritten with -j. Mm. I don't think I believe that. For example, of your makefile had at the top: .PHONY: all rx ry all: rx ry rx: ; $(MAKE) x ry: ; $(MAKE) y then I think "make -j all" gives you the same behavior. dj> $ make -j[whatever number] x y dj> will act differently except for special cases of x and y; dj> probably when both x and y have no dependencies. I doubt that as well. I've already described this in my last message: these two constructs are identical in function _UNLESS_ their prerequite subgraphs overlap somewhere. If they overlap, then the result is not deterministic in general: it may work fine, or it may fail miserably. -- --- Paul D. Smith <[EMAIL PROTECTED]> Find some GNU make tips at: http://www.gnu.org http://make.paulandlesley.org "Please remain calm...I may be mad, but I am a professional." --Mad Scientist ___ Bug-make mailing list [EMAIL PROTECTED] http://lists.gnu.org/mailman/listinfo/bug-make
Re: parallelism: job slots vs. levels
Dan Jacobson wrote: Anyways, $ make x& make y& wait cannot always be rewritten with -j. $ make -j[whatever number] x y will act differently except for special cases of x and y; probably when both x and y have no dependencies. make x& make y& wait will only work correctly if x and y have no dependencies in common. Otherwise, the two separate make's will get in each other's way when they work on whatever is in common. This fact has nothing to do with parallel make or "make -j". Anyways, with -j examples added to the manual, we would get on the right track about how to use -j. Since your two examples have nothing to do with each other, I don't see how you can reach this conclusion. -- -- Howard Chu Chief Architect, Symas Corp. Director, Highland Sun http://www.symas.com http://highlandsun.com/hyc Symas: Premier OpenSource Development and Support ___ Bug-make mailing list [EMAIL PROTECTED] http://lists.gnu.org/mailman/listinfo/bug-make
Re: parallelism: job slots vs. levels
Dan Jacobson wrote: Anyways, $ make x& make y& wait cannot always be rewritten with -j. $ make -j[whatever number] x y will act differently except for special cases of x and y; probably when both x and y have no dependencies. Anyways, with -j examples added to the manual, we would get on the right track about how to use -j. I'm not quite sure what the point of all this is. I guess the texinfo docs for parallel execution may be somewhat lacking. Here's a summary of Things You Must Know. In a typical Makefile with target and dependencies: foo: bar baz cmds the Make syntax specifies that all of the dependencies are of equal precedence. That means, bar and baz can be made in any order and the result must still be valid. If there is really a dependency such that bar must be created before baz, then the above rule is incorrect. (Or at least, insufficient.) It should be written instead as baz: bar cmds foo: baz cmds In serial make you can get away with the incorrect syntax, because make only fires off one command at a time and it typically runs dependencies in left-to-right order. But that's just an idiosyncracy of the particular implementation. You could run across a "make" implementation that always processes dependencies right to left. If your Makefile follows correct syntax, it will still work. If it depends on left-to-right serial execution, it breaks, as well it should. In parallel make, all the dependencies of a target can be processed simultaneously. As such, it is imperative that you *explicitly* define all dependency relationships in your Makefile. You can't rely on the *implicit* relationship that says that left-most dependencies will be built before right-most. If in your example of "make x& make y& wait" "x" and "y" are completely indepedent, then the result will be correct, as would "make -j x y". But if you're relying on some implicit behavior to complete "x" before starting "y" then your Makefile is broken. The point is that in a correctly written Makefile with all dependency relationships explicitly spelled out, there will be no difference whether make is run serially or in parallel. If there is any difference, then the Makefile rules didn't correctly capture all of the real dependencies of the project. -- -- Howard Chu Chief Architect, Symas Corp. Director, Highland Sun http://www.symas.com http://highlandsun.com/hyc Symas: Premier OpenSource Development and Support ___ Bug-make mailing list [EMAIL PROTECTED] http://lists.gnu.org/mailman/listinfo/bug-make
parallelism: job slots vs. levels
Anyways, $ make x& make y& wait cannot always be rewritten with -j. $ make -j[whatever number] x y will act differently except for special cases of x and y; probably when both x and y have no dependencies. Anyways, with -j examples added to the manual, we would get on the right track about how to use -j. ___ Bug-make mailing list [EMAIL PROTECTED] http://lists.gnu.org/mailman/listinfo/bug-make
Re: Make bug related to backquotes
%% Pierre B <[EMAIL PROTECTED]> writes:
pb> The problem is with the backquote AND/OR addprefix.
pb> The following makefile illustrates the bug.
I think you're misunderstanding something basic here.
Make does not process backquotes. The shell processes backquotes. Make
only processes functions, such as the $(shell ...) function.
So, the result of this line:
buggy_string:=`ls *.tst`#
is to create a variable named "buggy_string" with the literal contents
"`ls *.tst`", _NOT_ the expansion of the backquote.
Then later we see:
pb> @ echo xbeginx $(addprefix ./,${buggy_string}) xendx mucked up
What does this do? The addprefix function is invoked before the script
is passed to the shell, so the actual shell command that's being run
here is:
echo xbeginx ./`ls ./*.tst` xendx mucked up
Now you can see why you get the results you do.
It would be enlightening, perhaps, for you to invoke make with the -n
option so it prints the commands it would send to the shell without
actually sending them there. Then you can see what is being expanded by
make and what isn't.
Also, you could remove the "@" prefix characters until you know it's
working, to see exactly what's going on.
--
---
Paul D. Smith <[EMAIL PROTECTED]> Find some GNU make tips at:
http://www.gnu.org http://make.paulandlesley.org
"Please remain calm...I may be mad, but I am a professional." --Mad Scientist
___
Bug-make mailing list
[EMAIL PROTECTED]
http://lists.gnu.org/mailman/listinfo/bug-make
Make bug related to backquotes
The problem is with the backquote AND/OR addprefix.
The following makefile illustrates the bug.
Notice the doubled ././ in the test output.
Version information is in the output.
#
# Make file illustrates a problem with back quote
# by juxtaposing results with back quote and shell
#
#
nice_string:=$(shell ls *.tst)#
buggy_string:=`ls *.tst`#
cat_buggy:=foo ${buggy_string}#
cat_nice:=foo ${nice_string}#
test:
-rm *.tst
@ echo
echo test1 >test1.tst
echo test2 >test2.tst
echo test3 >test3.tst
@ echo
@ echo xbeginx ${buggy_string} xendx string looks fine
@ echo xbeginx ${nice_string} xendx string looks fine
@ echo
@ echo xbeginx $(addprefix ./,${buggy_string}) xendx mucked up
@ echo xbeginx $(addprefix ./,${nice_string}) xendx NOT mucked
@ echo
@ echo xbeginx $(addprefix ./,${cat_buggy}) xendx mucked up
@ echo xbeginx $(addprefix ./,${cat_nice}) xendx NOT mucked up
@ echo
@ date
@ echo
@ make --version
@ echo
@ uname -a
@ echo
@ sh --version
@ echo
#
# End Makefile
#
# Results begin below
#
rm *.tst
echo test1 >test1.tst
echo test2 >test2.tst
echo test3 >test3.tst
xbeginx test1.tst test2.tst test3.tst xendx string looks fine
xbeginx test1.tst test2.tst test3.tst xendx string looks fine
xbeginx ././test1.tst ./test2.tst ./test3.tst xendx mucked it up
xbeginx ./test1.tst ./test2.tst ./test3.tst xendx NOT mucked up
xbeginx ./foo ././test1.tst ./test2.tst ./test3.tst xendx mucked up
xbeginx ./foo ./test1.tst ./test2.tst ./test3.tst xendx NOT mucked up
Tue Aug 31 15:50:37 EDT 2004
GNU Make 3.80
Copyright (C) 2002 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Linux alfalfa 2.6.3-1-386 #2 Tue Feb 24 20:20:23 EST 2004 i686 GNU/Linux
GNU bash, version 2.05b.0(1)-release (i386-pc-linux-gnu)
Copyright (C) 2002 Free Software Foundation, Inc.
##
# end test results
##
___
Bug-make mailing list
[EMAIL PROTECTED]
http://lists.gnu.org/mailman/listinfo/bug-make
Re: fatal errors for missing include files.
Paul D. Smith <[EMAIL PROTECTED]> writes: > If it is a bug then what you say is true, but I have never termed it a > bug. It was a design decision taken between two alternative > implementations, and the code is operating the way it was designed and > intended to work. I don't believe you ever intended for out-of-date makefile to trigger failure when it could be rebuilt and succeed. It is like if a C compiler would still report errors that you've just fixed refusing to compile your code. > The design is perhaps not the one you would choose or would prefer, but > that does not make it a bug. This way we can call any bug in concept a design feature. > bk> What would be nice is to have another flavor of include that would > bk> be rebuilt sequentially without re-executing make. I.e., every > bk> time make encounters such include directive it tries to rebuild > bk> the makefile and then reads it in. > > This is also quite confusing. At no time during the current make > processing does it jump out from the middle of the parser phase and into > the rule execution phase. I never said it would be easy ;-). > Not only that but the behavior would be quite > surprising to people who are familiar with make, since no rules or > variables that were defined after the include directive could be used > during the rebuild of the included makefile. I think it is no more surprising that to find make code failing because included makefile that can be built is missing. Also I am not proposing changing current schema - I think it can be quite useful sometimes. I am suggesting addition of another include directive flavor that would be well documented along with all implications (like the ones you just mentioned). But I guess it will stay just a proposal like many others before it... -boris signature.asc Description: Digital signature ___ Bug-make mailing list [EMAIL PROTECTED] http://lists.gnu.org/mailman/listinfo/bug-make
Re: fatal errors for missing include files.
%% Boris Kolpackov <[EMAIL PROTECTED]> writes: bk> "Paul D. Smith" <[EMAIL PROTECTED]> writes: id> Ok, but I am surprised you don't you see any problem (bug) with id> the current way? >> Not really. The times where it makes any difference are very few, bk> I don't think the presence of the bug is in any way affected by bk> the number of situations it is manifested in. Also you cannot for bk> sure say how many such situations there are since not everybody is bk> using make the way you do. For what it's worth I am suffering from bk> this bug too. If it is a bug then what you say is true, but I have never termed it a bug. It was a design decision taken between two alternative implementations, and the code is operating the way it was designed and intended to work. The design is perhaps not the one you would choose or would prefer, but that does not make it a bug. >> the performance penalty that would be incurred to re-invoke make >> after each makefile was recreated could be quite large. bk> What would be nice is to have another flavor of include that would bk> be rebuilt sequentially without re-executing make. I.e., every bk> time make encounters such include directive it tries to rebuild bk> the makefile and then reads it in. This is also quite confusing. At no time during the current make processing does it jump out from the middle of the parser phase and into the rule execution phase. Not only that but the behavior would be quite surprising to people who are familiar with make, since no rules or variables that were defined after the include directive could be used during the rebuild of the included makefile. -- --- Paul D. Smith <[EMAIL PROTECTED]> Find some GNU make tips at: http://www.gnu.org http://make.paulandlesley.org "Please remain calm...I may be mad, but I am a professional." --Mad Scientist ___ Bug-make mailing list [EMAIL PROTECTED] http://lists.gnu.org/mailman/listinfo/bug-make
Re: fatal errors for missing include files.
"Paul D. Smith" <[EMAIL PROTECTED]> writes: > id> Ok, but I am surprised you don't you see any problem (bug) with > id> the current way? > > Not really. The times where it makes any difference are very few, I don't think the presence of the bug is in any way affected by the number of situations it is manifested in. Also you cannot for sure say how many such situations there are since not everybody is using make the way you do. For what it's worth I am suffering from this bug too. > the performance penalty that would be incurred to re-invoke make after > each makefile was recreated could be quite large. What would be nice is to have another flavor of include that would be rebuilt sequentially without re-executing make. I.e., every time make encounters such include directive it tries to rebuild the makefile and then reads it in. This way has a number of benefits: - bug mentioned by the original poster goes away - more deterministic: if some of the code after (missing) included makefile depends on some code from it (for example a function) then things will break. - more efficient since there is no make re-execution (hurts really badly in non-recursive makefiles where cost of re-parsing dependency graph can be quite significant). What do you think? -boris ___ Bug-make mailing list [EMAIL PROTECTED] http://lists.gnu.org/mailman/listinfo/bug-make
Re: fatal errors for missing include files.
%% "Ian Dunbar" <[EMAIL PROTECTED]> writes: id> Ok, but I am surprised you don't you see any problem (bug) with id> the current way? Not really. The times where it makes any difference are very few, and the performance penalty that would be incurred to re-invoke make after each makefile was recreated could be quite large. id> I don't really understand why does adding the order id> only-prerequsite like this solve the problem, if make truly does id> not re-read the makefiles between remaking them? Heh. You don't believe me? :). You can try adding $(warning ...) lines to your various makefiles and seeing when they get printed, if you like. Also, you can use make's debugging mode (-d) to see more info. id> subsub.mk: | sub.mk id> include subsub.mk This has nothing to do with the order-only prerequisite; if you removed it you'd see the same behavior. This is because you've declared the target subsub.mk as an empty target; once you do that it's no longer unknown to make. See the GNU make manual section on empty target files. -- --- Paul D. Smith <[EMAIL PROTECTED]> Find some GNU make tips at: http://www.gnu.org http://make.paulandlesley.org "Please remain calm...I may be mad, but I am a professional." --Mad Scientist ___ Bug-make mailing list [EMAIL PROTECTED] http://lists.gnu.org/mailman/listinfo/bug-make
Re: fatal errors for missing include files.
Hi Paul, Thanks for the clarification. I have still some questions though. From: "Paul D. Smith" <[EMAIL PROTECTED]> id> It would know that the order of making up to date has to be: id> Makefile, sub.mk, subsub.mk, subsubsub.mk, etc., because if sub.mk id> is out of date, there is no guarentee that the soon to be updated id> version will include the same files as before. Ah. There is an even deeper misunderstanding here. Note that when remaking makefiles, make does _NOT_ proceed like this: Ok, but I am surprised you don't you see any problem (bug) with the current way? This doesn't matter though: remaking sub.mk in no way changes the rules, variables, etc. that make knows about because the makefile is not re-read immediately after it's created. So, if there were no rules to build that target before sub.mk was rebuilt, there cannot be any afterwards either. I don't really understand why does adding the order only-prerequsite like this solve the problem, if make truly does not re-read the makefiles between remaking them? subsub.mk: | sub.mk include subsub.mk From my understanding, all that says is that you should build sub.mk before subsub.mk. By your rule of rebuilding everything first, make should still attempt and fail (fatally because we are past the no-fatal-errors stage mentioned previously) to make subsub.mk. However, it gives the same warning for "subsub.mk", but this time it's non-fatal. Original way: sub.mk:1: subsub.mk: No such file or directory make: *** No rule to make target `subsub.mk'. Stop. With order-only: sub.mk:2: subsub.mk: No such file or directory cp sub.prj sub.mk make: Nothing to be done for `default'. Why is this? In both ways, sub.mk is out of date, and subsub.mk can't be made? Best regards, Ian _ Add photos to your messages with MSN 8. Get 2 months FREE*. http://join.msn.com/?page=features/featuredemail ___ Bug-make mailing list [EMAIL PROTECTED] http://lists.gnu.org/mailman/listinfo/bug-make
