Re: [bug #27714] expansion of $(shell) in target forces serialization of targets

[Philip Guenther]

On Fri, Oct 16, 2009 at 4:34 AM, Paul D. Smith  wrote:
...
> But before it can invoke the recipe it has to expand all the variables and 
> functions.

I had to ponder that requirement for a bit before it occurred to me
that doing otherwise would either make $(eval) pointless in recipes or
make it an incredible, royal pain to implement.  Ditto for $(error).


Philip Guenther


___
Bug-make mailing list
Bug-make@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-make

Re: Documentation error

[Paul Smith]

On Fri, 2009-10-16 at 10:01 +0100, mohamed hassanine aissa wrote:
> Dear Sir or Madam,
> I think i've found a little error in the documentation of GNU make:
> section  : 9.5 Overriding Variables
>  
> citation "An argument that contains `=' specifies the value of a
> variable: `v=x' sets the value of the variable v to x"
>  
> error: v=x sets the value of the variable x to v and not v to x.
> yours

Nope, I think the docs are right.  v=x sets the value of the variable
named "v" to the value "x", which is what the docs say.

In other words, FOO=bar sets the (value of the) variable FOO to the
string "bar".

-- 
---
 Paul D. Smith   Find some GNU make tips at:
 http://www.gnu.org  http://make.mad-scientist.net
 "Please remain calm...I may be mad, but I am a professional." --Mad Scientist


___
Bug-make mailing list
Bug-make@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-make

Documentation error

[mohamed hassanine aissa]

Dear Sir or Madam,

I think i've found a little error in the documentation of GNU make:

section  : 9.5 Overriding Variables

 

citation "An argument that contains `=' specifies the value of a variable: 
`v=x' sets the value of the variable v to x"

 

error: v=x sets the value of the variable x to v and not v to x.

yours

  
_
http://redirect.gimas.net/?n=M0910xHotmail2_WW
Neu, Besser, Hotmail - Die neuen Features sind da!___
Bug-make mailing list
Bug-make@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-make

[bug #27714] expansion of $(shell) in target forces serialization of targets

[Paul D. Smith]

Follow-up Comment #3, bug #27714 (project make):

Hi Mike; no, sorry, I should have given an explicit example.  Your new test
case is not correct and doesn't prove anything.  I explicitly said do NOT put
the sleep inside the $(shell).  It's important to understand that make itself
is single threaded (it doesn't use pthread() etc.).  Its parallelism is gained
by having the recipes it invokes be run in parallel.  But before it can invoke
the recipe it has to expand all the variables and functions.  That includes
$(shell ...) functions.  Variables and functions (including $(shell ...)
functions) are NOT expanded "in parallel"; they are expanded one at a time as
they are seen in the recipe.

After the entire recipe is expanded, THEN make invokes the command, then it
leaves the command running in the background while it locates the next
possible recipe to start, evaluates it, then runs it in the background, etc.

So, causing the recipe expansion to take a long time (which is what you're
doing by adding the sleep to the $(shell ...) function) actually _decreases_
the amount of parallelism.  Hopefully that's clear.

What you need to do to show the make is working correctly is increase the
time it takes the recipe itself to run: that is, add the sleep OUTSIDE the
$(shell ...).  Something like:


$(t): ; @printf $@ $(shell :); sleep 5; printf $@


You should see the recipes all run in parallel (the first printf's all run,
then a pause, then the second printfs all run).

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/



___
Bug-make mailing list
Bug-make@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-make

[bug #27714] expansion of $(shell) in target forces serialization of targets

[Mike Frysinger]

Follow-up Comment #2, bug #27714 (project make):

i didnt think it a race condition because the original test case had a fairly
lengthy $(shell) code and the targets always ran in the same order (and
obviously only one cpu at a time -- the other cpus stayed idle)

using sleep shows that it is indeed a make bug.  i see one target show up
every 10 seconds:
$ cat makefile
t = 0 a b c 1 d e f 2 g h i 3 j k l 4 m n o 5 p q r 6
all: $(t) ; @echo
$(t): ; @printf $@ $(shell sleep 10)

$ make -f makefile
<10 seconds>0<10 seconds>a<10 seconds>b

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/



___
Bug-make mailing list
Bug-make@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-make

[bug #27714] expansion of $(shell) in target forces serialization of targets

[Paul D. Smith]

Follow-up Comment #1, bug #27714 (project make):

My suspicion is that this is just an artifact of your test case.  In your
scenario, make must evaluate the $(shell ...) function (invoking the shell)
while it's expanding the command to be invoked.  What I think is happening
here is that your actual command is so fast that by the time make is ready to
invoke the next command (after it ran the $(shell :) for that target), the
previous command is already completed.  So, it looks like things are happening
serially even though make is doing its best to run them in parallel.

Please try this test again, but have the command you invoke take longer; for
example add a "sleep 1" or something after your printf (not inside the $(shell
...) function!), then print it again maybe.  You should see all the first
prints show up (running in parallel) then all the second prints.

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/



___
Bug-make mailing list
Bug-make@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-make

[bug #27714] expansion of $(shell) in target forces serialization of targets

[Mike Frysinger]

URL:
  

 Summary: expansion of $(shell) in target forces
serialization of targets
 Project: make
Submitted by: vapier
Submitted on: Fri 16 Oct 2009 07:50:48 AM GMT
Severity: 3 - Normal
  Item Group: Bug
  Status: None
 Privacy: Public
 Assigned to: None
 Open/Closed: Open
 Discussion Lock: Any
   Component Version: 3.81
Operating System: POSIX-Based
   Fixed Release: None
   Triage Status: None

___

Details:

i noticed a package recently that would build serially even when i gave it -j
(on a quad machine).  looking into it, the reason was that $(shell) was
appended to a compiler flag leading to it being expanded when the target was
run:
CFLAGS += $(shell .)

forcing the $(shell) to be evaluated immediately allowed things to build in
parallel:
SOME_CHECK := $(shell .)
CFLAGS += $(SOME_CHECK)

behavior is seen with make-3.80 and make-3.81

here is a simple example:
$ cat makefile
t = 0 a b c 1 d e f 2 g h i 3 j k l 4 m n o 5 p q r 6
all: $(t)
@echo
$(t):
@printf $@ $(shell :)

$ while make -f makefile -j ; do : ; done
0abc1de2ghi3jkl4mno5pqr6
0abc1de2ghi3jkl4mno5pqr6
0abc1de2ghi3jkl4mno5pqr6
0abc1de2ghi3jkl4mno5pqr6
0abc1de2ghi3jkl4mno5pqr6
0abc1de2ghi3jkl4mno5pqr6
0abc1de2ghi3jkl4mno5pqr6
0abc1de2ghi3jkl4mno5pqr6
0abc1de2ghi3jkl4mno5pqr6
0abc1de2ghi3jkl4mno5pqr6
0abc1de2ghi3jkl4mno5pqr6

removing the $(shell :) allows for parallel evaluation:
0abc1de2gi3hjkl4mnoqr56p
0abc1de2ghi3jkl4mn5pqor6
0acde21gbhi3jkl4mno5pqr6
0abc1de2ghi3jkl4mn5qr6po
0abc1d2gehi3jkl4o5mpnqr6
0abc1de2ghi3jkl4mno5pq6r
0abc1de2ghi3jkl4mno5pqr6
0abc1de2ghi3jl4n5pkqorm6
0abc1de2ghi3jkl4mno5pqr6
0abc1de2ghi3jkl4mnpqor56




___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/



___
Bug-make mailing list
Bug-make@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-make