Re: Checking software build tries for “commands.cmo”

[SF Markus Elfring]

> (I would guess that most of the people on the bug-make mailing list have 
> never worked with ocaml and therefore don't know what needs to be done to 
> build it.)

Yesterday I realised that one source source file needed a special handling
in the mentioned subdirectory.


> The snippet of Makefile you provided didn't include any rules for
> building "commands.ml", so I don't understand why you expect make to build it.

The suffix “ml” denotes a source file in this use case.

I needed another moment to become really aware that this software module
is a bit special. It seems that it is intended that it will be compiled
without a corresponding interface description file (suffix “mli”).

Regards,
Markus

___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

RE: Checking software build tries for “commands.cmo”

[Martin Dorey]

> I have noticed a moment ago that an interface description file was missing
> somehow for the OCaml source file in this compilation attempt.
...
> I wonder then that the make tool did not give me a direct clue for a failed
> software dependency as I am used to in other cases.

Would you have it complain that commands.mli,v, RCS/commands.mli,v, 
RCS/commands.mli, s.commands.mli and SCCS/s.commands.mli were missing?  The 
existence of any of those would have let make build commands.mli, hence 
commands.cmo.  You can have it do that, with make --debug=all.  You'll see it 
tries harder than you'd imagine, even searching for a rule that could update 
your makefile to fix the problem.  Can you come up with a specification for 
finding which of the possible missing dependencies is the one that you actually 
care about?
___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

Re: Checking software build tries for “commands.cmo”

[Philip Guenther]

On Tue, 13 Jun 2017, SF Markus Elfring wrote:
> I am testing the following pattern rules with the program “GNU Make 4.2.1-1.7”
> on my openSUSE Tumbleweed system as I would like to adjust some areas
> in affected make scripts for another evolving software tool.
> 
> …
> %.cmi: %.mli
>   $(OCAMLC_CMD) -c $<
> 
> %.cmo: %.ml %.cmi
>   $(OCAMLC_CMD) -c $<
> 
> %.cmx: %.ml %.cmi
>   $(OCAMLOPT_CMD) -c $<
> …
> 
> I have tried a specific command out.
> 
> elfring@Sonne:~/Projekte/Coccinelle/20160205/commons> LANG=C make -d 
> common.cmo V=1 
...
> This test showed the expected results.

Two questions:
1) What files were present in the directory before you ran that command?
2) What _was_ that expected result?

(I would guess that most of the people on the bug-make mailing list have 
never worked with ocaml and therefore don't know what needs to be done to 
build it.)


> I have tried another command variant out.
> 
> elfring@Sonne:~/Projekte/Coccinelle/20160205/commons> LANG=C make -d 
> commands.cmo V=1
...
> Now I wonder why I do not get the desired software generation results for
> this test case. Why is the source file “commands.ml” not compiled again
> in the way as the other one?

The snippet of Makefile you provided didn't include any rules for building 
"commands.ml", so I don't understand why you expect make to build it.


Philip Guenther

___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

Re: Are prerequisites made in deterministic order when parallelism is disabled?

[Paul Smith]

On Tue, 2017-06-13 at 10:33 -0500, Brett Stahlman wrote:
> I don't see anything in the Make docs that guarantees prerequisites
> will be processed in left to right order. Opinions on the web seems to
> be split into 2 camps:
> 
> 1. Make always builds dependencies in left to right order, but a
> well-designed Makefile won't rely upon it.
> 2. Make is free to build dependencies in any order it likes, provided
> it respects the stated dependencies.

The answer is #1, with a caveat: all the prerequisites in the rule
defining the recipe are always checked first (in the order they appear
in that rule), regardless of the order in which the recipe rule
appears.  So, in both of these situations:

  foo: biz ; @echo $<
  foo: bar
  foo: boz

and also:

  foo: bar
  foo: boz
  foo: biz ; @echo $<

the order of dependency checking will always be "biz bar boz" (and $<
will be "biz" of course).

But, in this situation:

  foo: boz
  foo: biz blah bleah; @echo $<
  foo: bar

then the order will be "biz blah bleah boz bar".  The order is always
the same and deterministic in a non-parallel build.

When parallelism is introduced, GNU make still follows the same order
of checking prerequisites.  However, because some prerequisites will be
blocked waiting for others, the algorithm will skip those and move on
to other parts of the DAG and the actual order in which recipes are
invoked is not deterministic (unless the time taken for each build is
completely deterministic).

___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

Re: Are prerequisites made in deterministic order when parallelism is disabled?

[Henrik Carlqvist]

On Wed, 14 Jun 2017 11:25:35 -0400
Kyle Rose  wrote:

> The right answer is always to write your makefiles so the rules execute
> in the required order whether run in parallel or not. Relying on
> whichever arbitrary ordering GNU make employs, even if that behavior is
> stable(either historically or by design), is likely to result in sadness
> at some point, such as when someone calls into your makefile recursively
> from their own -j build.

Sometimes I write Makefiles considering the performance at parallel builds
and those Makefiles get their prerequisites ordered by approximately how
much time each prerequisite needs to be built. Let me give you an example
of such a rule:

final_target: 3_hour_prerequisite 2_hour_prerequisite 1_hour_prerequisite
do_something_quickly

With the above example calling make without -j will take about 6 hours for
all prerequisites to build. On a machine with 3 or more cores calling make
with "-j 3" all prerequisites will be finished in 3 hours when also the
most time consuming one is done.

But what if we have a build machine with only 2 CPU cores? If so, calling
make with "-j 2" will be the fastest way to compile, it will still be done
in 3 hours. First the 3 hour job and the 2 hour job will be started in
parallell, then after 2 hours one job will be finished and the remaining 1
hour job will be started. After a total of 3 hours the 3 hour job and 1
hour job will finish about at the same time.

What would happen with "-j 2" if GNU Make changed the order of how
prerequisites where compiled? Then it might start with the 1 hour job and
the 2 hour job. Not until the 1 hour job where finished the 3 hour job
would start and the total build time would be 4 hours instead.

Even though the compile would complete successfully I think that not being
able to depend on job start order would give a significant lack of
performance.

I can also come to think of examples when non parallel builds would
benefit from a deterministic build order of prerequisites even though none
of the prerequisites depend on each other.

regards Henrik

___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

Re: Checking software build tries for “commands.cmo”

[SF Markus Elfring]

> elfring@Sonne:~/Projekte/Coccinelle/20160205/commons> LANG=C make -d 
> commands.cmo V=1
> …
>  No implicit rule found for 'commands.cmo'.
>  Finished prerequisites of target file 'commands.cmo'.
> Must remake target 'commands.cmo'.
> Successfully remade target file 'commands.cmo'.
> make: Nothing to be done for 'commands.cmo'.
> 
> 
> Now I wonder why I do not get the desired software generation results for
> this test case. Why is the source file “commands.ml” not compiled again
> in the way as the other one?

I have noticed a moment ago that an interface description file was missing
somehow for the OCaml source file in this compilation attempt.

I have added the following extra rules as a special handling.


commands.cmo: commands.ml
$(OCAMLC_CMD) -c $<

commands.cmx: commands.ml
$(OCAMLOPT_CMD) -c $<


I wonder then that the make tool did not give me a direct clue for a failed
software dependency as I am used to in other cases.

Regards,
Markus

___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

Re: Are prerequisites made in deterministic order when parallelism is disabled?

[Kyle Rose]

The right answer is always to write your makefiles so the rules execute in
the required order whether run in parallel or not. Relying on whichever
arbitrary ordering GNU make employs, even if that behavior is stable
(either historically or by design), is likely to result in sadness at some
point, such as when someone calls into your makefile recursively from their
own -j build.

Tl;dr: underspecified dependencies create hidden technical debt.

Kyle

On Tue, Jun 13, 2017 at 11:33 AM, Brett Stahlman 
wrote:

> I don't see anything in the Make docs that guarantees prerequisites
> will be processed in left to right order. Opinions on the web seems to
> be split into 2 camps:
>
> 1. Make always builds dependencies in left to right order, but a
> well-designed Makefile won't rely upon it.
> 2. Make is free to build dependencies in any order it likes, provided
> it respects the stated dependencies.
>
> My own recent experience suggests #2 is the correct statement, but I
> can't rule out the possibility that a bug in my Makefile is producing
> the apparent non-determinism I'm observing. At any rate, can anyone
> point me to a definitive source on this?
>
> Thanks,
> Brett S.
>
> ___
> Bug-make mailing list
> Bug-make@gnu.org
> https://lists.gnu.org/mailman/listinfo/bug-make
>
___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

Re: Are prerequisites made in deterministic order when parallelism is disabled?

[David Boyce]

I've been waiting for Paul to show up with the definitive word but he may
be away. I'm 95% sure he's spoken on this before with the gist of it being
"While not required by standard, GNU make has always worked left-to-right
and this will not change".

While I *think* I've correctly channeled his answer, the real thing could
probably be found in the archives of the help-make or bug-make mailing
lists if you want to search there.

David

On Wed, Jun 14, 2017 at 7:36 AM, Brett Stahlman 
wrote:

> On Wed, Jun 14, 2017 at 2:00 AM, Edward Welbourne
>  wrote:
> > Brett Stahlman (13 June 2017 17:33)
> >> I don't see anything in the Make docs that guarantees prerequisites
> >> will be processed in left to right order. Opinions on the web seems to
> >> be split into 2 camps:
> >>
> >> 1. Make always builds dependencies in left to right order, but a
> >>well-designed Makefile won't rely upon it.
> >> 2. Make is free to build dependencies in any order it likes, provided
> >>it respects the stated dependencies.
> >>
> >> My own recent experience suggests #2 is the correct statement, but I
> >> can't rule out the possibility that a bug in my Makefile is producing
> >> the apparent non-determinism I'm observing. At any rate, can anyone
> >> point me to a definitive source on this?
> >
> > I suspect most implementations of make do in fact build in left-to-right
> > order but none guarantee it; and I won't be surprised if GNU make used
> > to but has lately stopped doing so, although I can't give you a
> > definitive source either way.  I certainly wouldn't ever assume
> > deterministic build order; if one prerequisite needs to be built before
> > another, the make-file should express that via a dependency (perhaps
> > just an order-only one).
>
> Makes sense. I saw it suggested somewhere that Make has to perform a
> complex, inherently unstable, topological sort, which may make it
> difficult to preserve left to right order. By experimentation, I
> discovered that the order seemed to be affected by the actual target
> names: e.g., names such as "foo" and "foo2" produced left to right
> order, whereas "foo2" did not. For the vast majority of target names
> tried, the order was strictly left to right.
>
> Thanks,
> Brett S.
>
> >
> > Eddy.
>
> ___
> Bug-make mailing list
> Bug-make@gnu.org
> https://lists.gnu.org/mailman/listinfo/bug-make
>
___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

[bug #51237] Deadlock in Ctrl-C handler on Windows

[Eli Zaretskii]

Follow-up Comment #1, bug #51237 (project make):

Thanks for the analysis and the patch.
Your patch is too large to accept without legal paperwork.  Would you be
willing to assign copyright for your changes to the FSF, so we could accept
your contribution?  If so, I can send you the legal form and instructions to
fill and submit it.

Also, I wonder if your implementation is the best one (or maybe I'm missing
something).  You've inserted a call to check_need_sleep into the main loop of
reap_children.  But if the 1st arg of reap_children is non-zero, Make will
block inside process_wait_for_any, and AFAIU signaling the event you added
will not interrupt that wait.  Wouldn't it be better to instead add the event
handle to the handles on which process_wait_for_any waits?  Then the reaction
to an interrupt should be much faster and more reliable, I think.  Or am I
missing something?


___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/


___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

Re: Are prerequisites made in deterministic order when parallelism is disabled?

[Brett Stahlman]

On Wed, Jun 14, 2017 at 2:00 AM, Edward Welbourne
 wrote:
> Brett Stahlman (13 June 2017 17:33)
>> I don't see anything in the Make docs that guarantees prerequisites
>> will be processed in left to right order. Opinions on the web seems to
>> be split into 2 camps:
>>
>> 1. Make always builds dependencies in left to right order, but a
>>well-designed Makefile won't rely upon it.
>> 2. Make is free to build dependencies in any order it likes, provided
>>it respects the stated dependencies.
>>
>> My own recent experience suggests #2 is the correct statement, but I
>> can't rule out the possibility that a bug in my Makefile is producing
>> the apparent non-determinism I'm observing. At any rate, can anyone
>> point me to a definitive source on this?
>
> I suspect most implementations of make do in fact build in left-to-right
> order but none guarantee it; and I won't be surprised if GNU make used
> to but has lately stopped doing so, although I can't give you a
> definitive source either way.  I certainly wouldn't ever assume
> deterministic build order; if one prerequisite needs to be built before
> another, the make-file should express that via a dependency (perhaps
> just an order-only one).

Makes sense. I saw it suggested somewhere that Make has to perform a
complex, inherently unstable, topological sort, which may make it
difficult to preserve left to right order. By experimentation, I
discovered that the order seemed to be affected by the actual target
names: e.g., names such as "foo" and "foo2" produced left to right
order, whereas "foo2" did not. For the vast majority of target names
tried, the order was strictly left to right.

Thanks,
Brett S.

>
> Eddy.

___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

[bug #51237] Deadlock in Ctrl-C handler on Windows

[Michael Builov]

URL:
  

 Summary: Deadlock in Ctrl-C handler on Windows
 Project: make
Submitted by: mbuilov
Submitted on: Wed 14 Jun 2017 12:43:28 PM UTC
Severity: 3 - Normal
  Item Group: Bug
  Status: None
 Privacy: Public
 Assigned to: None
 Open/Closed: Open
 Discussion Lock: Any
   Component Version: 4.2.1
Operating System: MS Windows
   Fixed Release: None
   Triage Status: None

___

Details:

Hello.

There is a bug in processing of Ctrl+C event under WINDOWS.

Gnu make handles Ctrl+C in fatal_error_signal() function, which suspends Main
thread.

But, if Main thread get suspended while it locks resources, where is a
possibility for dead-lock.

It's easy to reproduce dead-lock with this makefile:

# run make -j -O, then press Ctrl+C
all: a b c d
CMD := cmd.exe /c "echo off & for /l %a in (0,1,1000) do echo %a"
a:;$(CMD)
b:;$(CMD)
c:;$(CMD)
d:;$(CMD)
.PHONY: all a b c d

(backtrace1 attached)


Even without job-server and output synchronization, it is possible to
dead-lock make or put it to infinite loop:

# just run make without options, then press Ctrl+C
# (hard to reproduce dead-lock)
GOALS := a b c d e f g h i j k l m n o p q r s t u v w x y z
GOALS += $(GOALS:=1)
GOALS += $(GOALS:=2)
GOALS += $(GOALS:=3)
all: $(GOALS)
$(GOALS):;cmd.exe /c "echo off & echo 1 > NUL"
.PHONY: all $(GOALS)

(backtrace2 attached)


One of possible solutions to this dead-lock problem - instead of suspending
Main thread, Ctrl+C handler thread may notify Main thread to stop and wait
until it releases resources and goes to sleep.

Here is the patch:

https://github.com/mbuilov/gnumake-windows/blob/master/make-4.2.1-win32-ctrl-c.patch




___

File Attachments:


---
Date: Wed 14 Jun 2017 12:43:28 PM UTC  Name: backtrace1  Size: 1kB   By:
mbuilov
backtraces of dead-locked Main and Ctrl+C threads

---
Date: Wed 14 Jun 2017 12:43:28 PM UTC  Name: backtrace2  Size: 2kB   By:
mbuilov
backtraces of dead-locked Main and Ctrl+C threads


___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/


___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

Re: Are prerequisites made in deterministic order when parallelism is disabled?

[Edward Welbourne]

Brett Stahlman (13 June 2017 17:33)
> I don't see anything in the Make docs that guarantees prerequisites
> will be processed in left to right order. Opinions on the web seems to
> be split into 2 camps:
>
> 1. Make always builds dependencies in left to right order, but a
>well-designed Makefile won't rely upon it.
> 2. Make is free to build dependencies in any order it likes, provided
>it respects the stated dependencies.
>
> My own recent experience suggests #2 is the correct statement, but I
> can't rule out the possibility that a bug in my Makefile is producing
> the apparent non-determinism I'm observing. At any rate, can anyone
> point me to a definitive source on this?

I suspect most implementations of make do in fact build in left-to-right
order but none guarantee it; and I won't be surprised if GNU make used
to but has lately stopped doing so, although I can't give you a
definitive source either way.  I certainly wouldn't ever assume
deterministic build order; if one prerequisite needs to be built before
another, the make-file should express that via a dependency (perhaps
just an order-only one).

Eddy.

___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make