Re: make goes into an infinite loop

2013-01-08 Thread David Boyce
Ian,

You don't mention a platform so I will assume Linux.

GNU make 3.81 dates from 2006, and 3.82 was released in 2010. That's a lot
of bugfixes, so the first thing is to try at least 3.82 and ideally the CVS
version which has ~3 more years of fixes. Building the CVS versions is not
hard if you have a fairly modern system which either has or allows you to
add a couple of required packages (see the README.cvs file). I suggest
turning off any -O flags and adding -g; not only does this improve
debugging and stack dumps but there's a class of subtle bug which can be
triggered by optimization.

If you still see the problem with the CVS version (which identifies as
3.82.90 under --version) then you could try looking at what the process is
doing. The way I often start in situations like this is to echo $$ in the
window where you plan to run make, to get what will be its parent pid. Then
run make, and in another window run something like pstree -a -p -l pid
with that parent pid you collected. This should clarify the process tree
situation; either make will be stuck internally (no new children) or will
be waiting on some subprocess which doesn't return. Though in this case it
seems fairly clear there are no new processes.

Assuming the hang is within make, you could try forcing a core dump. This
can be done with kill -BUS (or there may be a better way, Linux is not my
native country). Something else to try first would be strace -p
make-pid. This may generate a lot of data but there may be a cycle which
provides a clue. Or make may just be sleeping, waiting for some resource,
in which case strace should help figure out what that resource is.

These may provide useful clues. However, based on the data you provided I'd
bet on a bug in make and your best hope is that it's been fixed in the last
~7 years.

David Boyce


On Mon, Jan 7, 2013 at 10:17 AM, Ian Lynagh ig...@earth.li wrote:


 Hi all,

 I've got a makefile for a large project which appears to send make into
 an infinite loop. I wonder if anyone could help me work out what is
 going wrong, and how to fix it, please?

 Unfortunately, it does so after about 30mins of compiling with parallel
 make (using e.g. -j5), and making a small testcase is unlikely to be
 easy.

 I've run make with the -d flag, and put the part of the log around where
 it starts to go wrong here:
 http://lambda.haskell.org/~igloo/make-loop.txt
 (14MB).

 My understand is:

 * around line 1, there are 2 child processes building things, one
   of which finishes

 * around line 2607, there is 1 child process building something, and
   another child is started

 * around line 18477 there are 2 child processes building things, one
   of which finishes

 * around line 36876 there is 1 process building something, and it
   finishes

 * on line 55282 an iteration of the infinite loop starts
 * on line 73678 the next iteration of the infinite loop starts
 * on line 92074 the next iteration of the infinite loop starts
 * on line 110470 the next iteration of the infinite loop starts
 * on line 128866 the next iteration of the infinite loop starts

 The end of one loop and start of the next looks like this:

 The prerequisites of `libraries/dist-haddock/index.html' are being
 made.
Finished prerequisites of target file `all_library_doc_index'.
   The prerequisites of `all_library_doc_index' are being made.
  Finished prerequisites of target file `all'.
 The prerequisites of `all' are being made.
 Considering target file `all'.
  File `all' does not exist.
   Considering target file `check_packages'.
   File `check_packages' was considered already.
   Considering target file `all_utils/mkdirhier'.

 Note that there are no Live child lines or anything.

 If I ^C it, and run make again, then compilation continues and succeeds.

 Is this a bug in make, or in my makefiles? If the former, is there a
 workaround? If the latter, have you got any advice on how to diagnose it
 please?


 I have GNU Make 3.81.


 Thanks
 Ian


 ___
 Bug-make mailing list
 Bug-make@gnu.org
 https://lists.gnu.org/mailman/listinfo/bug-make

___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make


Re: make goes into an infinite loop

2013-01-08 Thread Ian Lynagh

Hi David,

On Tue, Jan 08, 2013 at 07:13:18AM -0800, David Boyce wrote:
 
 You don't mention a platform so I will assume Linux.

Ah, sorry; the log was from OS X, but I had the same problem on
Linux/amd64.

 GNU make 3.81 dates from 2006, and 3.82 was released in 2010. That's a lot
 of bugfixes, so the first thing is to try at least 3.82 and ideally the CVS
 version which has ~3 more years of fixes.

OK, I'll give that a try.

 run make, and in another window run something like pstree -a -p -l pid

 Something else to try first would be strace -p make-pid.

I'll double check this when I test with a newer make, but from memory:
On Linux, ps aux --forest showed no child processes, and strace
printed no output (which is consistent with make just repeatedly walking
over its datastructures and not finding anything to do, so not making
any system calls).

 These may provide useful clues. However, based on the data you provided I'd
 bet on a bug in make

Thanks; that's what I thought, but I'm not familiar enough with make to
be sure.

 and your best hope is that it's been fixed in the last
 ~7 years.

Well, ideally I'd like to find a solution that doesn't require all the
users to install a newer make. But I'll do some more diagnosis and see
what I find.


Thanks
Ian


___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make


Re: make goes into an infinite loop

2013-01-08 Thread David Boyce
On Tue, Jan 8, 2013 at 8:37 AM, Ian Lynagh ig...@earth.li wrote:
 Well, ideally I'd like to find a solution that doesn't require all the
 users to install a newer make. But I'll do some more diagnosis and see
 what I find.

Yes. If it's a fixed bug then a little binary searching or a walk
through the Changes file should find the details of the specific bug
which may with luck lead to a workaround.

David

___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make


Re: make goes into an infinite loop

2013-01-08 Thread Ian Lynagh
On Tue, Jan 08, 2013 at 08:52:59AM -0800, David Boyce wrote:
 On Tue, Jan 8, 2013 at 8:37 AM, Ian Lynagh ig...@earth.li wrote:
  Well, ideally I'd like to find a solution that doesn't require all the
  users to install a newer make. But I'll do some more diagnosis and see
  what I find.

With 3.8.1, it's as I remembered:

$ top
15972 ian   20   0 84420  42m  884 R  100  1.2   4:20.49 make

$ pstree -a -p -l 15972
make,15972 -r --no-print-directory -f ghc.mk phase=final all

$ strace -p 15972
Process 15972 attached - interrupt to quit
^CProcess 15972 detached

With 3.82.90 the build succeeds.

 Yes. If it's a fixed bug then a little binary searching or a walk
 through the Changes file should find the details of the specific bug
 which may with luck lead to a workaround.

It's not an easy thing to binary search, as each run means 30mins of
compilation followed by looking to see if it's in an infinite loop.

I think what I'll do is to try to track down which change to the
makefiles tickled the bug, and see if that gives any clues for what the
problem might be, and how I might be able to work around it.


Thanks
Ian


___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make