Re: Build failures with high parallel make(1) jobs with GCC

2014-04-23 Thread Ian Lepore 
On Tue, 2014-04-22 at 00:13 -0400, Glen Barber wrote:
 On Mon, Apr 21, 2014 at 10:05:57PM -0600, Ian Lepore wrote:
  On Mon, 2014-04-21 at 23:26 -0400, Glen Barber wrote:
   On Mon, Apr 21, 2014 at 11:21:24PM -0400, Glen Barber wrote:
On Mon, Apr 21, 2014 at 09:09:42PM -0600, Ian Lepore wrote:
  The last successful build for powerpc on head/ was April 8.  But I 
  am
  having trouble tracking down what commits may (or may not) have
  contributed to recent high-parallel build failures.
  
 
 A couple weeks corresponds somewhat with the parallel subdir build
 changes (it's about 3 weeks now).  Try this patch I cooked up today 
 for
 $work, and in src/lib/Makefile add .WAIT (as if it were a directory
 name) between ${SUBDIR_ORDERED} and the rest of the directories.
 

The patch fails to apply cleanly, but as far as I can tell, it is due to
whitespace.

I'll hand-patch it, and report back.

   
   Nope, I'm getting conflicts on revisions as far back as r251749.
   
   Glen
   
  
  This one should work better.  The lib/Makefile is included this time.
  
 
 This patch applies fine.
 
 It seems to already have an effect, but I won't go so far to say it
 works until I see the 'World build completed: ' message.
 
 I'll follow up tomorrow once build is done.
 
 Thanks!
 
 Glen
 

I think the change to lib/Makefile was not sufficient.  After looking at
the notes on dependenices at the top of the makefile, I think the
attached has some chance of helping.  The bsd.subdir.mk changes are now
commited as r264822. 

-- Ian

Index: lib/Makefile
===
--- lib/Makefile	(revision 264768)
+++ lib/Makefile	(working copy)
@@ -31,9 +31,12 @@
 # Except it appears bind needs to be compiled last
 
 SUBDIR_ORDERED=	${_csu} \
+	.WAIT \
 	libc \
 	libc_nonshared \
+	.WAIT \
 	libbsm \
+	.WAIT \
 	libauditd \
 	libutil \
 	libpjdlog \
@@ -54,6 +57,7 @@ SUBDIR_ORDERED=	${_csu} \
 	libsbuf \
 	libtacplus \
 	${_libypclnt} \
+	.WAIT \
 	${_libcxxrt} \
 	${_libcplusplus}
 
@@ -62,6 +66,7 @@ SUBDIR_ORDERED+=	libcom_err
 .endif
 
 SUBDIR=	${SUBDIR_ORDERED} \
+	.WAIT \
 	libalias \
 	libarchive \
 	${_libatm} \
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Build failures with high parallel make(1) jobs with GCC

2014-04-23 Thread Glen Barber 
On Wed, Apr 23, 2014 at 07:04:46AM -0600, Ian Lepore wrote:
 I think the change to lib/Makefile was not sufficient.  After looking at
 the notes on dependenices at the top of the makefile, I think the
 attached has some chance of helping.  The bsd.subdir.mk changes are now
 commited as r264822. 
 

Thanks, I'll let you know in a few hours when the builds that are in
progress now are finished.

Glen



pgpinVhb98T9b.pgp
Description: PGP signature

Re: Build failures with high parallel make(1) jobs with GCC

2014-04-21 Thread Ian Lepore 
On Mon, 2014-04-21 at 22:54 -0400, Glen Barber wrote:
 I have been pounding my head against the desk for longer than I care to
 admit with this failure.
 
 I see this with powerpc, powerpc64, and now ia64.  I initially thought
 it was specific to powerpc{,64}, but now realize ia64 is also affected.
 
 This build was running with -j48 on a 48-core machine, when the
 following caused build failure:
 
 ===
 
 /usr/obj/ia64.ia64/usr/src/tmp/usr/bin/ld: cannot find -lm
 --- libstdc++.so.6 ---
 *** [libstdc++.so.6] Error code 1
 
 make[4]: stopped in /usr/src/gnu/lib/libstdc++
 A failure has been detected in another branch of the parallel make
 
 ===
 
 It is unclear to me when exactly this started happening, but it seems at
 least two weeks is a reasonable estimate.
 
 I realize this is not an entirely large chunk of useful information
 regarding the build failure, but I have determined it is entirely
 reproducible.  I have determined that the failure case seems to
 disappear with make(1) jobs = 10, at least for powerpc.
 
 The last successful build for powerpc on head/ was April 8.  But I am
 having trouble tracking down what commits may (or may not) have
 contributed to recent high-parallel build failures.
 
 Glen
 

A couple weeks corresponds somewhat with the parallel subdir build
changes (it's about 3 weeks now).  Try this patch I cooked up today for
$work, and in src/lib/Makefile add .WAIT (as if it were a directory
name) between ${SUBDIR_ORDERED} and the rest of the directories.

-- Ian

diff -r 67802e319fc6 share/mk/bsd.subdir.mk
--- a/share/mk/bsd.subdir.mk	Sun Apr 20 21:01:07 2014 -0600
+++ b/share/mk/bsd.subdir.mk	Mon Apr 21 06:59:37 2014 -0600
@@ -4,10 +4,10 @@
 # The include file bsd.subdir.mk contains the default targets
 # for building subdirectories.
 #
-# For all of the directories listed in the variable SUBDIRS, the
+# For all of the directories listed in the variable SUBDIR, the
 # specified directory will be visited and the target made. There is
 # also a default target which allows the command make subdir where
-# subdir is any directory listed in the variable SUBDIRS.
+# subdir is any directory listed in the variable SUBDIR.
 #
 #
 # +++ variables +++
@@ -42,7 +42,7 @@ distribute:
 
 _SUBDIR: .USE
 .if defined(SUBDIR)  !empty(SUBDIR)  !defined(NO_SUBDIR)
-	@${_+_}for entry in ${SUBDIR}; do \
+	@${_+_}for entry in ${SUBDIR:N.WAIT}; do \
 		if test -d ${.CURDIR}/$${entry}.${MACHINE_ARCH}; then \
 			${ECHODIR} === ${DIRPRFX}$${entry}.${MACHINE_ARCH} (${.TARGET:realinstall=install}); \
 			edir=$${entry}.${MACHINE_ARCH}; \
@@ -57,7 +57,7 @@ distribute:
 	done
 .endif
 
-${SUBDIR}: .PHONY
+${SUBDIR:N.WAIT}: .PHONY
 	${_+_}@if test -d ${.TARGET}.${MACHINE_ARCH}; then \
 		cd ${.CURDIR}/${.TARGET}.${MACHINE_ARCH}; \
 	else \
@@ -65,13 +65,18 @@ distribute:
 	fi; \
 	${MAKE} all
 
+__wait=.WAIT
 .for __target in all all-man checkdpadd clean cleandepend cleandir \
 depend distribute lint maninstall manlint \
 obj objlink realinstall regress tags \
 ${SUBDIR_TARGETS}
 .ifdef SUBDIR_PARALLEL
+__subdir_targets=
 .for __dir in ${SUBDIR}
-${__target}: ${__target}_subdir_${__dir}
+.if ${__wait} == ${__dir}
+__subdir_targets+= .WAIT
+.else
+__subdir_targets+= ${__target}_subdir_${__dir}
 ${__target}_subdir_${__dir}: .MAKE
 	@${_+_}set -e; \
 		if test -d ${.CURDIR}/${__dir}.${MACHINE_ARCH}; then \
@@ -85,7 +90,9 @@ distribute:
 		fi; \
 		${MAKE} ${__target:realinstall=install} \
 		DIRPRFX=${DIRPRFX}$$edir/
+.endif
 .endfor
+${__target}: ${__subdir_targets}
 .else
 ${__target}: _SUBDIR
 .endif
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Build failures with high parallel make(1) jobs with GCC

2014-04-21 Thread Glen Barber 
On Mon, Apr 21, 2014 at 09:09:42PM -0600, Ian Lepore wrote:
  The last successful build for powerpc on head/ was April 8.  But I am
  having trouble tracking down what commits may (or may not) have
  contributed to recent high-parallel build failures.
  
 
 A couple weeks corresponds somewhat with the parallel subdir build
 changes (it's about 3 weeks now).  Try this patch I cooked up today for
 $work, and in src/lib/Makefile add .WAIT (as if it were a directory
 name) between ${SUBDIR_ORDERED} and the rest of the directories.
 

The patch fails to apply cleanly, but as far as I can tell, it is due to
whitespace.

I'll hand-patch it, and report back.

Thanks.

Glen



pgp87jr3EfwFQ.pgp
Description: PGP signature

Re: Build failures with high parallel make(1) jobs with GCC

2014-04-21 Thread Glen Barber 
On Mon, Apr 21, 2014 at 11:21:24PM -0400, Glen Barber wrote:
 On Mon, Apr 21, 2014 at 09:09:42PM -0600, Ian Lepore wrote:
   The last successful build for powerpc on head/ was April 8.  But I am
   having trouble tracking down what commits may (or may not) have
   contributed to recent high-parallel build failures.
   
  
  A couple weeks corresponds somewhat with the parallel subdir build
  changes (it's about 3 weeks now).  Try this patch I cooked up today for
  $work, and in src/lib/Makefile add .WAIT (as if it were a directory
  name) between ${SUBDIR_ORDERED} and the rest of the directories.
  
 
 The patch fails to apply cleanly, but as far as I can tell, it is due to
 whitespace.
 
 I'll hand-patch it, and report back.
 

Nope, I'm getting conflicts on revisions as far back as r251749.

Glen



pgp_zIs2itvGG.pgp
Description: PGP signature

Re: Build failures with high parallel make(1) jobs with GCC

2014-04-21 Thread Ian Lepore 
On Mon, 2014-04-21 at 23:26 -0400, Glen Barber wrote:
 On Mon, Apr 21, 2014 at 11:21:24PM -0400, Glen Barber wrote:
  On Mon, Apr 21, 2014 at 09:09:42PM -0600, Ian Lepore wrote:
The last successful build for powerpc on head/ was April 8.  But I am
having trouble tracking down what commits may (or may not) have
contributed to recent high-parallel build failures.

   
   A couple weeks corresponds somewhat with the parallel subdir build
   changes (it's about 3 weeks now).  Try this patch I cooked up today for
   $work, and in src/lib/Makefile add .WAIT (as if it were a directory
   name) between ${SUBDIR_ORDERED} and the rest of the directories.
   
  
  The patch fails to apply cleanly, but as far as I can tell, it is due to
  whitespace.
  
  I'll hand-patch it, and report back.
  
 
 Nope, I'm getting conflicts on revisions as far back as r251749.
 
 Glen
 

Doh!  I completely forgot that's against 8.2 that we use at work.  I'll
re-spin it for -current.

-- Ina


___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Build failures with high parallel make(1) jobs with GCC

2014-04-21 Thread Ian Lepore 
On Mon, 2014-04-21 at 23:26 -0400, Glen Barber wrote:
 On Mon, Apr 21, 2014 at 11:21:24PM -0400, Glen Barber wrote:
  On Mon, Apr 21, 2014 at 09:09:42PM -0600, Ian Lepore wrote:
The last successful build for powerpc on head/ was April 8.  But I am
having trouble tracking down what commits may (or may not) have
contributed to recent high-parallel build failures.

   
   A couple weeks corresponds somewhat with the parallel subdir build
   changes (it's about 3 weeks now).  Try this patch I cooked up today for
   $work, and in src/lib/Makefile add .WAIT (as if it were a directory
   name) between ${SUBDIR_ORDERED} and the rest of the directories.
   
  
  The patch fails to apply cleanly, but as far as I can tell, it is due to
  whitespace.
  
  I'll hand-patch it, and report back.
  
 
 Nope, I'm getting conflicts on revisions as far back as r251749.
 
 Glen
 

This one should work better.  The lib/Makefile is included this time.

-- Ian

Index: share/mk/bsd.subdir.mk
===
--- share/mk/bsd.subdir.mk	(revision 264744)
+++ share/mk/bsd.subdir.mk	(working copy)
@@ -45,7 +45,7 @@ distribute: .MAKE
 
 _SUBDIR: .USE .MAKE
 .if defined(SUBDIR)  !empty(SUBDIR)  !defined(NO_SUBDIR)
-	@${_+_}set -e; for entry in ${SUBDIR}; do \
+	@${_+_}set -e; for entry in ${SUBDIR:N.WAIT}; do \
 		if test -d ${.CURDIR}/$${entry}.${MACHINE_ARCH}; then \
 			${ECHODIR} === ${DIRPRFX}$${entry}.${MACHINE_ARCH} (${.TARGET:realinstall=install}); \
 			edir=$${entry}.${MACHINE_ARCH}; \
@@ -60,7 +60,7 @@ _SUBDIR: .USE .MAKE
 	done
 .endif
 
-${SUBDIR}: .PHONY .MAKE
+${SUBDIR:N.WAIT}: .PHONY .MAKE
 	${_+_}@if test -d ${.TARGET}.${MACHINE_ARCH}; then \
 		cd ${.CURDIR}/${.TARGET}.${MACHINE_ARCH}; \
 	else \
@@ -68,12 +68,18 @@ _SUBDIR: .USE .MAKE
 	fi; \
 	${MAKE} all
 
+# Work around parsing of .if nested in .for by putting .WAIT string into a var.
+__wait= .WAIT
 .for __target in all all-man checkdpadd clean cleandepend cleandir \
 cleanilinks depend distribute lint maninstall manlint obj objlink \
 realinstall regress tags ${SUBDIR_TARGETS}
 .ifdef SUBDIR_PARALLEL
+__subdir_targets=
 .for __dir in ${SUBDIR}
-${__target}: ${__target}_subdir_${__dir}
+.if ${__wait} == ${__dir}
+__subdir_targets+= .WAIT
+.else
+__subdir_targets+= ${__target}_subdir_${__dir}
 ${__target}_subdir_${__dir}: .MAKE
 	@${_+_}set -e; \
 		if test -d ${.CURDIR}/${__dir}.${MACHINE_ARCH}; then \
@@ -87,7 +93,9 @@ ${__target}_subdir_${__dir}: .MAKE
 		fi; \
 		${MAKE} ${__target:realinstall=install} \
 		DIRPRFX=${DIRPRFX}$$edir/
+.endif
 .endfor
+${__target}: ${__subdir_targets}
 .else
 ${__target}: _SUBDIR
 .endif
Index: lib/Makefile
===
--- lib/Makefile	(revision 264744)
+++ lib/Makefile	(working copy)
@@ -62,6 +62,7 @@ SUBDIR_ORDERED+=	libcom_err
 .endif
 
 SUBDIR=	${SUBDIR_ORDERED} \
+	.WAIT \
 	libalias \
 	libarchive \
 	${_libatm} \
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Build failures with high parallel make(1) jobs with GCC

2014-04-21 Thread Glen Barber 
On Mon, Apr 21, 2014 at 10:05:57PM -0600, Ian Lepore wrote:
 On Mon, 2014-04-21 at 23:26 -0400, Glen Barber wrote:
  On Mon, Apr 21, 2014 at 11:21:24PM -0400, Glen Barber wrote:
   On Mon, Apr 21, 2014 at 09:09:42PM -0600, Ian Lepore wrote:
 The last successful build for powerpc on head/ was April 8.  But I am
 having trouble tracking down what commits may (or may not) have
 contributed to recent high-parallel build failures.
 

A couple weeks corresponds somewhat with the parallel subdir build
changes (it's about 3 weeks now).  Try this patch I cooked up today for
$work, and in src/lib/Makefile add .WAIT (as if it were a directory
name) between ${SUBDIR_ORDERED} and the rest of the directories.

   
   The patch fails to apply cleanly, but as far as I can tell, it is due to
   whitespace.
   
   I'll hand-patch it, and report back.
   
  
  Nope, I'm getting conflicts on revisions as far back as r251749.
  
  Glen
  
 
 This one should work better.  The lib/Makefile is included this time.
 

This patch applies fine.

It seems to already have an effect, but I won't go so far to say it
works until I see the 'World build completed: ' message.

I'll follow up tomorrow once build is done.

Thanks!

Glen



pgpLJk4hZ47Ik.pgp
Description: PGP signature