Re: [libstdc++, patch] Fix build on APFS file system

[Jonathan Wakely]

On 24/10/17 17:14 +0200, FX wrote:

Presumably for the i386 and x86_64 multilibs (does it make any
difference if you do --disable-multlib? It would be easier to debug the
output if each command was only done once).


Same problem without multilib.


OK good, so then test without multilib, because there's half as much
output to examine.


I can't really do much more by email, somebody with access to one of
these failing systems needs to debug this. The problem has nothing to
do with C++ so I don't see why it needs to be solved by libstdc++
people -- just modify the makefile and inspect the results.


Thanks for the help so far. If I could debug this I would, but the complexity 
of this build machinery puts it out of my reach.


It's "only" a bunch of shell commands :-)

Re: [libstdc++, patch] Fix build on APFS file system

[FX]

> Presumably for the i386 and x86_64 multilibs (does it make any
> difference if you do --disable-multlib? It would be easier to debug the
> output if each command was only done once).

Same problem without multilib.

> I can't really do much more by email, somebody with access to one of
> these failing systems needs to debug this. The problem has nothing to
> do with C++ so I don't see why it needs to be solved by libstdc++
> people -- just modify the makefile and inspect the results.

Thanks for the help so far. If I could debug this I would, but the complexity 
of this build machinery puts it out of my reach.

FX

Re: [libstdc++, patch] Fix build on APFS file system

[Jonathan Wakely]

On 24/10/17 16:03 +0200, FX wrote:

OK, could you try this, and share the full output?


Attached is the output of “make -j4” in a build at stage1, where I have just run 
"rm -rf x86_64-apple-darwin17.0.0/**/libstdc++-v3”, with the patch applied.

FX


There are two commands that create the include/ext/new_allocator.h
symlink:


cd ./ext && ln -s /Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/algorithm 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/aligned_buffer.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/alloc_traits.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/atomicity.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/array_allocator.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/bitmap_allocator.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/cast.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/cmath 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/codecvt_specializations.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/concurrence.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/debug_allocator.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/enc_filebuf.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/extptr_allocator.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/stdio_filebuf.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/stdio_sync_filebuf.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/functional 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/iterator 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/malloc_allocator.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/memory 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/mt_allocator.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/new_allocator.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/numeric 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/numeric_traits.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/pod_char_traits.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/pointer.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/pool_allocator.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/rb_tree 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/random 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/random.tcc 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/rope 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/ropeimpl.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/slist 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/string_conversions.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/throw_allocator.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/typelist.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/type_traits.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/rc_string_base.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/sso_string_base.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/vstring.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/vstring.tcc 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/vstring_fwd.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/vstring_util.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/backward/hash_set 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/backward/hash_map .


and:


cd ./ext && ln -s /Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/algorithm 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/aligned_buffer.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/alloc_traits.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/atomicity.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/array_allocator.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/bitmap_allocator.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/cast.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/cmath 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/codecvt_specializations.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/concurrence.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/debug_allocator.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/enc_filebuf.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/extptr_allocator.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/stdio_filebuf.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/stdio_sync_filebuf.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/functional 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/iterator 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/malloc_allocator.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/memory 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/mt_allocator.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/new_allocator.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/numeric 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/numeric_traits.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/pod_char_traits.h 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/ext/pointer.h 

Re: [libstdc++, patch] Fix build on APFS file system

[FX]

The output of “make -j4” in a build at stage1, where I have just run "rm -rf 
x86_64-apple-darwin17.0.0/**/libstdc++-v3”, with the patch applied: 
https://gist.github.com/fxcoudert/46f0026c44eb3db2937ac0e92a477338

FX

Re: [libstdc++, patch] Fix build on APFS file system

[Jonathan Wakely]

On 24/10/17 11:06 +0200, FX wrote:

Thanks Jonathan. I tried, and it does not work, I still get the same error:

Making all in include
mkdir -p ./x86_64-apple-darwin17.0.0/bits/stdc++.h.gch
mkdir -p ./x86_64-apple-darwin17.0.0/bits/stdc++.h.gch
/Users/fx/devel/gcc/ibin/./gcc/xgcc -shared-libgcc 
-B/Users/fx/devel/gcc/ibin/./gcc -nostdinc++ 
-L/Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/src 
-L/Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/src/.libs
 
-L/Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/libsupc++/.libs
 -B/Users/fx/devel/gcc/irun/x86_64-apple-darwin17.0.0/bin/ 
-B/Users/fx/devel/gcc/irun/x86_64-apple-darwin17.0.0/lib/ -isystem 
/Users/fx/devel/gcc/irun/x86_64-apple-darwin17.0.0/include -isystem 
/Users/fx/devel/gcc/irun/x86_64-apple-darwin17.0.0/sys-include  -m32 -x 
c++-header -nostdinc++ -g -O2  -m32  
-I/Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/include/x86_64-apple-darwin17.0.0
 -I/Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/include 
-I/Users/fx/devel/gcc/trunk/libstdc++-v3/libsupc++  -O2 -g -std=gnu++0x 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/precompiled/stdc++.h \
-o x86_64-apple-darwin17.0.0/bits/stdc++.h.gch/O2ggnu++0x.gch
/Users/fx/devel/gcc/ibin/./gcc/xgcc -shared-libgcc 
-B/Users/fx/devel/gcc/ibin/./gcc -nostdinc++ 
-L/Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/src 
-L/Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/src/.libs
 
-L/Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/libsupc++/.libs
 -B/Users/fx/devel/gcc/irun/x86_64-apple-darwin17.0.0/bin/ 
-B/Users/fx/devel/gcc/irun/x86_64-apple-darwin17.0.0/lib/ -isystem 
/Users/fx/devel/gcc/irun/x86_64-apple-darwin17.0.0/include -isystem 
/Users/fx/devel/gcc/irun/x86_64-apple-darwin17.0.0/sys-include  -m32 -x 
c++-header -nostdinc++ -g -O2  -m32  
-I/Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/include/x86_64-apple-darwin17.0.0
 -I/Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/include 
-I/Users/fx/devel/gcc/trunk/libstdc++-v3/libsupc++  -O2 -g 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/precompiled/stdc++.h -o 
x86_64-apple-darwin17.0.0/bits/stdc++.h.gch/O2g.gch
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/precompiled/stdc++.h:44:10: 
fatal error: cstdarg: No such file or directory
#include 
 ^
compilation terminated.
make[7]: *** [x86_64-apple-darwin17.0.0/bits/stdc++.h.gch/O2ggnu++0x.gch] Error 
1


OK, could you try this, and share the full output?


diff --git a/libstdc++-v3/include/Makefile.am b/libstdc++-v3/include/Makefile.am
index 2c4d193d0a4..92b9db80143 100644
--- a/libstdc++-v3/include/Makefile.am
+++ b/libstdc++-v3/include/Makefile.am
@@ -1022,43 +1022,43 @@ all-local: ${allstamped} ${allcreated}
 # Ignore errors from $(LN_S) because the links may already exist.
 stamp-std: ${std_headers}
 	@-mkdir -p ${std_builddir}
-	@-cd ${std_builddir} && $(LN_S) $? . 2>/dev/null
+	-cd ${std_builddir} && $(LN_S) $? .
 	@$(STAMP) stamp-std
 
 stamp-bits: ${bits_headers}
 	@-mkdir -p ${bits_builddir}
-	@-cd ${bits_builddir} && $(LN_S) $? . 2>/dev/null
+	-cd ${bits_builddir} && $(LN_S) $? .
 	@$(STAMP) stamp-bits
 
 stamp-bits-sup: stamp-bits ${bits_sup_headers}
-	@-cd ${bits_builddir} && $(LN_S) $? . 2>/dev/null
+	-cd ${bits_builddir} && $(LN_S) $? .
 	@$(STAMP) stamp-bits-sup
 
 stamp-c_base: ${c_base_headers}
 	@-mkdir -p ${c_base_builddir}
-	@-cd ${c_base_builddir} && $(LN_S) $? . 2>/dev/null
+	-cd ${c_base_builddir} && $(LN_S) $? .
 	@$(STAMP) stamp-c_base
 
 stamp-c_base_extra: ${c_base_headers_extra}
 	@-mkdir -p ${bits_builddir}
-	@-cd ${bits_builddir} && $(LN_S) $? . 2>/dev/null
+	-cd ${bits_builddir} && $(LN_S) $? .
 	@$(STAMP) stamp-c_base_extra
 
 stamp-c_compatibility: ${c_compatibility_headers_extra}
 	@-mkdir -p ${c_compatibility_builddir}
-	@-if [ ! -z "${c_compatibility_headers_extra}" ]; then \
-	  cd ${c_compatibility_builddir} && $(LN_S) $? . 2>/dev/null ;\
+	-if [ ! -z "${c_compatibility_headers_extra}" ]; then \
+	  cd ${c_compatibility_builddir} && $(LN_S) $? . ;\
 	fi
 	@$(STAMP) stamp-c_compatibility
 
 stamp-backward: ${backward_headers}
 	@-mkdir -p ${backward_builddir}
-	@-cd ${backward_builddir} && $(LN_S) $? . 2>/dev/null
+	-cd ${backward_builddir} && $(LN_S) $? .
 	@$(STAMP) stamp-backward
 
 stamp-ext: ${ext_headers}
 	@-mkdir -p ${ext_builddir}
-	@-cd ${ext_builddir} && $(LN_S) $? . 2>/dev/null
+	-cd ${ext_builddir} && $(LN_S) $? .
 	@$(STAMP) stamp-ext
 
 # Have to deal with nested include directories, gah! Strip off source
@@ -1114,47 +1114,47 @@ stamp-pb:
 
 stamp-tr1: ${tr1_headers}
 	@-mkdir -p ${tr1_builddir}
-	@-cd ${tr1_builddir} && $(LN_S) $? . 2>/dev/null
+	-cd ${tr1_builddir} && $(LN_S) $? .
 	@$(STAMP) stamp-tr1
 
 stamp-tr2: ${tr2_headers}
 	@-mkdir -p ${tr2_builddir}
-	@-cd ${tr2_builddir} && $(LN_S) $? . 2>/dev/null
+	-cd 

Re: [libstdc++, patch] Fix build on APFS file system

[FX]

Thanks Jonathan. I tried, and it does not work, I still get the same error:

Making all in include
mkdir -p ./x86_64-apple-darwin17.0.0/bits/stdc++.h.gch
mkdir -p ./x86_64-apple-darwin17.0.0/bits/stdc++.h.gch
/Users/fx/devel/gcc/ibin/./gcc/xgcc -shared-libgcc 
-B/Users/fx/devel/gcc/ibin/./gcc -nostdinc++ 
-L/Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/src 
-L/Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/src/.libs
 
-L/Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/libsupc++/.libs
 -B/Users/fx/devel/gcc/irun/x86_64-apple-darwin17.0.0/bin/ 
-B/Users/fx/devel/gcc/irun/x86_64-apple-darwin17.0.0/lib/ -isystem 
/Users/fx/devel/gcc/irun/x86_64-apple-darwin17.0.0/include -isystem 
/Users/fx/devel/gcc/irun/x86_64-apple-darwin17.0.0/sys-include  -m32 -x 
c++-header -nostdinc++ -g -O2  -m32  
-I/Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/include/x86_64-apple-darwin17.0.0
 -I/Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/include 
-I/Users/fx/devel/gcc/trunk/libstdc++-v3/libsupc++  -O2 -g -std=gnu++0x 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/precompiled/stdc++.h \
-o x86_64-apple-darwin17.0.0/bits/stdc++.h.gch/O2ggnu++0x.gch
/Users/fx/devel/gcc/ibin/./gcc/xgcc -shared-libgcc 
-B/Users/fx/devel/gcc/ibin/./gcc -nostdinc++ 
-L/Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/src 
-L/Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/src/.libs
 
-L/Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/libsupc++/.libs
 -B/Users/fx/devel/gcc/irun/x86_64-apple-darwin17.0.0/bin/ 
-B/Users/fx/devel/gcc/irun/x86_64-apple-darwin17.0.0/lib/ -isystem 
/Users/fx/devel/gcc/irun/x86_64-apple-darwin17.0.0/include -isystem 
/Users/fx/devel/gcc/irun/x86_64-apple-darwin17.0.0/sys-include  -m32 -x 
c++-header -nostdinc++ -g -O2  -m32  
-I/Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/include/x86_64-apple-darwin17.0.0
 -I/Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/include 
-I/Users/fx/devel/gcc/trunk/libstdc++-v3/libsupc++  -O2 -g 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/precompiled/stdc++.h -o 
x86_64-apple-darwin17.0.0/bits/stdc++.h.gch/O2g.gch
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/precompiled/stdc++.h:44:10: 
fatal error: cstdarg: No such file or directory
 #include 
  ^
compilation terminated.
make[7]: *** [x86_64-apple-darwin17.0.0/bits/stdc++.h.gch/O2ggnu++0x.gch] Error 
1

Re: [libstdc++, patch] Fix build on APFS file system

[Jonathan Wakely]

On 23/10/17 19:48 +0200, FX wrote:

The patch seems like a rough bandaid to hide the real bug.  Better to identify 
the real bug.  If there is a missing dependency, then I'd like to think that 
adding the right dependency should resolve the issue.


So far, apart from a suggestion from Marc, I haven’t received any help or 
advice in identifying or debugging the issue.

FX


You could try this.


diff --git a/libstdc++-v3/include/Makefile.am b/libstdc++-v3/include/Makefile.am
index 2c4d193d0a4..39083cc4ebc 100644
--- a/libstdc++-v3/include/Makefile.am
+++ b/libstdc++-v3/include/Makefile.am
@@ -1016,6 +1016,8 @@ allcreated = \
 # Here are the rules for building the headers
 all-local: ${allstamped} ${allcreated}
 
+${pch_output} : | ${allstamped}
+
 # Ignore errors from 'mkdir -p' to avoid parallel make failure on
 # systems with broken mkdir.  Call mkdir unconditionally because
 # it is just as cheap to avoid going through the shell.
diff --git a/libstdc++-v3/include/Makefile.in b/libstdc++-v3/include/Makefile.in
index bc8556c68d2..e1a852e2906 100644
--- a/libstdc++-v3/include/Makefile.in
+++ b/libstdc++-v3/include/Makefile.in
@@ -1465,6 +1465,8 @@ uninstall-am:
 # Here are the rules for building the headers
 all-local: ${allstamped} ${allcreated}
 
+${pch_output} : | ${allstamped}
+
 # Ignore errors from 'mkdir -p' to avoid parallel make failure on
 # systems with broken mkdir.  Call mkdir unconditionally because
 # it is just as cheap to avoid going through the shell.

Re: [libstdc++, patch] Fix build on APFS file system

[FX]

> The patch seems like a rough bandaid to hide the real bug.  Better to 
> identify the real bug.  If there is a missing dependency, then I'd like to 
> think that adding the right dependency should resolve the issue.

So far, apart from a suggestion from Marc, I haven’t received any help or 
advice in identifying or debugging the issue.

FX

Re: [libstdc++, patch] Fix build on APFS file system

[Mike Stump]

On Oct 18, 2017, at 7:51 AM, FX  wrote:
> 
> Parallel builds of libstdc++ on APFS filesystem (with 1 ns granularity) on 
> macOS 10.13 often fail (failure rate for “make -j2” to “make -j8” is about 
> 60% from my own builds and results reported by others): 
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81797
> This is reproducible with several versions of GNU make.
> 
> Changing libstdc++’s makefile to mark install-headers with .NOTPARALLEL fixes 
> the issue. We've carried that patch in Homebrew (https://brew.sh) for a few 
> months now, and have had no report of build issues since then.
> 
> Bootstrapped and regtested on x86_64-apple-darwin17 (as well as other 
> platforms). OK to commit?

The patch seems like a rough bandaid to hide the real bug.  Better to identify 
the real bug.  If there is a missing dependency, then I'd like to think that 
adding the right dependency should resolve the issue.

Re: [libstdc++, patch] Fix build on APFS file system

[Petr Ovtchenkov]

On Thu, 19 Oct 2017 10:37:14 +0200
Richard Biener  wrote:

> On Wed, Oct 18, 2017 at 11:58 PM, FX  wrote:
> >> Could you test using .PHONY: install-headers instead?
> >> That target *is* phony, so telling make that seems sensible.
> >
> > I’ve tried adding it to the existing .PHONY list:
> >
> > Index: libstdc++-v3/include/Makefile.in
> > ===
> > --- libstdc++-v3/include/Makefile.in(revision 253855)
> > +++ libstdc++-v3/include/Makefile.in(working copy)
> > @@ -1449,6 +1449,7 @@ uninstall-am:
> > distclean-libtool dvi dvi-am html html-am info info-am install \
> > install-am install-data install-data-am install-data-local \
> > install-dvi install-dvi-am install-exec install-exec-am \
> > +   install-freestanding-headers install-headers \
> > install-html install-html-am install-info install-info-am \
> > install-man install-pdf install-pdf-am install-ps \
> > install-ps-am install-strip installcheck installcheck-am \
> >
> >
> > But that doesn’t work:
> >
> > In file included
> > from 
> > /Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/include/bits/exception_ptr.h:39:0,
> > from /Users/fx/devel/gcc/trunk/libstdc++-v3/libsupc++/exception:143,
> > from 
> > /Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/include/ios:39,
> > from 
> > /Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/include/istream:38,
> > from 
> > /Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/include/sstream:38,
> > from 
> > /Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/include/complex:45,
> > from 
> > /Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/include/ccomplex:39,
> > from 
> > /Users/fx/devel/gcc/trunk/libstdc++-v3/include/precompiled/stdc++.h:52: 
> > /Users/fx/devel/gcc/trunk/libstdc++-v3/libsupc++/typeinfo:36:10:
> > fatal error: bits/hash_bytes.h: No such file or directory #include 
> > 
> >   ^~~
> 
> So the issue is PCH generation (why's that re-generated at install time?).

May be indeed: I have a lot parallel builds and never see problem like this,
but I skip PCH generation.

--

   - ptr

Re: [libstdc++, patch] Fix build on APFS file system

[FX]

> So the issue is PCH generation (why's that re-generated at install time?).

As suggested by Marc in the PR, I've removed the @ from include/Makefile.in, 
and removed the leading - for lines with LN_S.

The result of "make -d --trace -j8 all-target-libstdc++-v3", in a build where 
x86_64-apple-darwin17.0.0/libstdc++-v3 was entirely removed, can be found here: 
https://gist.github.com/fxcoudert/b621465a794d968593bc7ed90c0fc1fb

FX

Re: [libstdc++, patch] Fix build on APFS file system

[Richard Biener]

On Wed, Oct 18, 2017 at 11:58 PM, FX  wrote:
>> Could you test using .PHONY: install-headers instead?
>> That target *is* phony, so telling make that seems sensible.
>
> I’ve tried adding it to the existing .PHONY list:
>
> Index: libstdc++-v3/include/Makefile.in
> ===
> --- libstdc++-v3/include/Makefile.in(revision 253855)
> +++ libstdc++-v3/include/Makefile.in(working copy)
> @@ -1449,6 +1449,7 @@ uninstall-am:
> distclean-libtool dvi dvi-am html html-am info info-am install \
> install-am install-data install-data-am install-data-local \
> install-dvi install-dvi-am install-exec install-exec-am \
> +   install-freestanding-headers install-headers \
> install-html install-html-am install-info install-info-am \
> install-man install-pdf install-pdf-am install-ps \
> install-ps-am install-strip installcheck installcheck-am \
>
>
> But that doesn’t work:
>
> In file included from 
> /Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/include/bits/exception_ptr.h:39:0,
>  from 
> /Users/fx/devel/gcc/trunk/libstdc++-v3/libsupc++/exception:143,
>  from 
> /Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/include/ios:39,
>  from 
> /Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/include/istream:38,
>  from 
> /Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/include/sstream:38,
>  from 
> /Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/include/complex:45,
>  from 
> /Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/include/ccomplex:39,
>  from 
> /Users/fx/devel/gcc/trunk/libstdc++-v3/include/precompiled/stdc++.h:52:
> /Users/fx/devel/gcc/trunk/libstdc++-v3/libsupc++/typeinfo:36:10: fatal error: 
> bits/hash_bytes.h: No such file or directory
>  #include 
>   ^~~

So the issue is PCH generation (why's that re-generated at install time?).

Richard.

Re: [libstdc++, patch] Fix build on APFS file system

[Jonathan Wakely]

On 18/10/17 18:40 -0400, Hans-Peter Nilsson wrote:

On Wed, 18 Oct 2017, FX wrote:


Parallel builds of libstdc++ on APFS filesystem (with 1 ns granularity) on 
macOS 10.13 often fail (failure rate for ?make -j2? to ?make -j8? is about 60% 
from my own builds and results reported by others): 
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81797
This is reproducible with several versions of GNU make.

Changing libstdc++?s makefile to mark install-headers with .NOTPARALLEL fixes 
the issue. We've carried that patch in Homebrew (https://brew.sh) for a few 
months now, and have had no report of build issues since then.

Bootstrapped and regtested on x86_64-apple-darwin17 (as well as other 
platforms). OK to commit?


Maybe moot now, but .NOTPARALLEL doesn't work the way you imply
in the patch.  From TFM of GNU Make 3.81 and 4: "Any
prerequisites on this target are ignored."

To wit, it seems you'd get the effect, but for *every target* in
this Makefile.  (If I were you and the patch is still proposed,
I'd remove the unused target and add a comment on the intent.)

Assuming the documentation and my reading is correct.


I think it is.

Re: [libstdc++, patch] Fix build on APFS file system

[Jonathan Wakely]

On 18/10/17 22:05 +0100, Jonathan Wakely wrote:

On 18/10/17 16:51 +0200, FX wrote:

Parallel builds of libstdc++ on APFS filesystem (with 1 ns granularity) on 
macOS 10.13 often fail (failure rate for “make -j2” to “make -j8” is about 60% 
from my own builds and results reported by others): 
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81797
This is reproducible with several versions of GNU make.

Changing libstdc++’s makefile to mark install-headers with .NOTPARALLEL fixes 
the issue. We've carried that patch in Homebrew (https://brew.sh) for a few 
months now, and have had no report of build issues since then.

Bootstrapped and regtested on x86_64-apple-darwin17 (as well as other 
platforms). OK to commit?


Could you test using .PHONY: install-headers instead?

That target *is* phony, so telling make that seems sensible.

Presumably the same change is needed for install-freestanding-headers,
since it's very similar.



.NOTPARALLEL ignores any prerequisites, so there's no point listing
install-headers as the prerequisite. Your patch disables parallelism
for all targets in the libstdc++-v3/install directory.

There must be another solution.

Re: [libstdc++, patch] Fix build on APFS file system

[Hans-Peter Nilsson]

On Wed, 18 Oct 2017, FX wrote:

> Parallel builds of libstdc++ on APFS filesystem (with 1 ns granularity) on 
> macOS 10.13 often fail (failure rate for ?make -j2? to ?make -j8? is about 
> 60% from my own builds and results reported by others): 
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81797
> This is reproducible with several versions of GNU make.
>
> Changing libstdc++?s makefile to mark install-headers with .NOTPARALLEL fixes 
> the issue. We've carried that patch in Homebrew (https://brew.sh) for a few 
> months now, and have had no report of build issues since then.
>
> Bootstrapped and regtested on x86_64-apple-darwin17 (as well as other 
> platforms). OK to commit?

Maybe moot now, but .NOTPARALLEL doesn't work the way you imply
in the patch.  From TFM of GNU Make 3.81 and 4: "Any
prerequisites on this target are ignored."

To wit, it seems you'd get the effect, but for *every target* in
this Makefile.  (If I were you and the patch is still proposed,
I'd remove the unused target and add a comment on the intent.)

Assuming the documentation and my reading is correct.

brgds, H-P

Re: [libstdc++, patch] Fix build on APFS file system

[FX]

> Could you test using .PHONY: install-headers instead?
> That target *is* phony, so telling make that seems sensible.

I’ve tried adding it to the existing .PHONY list:

Index: libstdc++-v3/include/Makefile.in
===
--- libstdc++-v3/include/Makefile.in(revision 253855)
+++ libstdc++-v3/include/Makefile.in(working copy)
@@ -1449,6 +1449,7 @@ uninstall-am:
distclean-libtool dvi dvi-am html html-am info info-am install \
install-am install-data install-data-am install-data-local \
install-dvi install-dvi-am install-exec install-exec-am \
+   install-freestanding-headers install-headers \
install-html install-html-am install-info install-info-am \
install-man install-pdf install-pdf-am install-ps \
install-ps-am install-strip installcheck installcheck-am \


But that doesn’t work:

In file included from 
/Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/include/bits/exception_ptr.h:39:0,
 from 
/Users/fx/devel/gcc/trunk/libstdc++-v3/libsupc++/exception:143,
 from 
/Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/include/ios:39,
 from 
/Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/include/istream:38,
 from 
/Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/include/sstream:38,
 from 
/Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/include/complex:45,
 from 
/Users/fx/devel/gcc/ibin/x86_64-apple-darwin17.0.0/i386/libstdc++-v3/include/ccomplex:39,
 from 
/Users/fx/devel/gcc/trunk/libstdc++-v3/include/precompiled/stdc++.h:52:
/Users/fx/devel/gcc/trunk/libstdc++-v3/libsupc++/typeinfo:36:10: fatal error: 
bits/hash_bytes.h: No such file or directory
 #include 
  ^~~

Re: [libstdc++, patch] Fix build on APFS file system

[Jonathan Wakely]

On 18/10/17 16:51 +0200, FX wrote:

Parallel builds of libstdc++ on APFS filesystem (with 1 ns granularity) on 
macOS 10.13 often fail (failure rate for “make -j2” to “make -j8” is about 60% 
from my own builds and results reported by others): 
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81797
This is reproducible with several versions of GNU make.

Changing libstdc++’s makefile to mark install-headers with .NOTPARALLEL fixes 
the issue. We've carried that patch in Homebrew (https://brew.sh) for a few 
months now, and have had no report of build issues since then.

Bootstrapped and regtested on x86_64-apple-darwin17 (as well as other 
platforms). OK to commit?


Could you test using .PHONY: install-headers instead?

That target *is* phony, so telling make that seems sensible.

Presumably the same change is needed for install-freestanding-headers,
since it's very similar.

Re: [libstdc++, patch] Fix build on APFS file system

[FX]

> From supplied info not follow that problem is on gcc build side.
> I can suspect that APFS has problem with direntry intensive
> modification, for example.

As part of Homebrew, we have built 4000+ open source codes on this new 
filesystem, with parallel compilation. Some of them pretty intensive (clang, 
llvm, rust, ghc, and a ton of compilers, libraries, databases, etc.). Many of 
them several times (for testing, etc.). Macports has done the same, probably 
other projects as well.

We have not seen any evidence of a generic issue related to this filesystem. 
Apple is not aware of such an issue either, apparently. Yet, libstdc++ parallel 
builds have a very high failure rate. Other GCC libraries build fine, too.

I do not know how to debug parallel makefiles, otherwise I would do it. I have 
asked help on the GNU Make mailing-list 
(http://lists.gnu.org/archive/html/bug-make/2017-08/msg00034.html), but didn’t 
get any. So I cannot prove it (and fix it), but there is overwhelming evidence 
that the problem is in libstdc++.

FX

Re: [libstdc++, patch] Fix build on APFS file system

[Petr Ovtchenkov]

On Wed, 18 Oct 2017 16:51:37 +0200
FX  wrote:

> Parallel builds of libstdc++ on APFS filesystem (with 1 ns granularity) on 
> macOS 10.13 often fail
> (failure rate for “make -j2” to “make -j8” is about 60% from my own builds 
> and results reported
> by others): https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81797 This is 
> reproducible with several
> versions of GNU make.
> 
> Changing libstdc++’s makefile to mark install-headers with .NOTPARALLEL fixes 
> the issue. We've
> carried that patch in Homebrew (https://brew.sh) for a few months now, and 
> have had no report of
> build issues since then.
> 
> Bootstrapped and regtested on x86_64-apple-darwin17 (as well as other 
> platforms). OK to commit?
> 
> FX
> 
> 

From supplied info not follow that problem is on gcc build side.
I can suspect that APFS has problem with direntry intensive
modification, for example.

--

  - ptr

[libstdc++, patch] Fix build on APFS file system

[FX]

Parallel builds of libstdc++ on APFS filesystem (with 1 ns granularity) on 
macOS 10.13 often fail (failure rate for “make -j2” to “make -j8” is about 60% 
from my own builds and results reported by others): 
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81797
This is reproducible with several versions of GNU make.

Changing libstdc++’s makefile to mark install-headers with .NOTPARALLEL fixes 
the issue. We've carried that patch in Homebrew (https://brew.sh) for a few 
months now, and have had no report of build issues since then.

Bootstrapped and regtested on x86_64-apple-darwin17 (as well as other 
platforms). OK to commit?

FX




parallel.ChangeLog
Description: Binary data


parallel.diff
Description: Binary data