'make -jN install' currently has a few race conditions causing
intermittent failures.
- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=33119
which has been fixed since, AFAICS, but
- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42980
exposes a couple of separate issues:
a) The toplevel targets install-libiberty and install-target-libiberty
may both be executed, and in that case, the install_to_libdir rules from
libiberty/ may race with the same rules from $target/libiberty/ (quite
likely) or with those from a $target$MULTISUBDIR/libiberty (a lot less
likely, only a problem for header installation).
b) libgcc has a make recursion lacking $(MAKE), needed for parallel
submake.
c) Inside libgomp, libssp and libmudflap (and possibly other target
libraries), data (non-exec) install targets may race against the
respective targets of the same name in one of the other multilibs of the
same library.
Issue (a) and (b) are addressed in a followup ((a) concerns also a
src/combined tree); AFAICS only (a) has the chance to cause silent
breakage in the install tree.
There are a couple of ways to fix (c):
0) Forbid parallel install, in documentation or in code. Since install
is mostly IO-bound (at least with the multi-file installations present
in the tree and in Automake 1.11), this is not a big problem. OTOH,
users might have bigger iron still making it worthwhile; automated build
systems might just pass a parallel make from higher up,
1) in Automake: ensure that install-multi is not run concurrently with
install-exec-am or install-data-am or a prerequisite of these,
2) in all multilib directories: ensure that no file is ever installed
twice to the same location from a multilib directory. This would
require going through all multilibbed directories, identify all files
installed in a common location (typically headers), and let only one of
the multilibs actually install them.
My gut feeling is that the latter is a maintenance hassle. So below is
a proposed patch for Automake that implements (1). The patch changes
semantics of multilibbed directories a bit: while formerly,
cd $target/libFOO make install-exec
would have invoked 'make install' in each $target$MULTISUBDIR/libFOO,
now install-exec remains local to the $target/libFOO subtree. This
might have implications for non-GCC packages using multilibs. GCC
itself does not seem to care much about the 'install' vs. 'install-data'
vs. 'install-exec' issues.
As an alternative solution, it would be possible to let each of
install-{data,exec} have multilib recursion semantics and let the
toplevel serialize each wrt. $MULTISUBDIRs. That would require changing
config-ml.in, and haveing install-{data,exec}-multi, and a few ripples
throughout GCC and src. Not implemented yet, but I can do that if it is
preferable.
My question for gcc-patches is whether you agree with this conclusion.
For actual inclusion in GCC, I think it's fine to wait until the next
time a new Automake release is used in GCC.
For now, I have commented out the (nonportable) loop to expose the race
condition in the Automake test suite, as it's currently far too unlikely
thus needs very long to ever trigger on systems I test. Time for
$expensive_tests I guess.
Thanks,
Ralf
(Adding Gary to THANKS not shown)
Serialize multilib install rule, for concurrent same-file installs.
* lib/am/multilib.am: Typo fixes in comments.
(install-exec-am): Do not depend on install-multi.
* automake.in (handle_install): New transform %?MULTI%, for
toplevel multilib dirs.
* lib/am/install.am (install-am) [MULTI]: Invoke install-multi
in a sub-make, for ordering constraints.
* tests/multlib.test: Actually install a few headers and
libraries, so that a loop over `make -j install' has a chance of
eventually failing.
* NEWS, THANKS: Update.
Reported against GCC by Gary Funck in
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42980.
diff --git a/NEWS b/NEWS
index 4f8f8bb..bafe37e 100644
--- a/NEWS
+++ b/NEWS
@@ -53,6 +53,13 @@ Bugs fixed in 1.11a:
- Several scripts as well as the parallel-tests testsuite driver now
exit with the right exit status upon receiving a signal.
+ - The multilib `install' helper rule `install-multi' is now run off of
+the internal `install-am' rule rather than `install-exec-am', to avoid
+race conditions with parallel install due to mutliple versions of the
+same data files being installed by prerequisites of install-data-am.
+This changes implies that `make install-exec install-data' will not
+descend sibling multilib directories any more, you need to use `make
+install'.
New in 1.11:
diff --git a/automake.in b/automake.in
index c0bf146..f7b5e07 100644
--- a/automake.in
+++ b/automake.in
@@ -4663,7 +4663,8 @@ sub handle_install ()
: ''),
'installdirs-local' = (user_phony_rule 'installdirs-local'
? ' installdirs-local' : ''),
-
