Hi,
On 2022-04-06 11:03:37 -0400, Andrew Dunstan wrote:
> On 3/30/22 20:26, Andres Freund wrote:
> > Could you try using dash to invoke configure here, and whether it makes
> > configure faster?
> I got weird failures re libxml/parser.h when I tried with dash. See
> <https://cirrus-ci.com/task/5963254039052288> (It would be nice if we
> could see config.log on failure.)
Since dash won't help us to get the build time down sufficiently, and the
tests don't pass without a separate build tree, I looked at what makes
config/prep_buildtree so slow.
It's largely just bad code. The slowest part are spawning one expr and mkdir
-p for each directory. One 'cmp' for each makefile doesn't help either.
The expr can be replaced with
subdir=${item#$sourcetree}
that's afaics posix syntax ([1]), not bash.
Spawning one mkdir for each directory can be replaced by a single mkdir
invocation with all the directories. On my linux workstation that gets the
time for the first loop down from 1005ms to 38ms, really.
That has the danger of the commandline getting too long. But since we rely on
the final link of the backend to be done in a single command, I don't think
it's making things worse? We could try to use xargs otherwise, iirc that's in
posix as well.
Using parameter substitution in the second loop takes it down from 775ms to
533ms. Not calling cmp when the file doesn't exist cuts it down to 337ms.
I don't know of a way to batch the call to ln. The time with ln replaced with
: is 151ms, fwiw.
On windows that makes prep_buildtree go from 42.4s to 5.8s for me.
Greetings,
Andres Freund
[1] https://pubs.opengroup.org/onlinepubs/009604499/utilities/xcu_chap02.html
diff --git i/config/prep_buildtree w/config/prep_buildtree
index a0eabd3dee2..d26de429e61 100644
--- i/config/prep_buildtree
+++ w/config/prep_buildtree
@@ -26,19 +26,30 @@ buildtree=`cd ${2:-'.'} && pwd`
# If we did, it would interfere with installation of prebuilt docs from
# the source tree, if a VPATH build is done from a distribution tarball.
# See bug #5595.
-for item in `find "$sourcetree" -type d \( \( -name CVS -prune \) -o \( -name .git -prune \) -o -print \) | grep -v "$sourcetree/doc/src/sgml/\+"`; do
- subdir=`expr "$item" : "$sourcetree\(.*\)"`
- if test ! -d "$buildtree/$subdir"; then
- mkdir -p "$buildtree/$subdir" || exit 1
- fi
-done
+
+mkdir -p $(
+ for item in `find "$sourcetree" -type d \( \( -name CVS -prune \) -o \( -name .git -prune \) -o -print \) | grep -v "$sourcetree/doc/src/sgml/\+"`; do
+ subdir=${item#$sourcetree}
+
+ if test ! -d "$buildtree/$subdir"; then
+ echo "$buildtree/$subdir"
+ fi
+ done
+)
for item in `find "$sourcetree" -name Makefile -print -o -name GNUmakefile -print | grep -v "$sourcetree/doc/src/sgml/images/"`; do
- filename=`expr "$item" : "$sourcetree\(.*\)"`
- if test ! -f "${item}.in"; then
- if cmp "$item" "$buildtree/$filename" >/dev/null 2>&1; then : ; else
- ln -fs "$item" "$buildtree/$filename" || exit 1
- fi
+ filename=${item#$sourcetree}
+
+ if test -f "${item}.in"; then
+ continue;
+ fi
+
+ if test ! -e "$buildtree/$filename"; then
+ ln -s "$item" "$buildtree/$filename" || exit 1;
+ elif cmp "$item" "$buildtree/$filename" >/dev/null 2>&1; then
+ : ;
+ else
+ ln -fs "$item" "$buildtree/$filename" || exit 1
fi
done
