bug#12715: coreutils-8.20: bad dependency information with man page generation
Hi Pádraig, Mike. On 10/24/2012 01:48 AM, Pádraig Brady wrote: On 10/23/2012 11:27 PM, Mike Frysinger wrote: if i look at vanilla coreutils-8.20, i see: Makefile.in:man/uname.1: src/uname.c which seems to have originated from man/local.mk, but munged: man/uname.1: src/uname this causes parallel build problems because man/uname.1 generation can get scheduled before src/uname has been linked. easy way to reproduce: ./configure touch src/uname.c make -j ... GENman/uname.1 CC src/hostname.o CC src/uptime.o CC src/kill.o CC src/groups.o GENlib/charset.alias GENlib/ref-add.sed GENlib/ref-del.sed CC lib/set-mode-acl.o CC lib/copy-acl.o CC lib/file-has-acl.o CC lib/allocator.o help2man: can't get '--help' info from man/uname.td/uname CC lib/areadlink.o make[2]: *** [man/uname.1] Error 127 make[2]: *** Waiting for unfinished jobs CC lib/areadlink-with-size.o -mike Ouch. There was a recent commit related to that exact issue which should have fixed this? http://git.sv.gnu.org/gitweb/?p=coreutils.git;a=commit;h=11d6386 The Makefile.in in my local git repo is correct, i.e. man/uname.1: src/uname whereas in the dist tarball it's man/uname.1: src/uname.c I might get time to investigate some time tomorrow. This issue has already cropped up before, and I think I have managed to diagnose it. See: http://lists.gnu.org/archive/html/coreutils/2012-09/msg00110.html There is also a patch available, but it doesn't seem to have encountered much acceptance unfortunately: http://lists.gnu.org/archive/html/coreutils/2012-09/msg00132.html Regards, Stefano
bug#12715: coreutils-8.20: bad dependency information with man page generation
Stefano Lattarini stefano.lattar...@gmail.com writes: There is also a patch available, but it doesn't seem to have encountered much acceptance unfortunately: http://lists.gnu.org/archive/html/coreutils/2012-09/msg00132.html The easiest way to solve that is to use order-only dependencies. Andreas. -- Andreas Schwab, sch...@linux-m68k.org GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 And now for something completely different.
bug#12715: coreutils-8.20: bad dependency information with man page generation
Andreas Schwab wrote: Stefano Lattarini stefano.lattar...@gmail.com writes: There is also a patch available, but it doesn't seem to have encountered much acceptance unfortunately: http://lists.gnu.org/archive/html/coreutils/2012-09/msg00132.html The easiest way to solve that is to use order-only dependencies. Yes, that would be perfect if we could rely on GNU make: http://www.gnu.org/s/make/manual/html_node/Prerequisite-Types.html If you had to require one, which would you rather mandate (rhetorical): Gnu make perl
bug#12715: coreutils-8.20: bad dependency information with man page generation
On 10/24/2012 10:25 AM, Andreas Schwab wrote: Stefano Lattarini stefano.lattar...@gmail.com writes: There is also a patch available, but it doesn't seem to have encountered much acceptance unfortunately: http://lists.gnu.org/archive/html/coreutils/2012-09/msg00132.html The easiest way to solve that is to use order-only dependencies. Which we unfortunately can't do, because they are a GNU make only feature. Regards, Stefano
bug#12715: [PATCH] build: do not require help2man at build-from-tarball time
On 10/24/2012 11:31 AM, Stefano Lattarini wrote: On 10/24/2012 10:54 AM, Jim Meyering wrote: Pádraig Brady wrote: `echo warning; touch $manpage`. Sounds good. I.e., it sounds like we want Stefano's patch, after all. As you recall, a nice side effect is that we will no longer distribute the generated man/*.1 files. Stefano, would you care to rebase your patch? Will do soon. Just give me some time to also re-test it ... Done. The patch is inlined below. Regards, Stefano 88888888888 From 6f35e53d98e6dc0c7843b9e434addf81d901aefa Mon Sep 17 00:00:00 2001 Message-Id: 6f35e53d98e6dc0c7843b9e434addf81d901aefa.1351075054.git.stefano.lattar...@gmail.com From: Stefano Lattarini stefano.lattar...@gmail.com Date: Tue, 11 Sep 2012 20:54:30 +0200 Subject: [PATCH] build: graceful degradation in man pages generation if perl is lacking Fixes coreutils bug#12715. Since commit v8.19-118-g57da212, our 'dist-hook' rule tweaks the distributed Makefile.in so that each man page 'man/foo.1' depends on the corresponding source 'man/foo.c' rather than the corresponding program 'man/foo'. That is done to accommodate inferior systems that, lacking perl, cannot run 'help2man' to regenerate the manpage after its corresponding program has been built. This seems a right and proper graceful degradation, in that the man pages dependencies are still 100% correct in a git checkout, while being laxer but more portable in a distribution tarball. Alas, that is not the case in practice, as it turns out the tweaked Makefile makes the building of man pages unreliable and potentially incorrect! In fact, assume that instead of the correct a dependency: man/ls.1: src/ls we have the laxer one: man/ls.1: src/ls.c and think of what happens if a user modifies, say, 'src/ls.c', and then runs make -j4 to rebuild everything. The make process will see that it has to rebuild the man page 'man/ls.1' (because its prerequisite 'src/ls.c' has changed), but won't see that it has to rebuild 'src/ls' *before* re-running 'help2man' to generate that man page; so, if 'man/ls.1' is rebuilt before 'src/ls' (which can happen with concurrent make), our user will get either a build error (if 'src/ls' did non exist) or, worse, a man page with an up-to-date timestamp but an out-of-date content. And what's even worse in all of this is that this problem will be present also for users who have perl installed: this is not a graceful degradation at all! In our situation, the best and simplest way to implement a graceful degradation it to keep the correct dependencies for man pages (that is, man/ls.1: src/ls), and if perl is not present, just generate dummy man pages reporting that built-time issue and redirecting the user back to either the info documentation or the '--help' output. As a consequence of this change, we also stop distributing man pages, since they would be anyway unconditionally rebuilt * Makefile.am (do-not-require-help2man): Remove. (dist-hook): Don't depend on it. * man/local.mk: Remove an obsolete comment. (EXTRA_DIST): Stop distributing generated man pages. ($(EXTRA_MANS)): This no longer needs to depend on $(all_programs). (MAINTAINERCLEANFILES): $(ALL_MANS) should not be listed here ... (CLEANFILES): ... but here. (.x.1): Instead of warning if perl is missing, but then trying to run 'help2man' unconditionally, simply run ... (run_help2man): ... the command referenced by this new variable, that expands to a proper invocation of 'help2man' if perl is present, and to an invocation of a shell script generating a dummy manpage if it's not. (EXTRA_DIST): Distribute that shell script. * man/dummy-man: Be that shell script --- Makefile.am | 20 + man/dummy-man | 72 +++ man/local.mk | 25 +++-- 3 files changed, 86 insertions(+), 31 deletions(-) create mode 100755 man/dummy-man diff --git a/Makefile.am b/Makefile.am index 0232090..5eaa45b 100644 --- a/Makefile.am +++ b/Makefile.am @@ -93,29 +93,11 @@ BUILT_SOURCES = .version .version: $(AM_V_GEN)echo $(VERSION) $@-t mv $@-t $@ -# In general, we run help2man to build a man page from the binary's --help -# output, but when building from a just-unpacked distribution tarball, we -# must not do that, since help2man uses perl. We don't want to depend on -# perl in that case. In general, the .1 file does indeed depend on the -# binary. I.e., for cat, we have this Makefile dependency: -# man/cat.1: src/cat -# That means that once we build src/cat, we would trigger the .x.1 -# rule which runs help2man. The trick is simply to change the RHS to -# src/cat.c in the $(distdir) that we're about to tar and compress. -# Also handle the three exceptions corresponding to the three binaries -# for which there is no like-named .c file: dir, vdir, ginstall. -.PHONY: do-not-require-help2man -do-not-require-help2man: - perl -pi -e
bug#12715: [PATCH] build: do not require help2man at build-from-tarball time
On 10/24/2012 12:38 PM, Stefano Lattarini wrote: On 10/24/2012 11:31 AM, Stefano Lattarini wrote: On 10/24/2012 10:54 AM, Jim Meyering wrote: Pádraig Brady wrote: `echo warning; touch $manpage`. Sounds good. I.e., it sounds like we want Stefano's patch, after all. As you recall, a nice side effect is that we will no longer distribute the generated man/*.1 files. Stefano, would you care to rebase your patch? Will do soon. Just give me some time to also re-test it ... Done. The patch is inlined below. Yikes, some blunders slipped in. We need at least the following diffs squashed in: -*-*-*- diff --git a/man/dummy-man b/man/dummy-man index cba661c..2e32320 100755 --- a/man/dummy-man +++ b/man/dummy-man @@ -1,5 +1,5 @@ #!/bin/sh -# Poor place holder for help2man invocation on systems lacking perl; +# Poor placeholder for help2man invocation on systems lacking perl; # it generates a dummy manpage stating that as proper manpage could # not be generated, and redirecting the user back to either the info # documentation or the '--help' output. @@ -21,7 +21,7 @@ output= source=GNU coreutils while test $# -gt 0; do case $1 in -# Help2man options we recognize handle. +# Help2man options we recognize and handle. --output=*) output=`expr x$1 : x'--output=\(.*\)'`;; --output) shift; output=$1;; --source=*) source=`expr x$1 : x'--source=\(.*\)'`;; @@ -40,7 +40,7 @@ done test $# -gt 0 || fatal_ missing argument test $# -le 1 || fatal_ too many non-option arguments -baseout=`basename $output` +baseout=`basename_ $output` sed 's/^/WARNING: /' 2 END Cannot create proper man page '$baseout' since perl is missing or inadequate on this system. Will create a stub man page instead. -*-*-*- Here is the updated patch. Sorry for the noise, Stefano 88888888888 From f61dcb763975c1aab299fa9678ea180d70db6acf Mon Sep 17 00:00:00 2001 Message-Id: f61dcb763975c1aab299fa9678ea180d70db6acf.1351075572.git.stefano.lattar...@gmail.com From: Stefano Lattarini stefano.lattar...@gmail.com Date: Tue, 11 Sep 2012 20:54:30 +0200 Subject: [PATCH] build: graceful degradation in man pages generation if perl is lacking Fixes coreutils bug#12715. Since commit v8.19-118-g57da212, our 'dist-hook' rule tweaks the distributed Makefile.in so that each man page 'man/foo.1' depends on the corresponding source 'man/foo.c' rather than the corresponding program 'man/foo'. That is done to accommodate inferior systems that, lacking perl, cannot run 'help2man' to regenerate the manpage after its corresponding program has been built. This seems a right and proper graceful degradation, in that the man pages dependencies are still 100% correct in a git checkout, while being laxer but more portable in a distribution tarball. Alas, that is not the case in practice, as it turns out the tweaked Makefile makes the building of man pages unreliable and potentially incorrect! In fact, assume that instead of the correct a dependency: man/ls.1: src/ls we have the laxer one: man/ls.1: src/ls.c and think of what happens if a user modifies, say, 'src/ls.c', and then runs make -j4 to rebuild everything. The make process will see that it has to rebuild the man page 'man/ls.1' (because its prerequisite 'src/ls.c' has changed), but won't see that it has to rebuild 'src/ls' *before* re-running 'help2man' to generate that man page; so, if 'man/ls.1' is rebuilt before 'src/ls' (which can happen with concurrent make), our user will get either a build error (if 'src/ls' did non exist) or, worse, a man page with an up-to-date timestamp but an out-of-date content. And what's even worse in all of this is that this problem will be present also for users who have perl installed: this is not a graceful degradation at all! In our situation, the best and simplest way to implement a graceful degradation it to keep the correct dependencies for man pages (that is, man/ls.1: src/ls), and if perl is not present, just generate dummy man pages reporting that built-time issue and redirecting the user back to either the info documentation or the '--help' output. As a consequence of this change, we also stop distributing man pages, since they would be anyway unconditionally rebuilt * Makefile.am (do-not-require-help2man): Remove. (dist-hook): Don't depend on it. * man/local.mk: Remove an obsolete comment. (EXTRA_DIST): Stop distributing generated man pages. ($(EXTRA_MANS)): This no longer needs to depend on $(all_programs). (MAINTAINERCLEANFILES): $(ALL_MANS) should not be listed here ... (CLEANFILES): ... but here. (.x.1): Instead of warning if perl is missing, but then trying to run 'help2man' unconditionally, simply run ... (run_help2man): ... the command referenced by this new variable, that expands to a proper invocation of 'help2man' if perl is present, and to an invocation of a shell script generating a dummy manpage if it's not. (EXTRA_DIST): Distribute that
bug#12715: [PATCH] build: do not require help2man at build-from-tarball time
g@free.fr wrote: - Mail original - De: Jim Meyering j...@meyering.net À: Stefano Lattarini stefano.lattar...@gmail.com Cc: Pádraig Brady p...@draigbrady.com, Mike Frysinger vap...@gentoo.org, coreut...@gnu.org, 12...@debbugs.gnu.org Envoyé: Mercredi 24 Octobre 2012 15:07:18 Objet: bug#12715: [PATCH] build: do not require help2man at build-from-tarball time Stefano Lattarini wrote: Here is the updated patch. Sorry for the noise, ... Subject: [PATCH] build: graceful degradation in man pages generation if perl is lacking Does anyone know of a well-known environment/distro in which coreutils is built without perl? I.e., if virtually no one builds coreutils without perl, then any attempt to coddle those users is wasted complexity. Speaking for what I know, this is not an issue with 'linux from scratch' based systems. - perl is an host requirement (coreutils is build a first time on toolchain stage), - perl is build on toolchain stage before final coreutils is build. Any toolchain that really cross-compile often avoid perl as notoriously hard to cross-compile. Often busybox is used instead of coreutils. Openwrt for example does that. Openwrt could optionally build coreutils. The build script use 8.16 and patch to not build the doc. Nice. That's reassuring. Thanks for the info.
bug#12714: [PATCH] build: do not redefine _FORTIFY_SOURCE
Mike Frysinger wrote: On Tuesday 23 October 2012 15:26:14 Jim Meyering wrote: Mike Frysinger wrote: If the active compiler or flags have already defined _FORTIFY_SOURCE, don't go overriding that. Otherwise we get a lot of spew about the flag being redefined. * configure.ac (FORTIFY_SOURCE): Check if _FORTIFY_SOURCE is defined. --- ... Thanks. Please adjust the log (or just tell us and we'll adjust it) so that we know which compiler motivates this change. Ubuntu will enable _FORTIFY_SOURCE by default. not sure about Debian. Gentoo has been doing it for a while. Thanks. I've adjusted the log. Ok with you? From 9517680ed3d4d7baff8a842a052ad405f9e9049f Mon Sep 17 00:00:00 2001 From: Mike Frysinger vap...@gentoo.org Date: Tue, 23 Oct 2012 15:20:07 -0400 Subject: [PATCH] build: do not redefine _FORTIFY_SOURCE When _FORTIFY_SOURCE is defined by the compiler or via flags, as it is on Gentoo, do not override it. Otherwise we get many redefinition warnings. * configure.ac (FORTIFY_SOURCE): Check if _FORTIFY_SOURCE is defined. This addresses http://bugs.gnu.org/12714 --- configure.ac | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/configure.ac b/configure.ac index e5e05be..6504144 100644 --- a/configure.ac +++ b/configure.ac @@ -170,7 +170,7 @@ if test $gl_gcc_warnings = yes; then AH_VERBATIM([FORTIFY_SOURCE], [/* Enable compile-time and run-time bounds-checking, and some warnings, without upsetting glibc 2.15+. */ - #if defined __OPTIMIZE__ __OPTIMIZE__ + #if !defined _FORTIFY_SOURCE defined __OPTIMIZE__ __OPTIMIZE__ # define _FORTIFY_SOURCE 2 #endif ]) -- 1.8.0
bug#12714: [PATCH] build: do not redefine _FORTIFY_SOURCE
On Wednesday 24 October 2012 12:09:27 Jim Meyering wrote: Mike Frysinger wrote: On Tuesday 23 October 2012 15:26:14 Jim Meyering wrote: Mike Frysinger wrote: If the active compiler or flags have already defined _FORTIFY_SOURCE, don't go overriding that. Otherwise we get a lot of spew about the flag being redefined. * configure.ac (FORTIFY_SOURCE): Check if _FORTIFY_SOURCE is defined. --- ... Thanks. Please adjust the log (or just tell us and we'll adjust it) so that we know which compiler motivates this change. Ubuntu will enable _FORTIFY_SOURCE by default. not sure about Debian. Gentoo has been doing it for a while. Thanks. I've adjusted the log. Ok with you? looks fine, thanks -mike signature.asc Description: This is a digitally signed message part.
