Re: 1.16.90 regression: configure now takes 7 seconds to start

[Nick Bowler]

On 2024-06-07 19:26, Jacob Bachmeyer wrote:
> Bruno Haible wrote:
>> [I'm writing to automake@gnu.org because bug-autom...@gnu.org appears
>> to be equivalent to /dev/null: no echo in
>> https://lists.gnu.org/archive/html/bug-automake/2024-06/threads.html
>> nor in https://debbugs.gnu.org/cgi/pkgreport.cgi?package=automake,
>> even after several hours.]
>>
>> In configure scripts generated by Autoconf 2.72 and Automake 1.16.90,
>> one of the early tests checking filesystem timestamp resolution...
>> takes 7 seconds! Seen e.g. on NetBSD 10.0.
[...]
> The problem with the proposed patch is that it tries to read a
> filesystem name instead of testing for the feature.  This would not be
> portable to new systems that use a different name for their FAT
> filesystem driver.

Maybe this is a silly question, but, is there a reason why this test
needs to be performed in every single package that uses Automake?

I was under the impression that the purpose of this test was merely
to speed up running Automake's own test suite.

Cheers,
  Nick

bug#70410: Fwd: [PATCH] gotools: Workaround non-reproduceability of automake

[Nick Bowler]

On 2024-04-17 11:55, Karl Berry wrote:
> so whether it is
> @NATIVE_FALSE@install-exec-local:
> @NATIVE_FALSE@uninstall-local:
> or
> @NATIVE_FALSE@uninstall-local:
> @NATIVE_FALSE@install-exec-local:
> depends on some hash table traversal or what.
> 
> Thanks for the report. Any chance of a Makefile.am that can reproduce
> the problem? I wonder if it's related to the @NATIVE_FALSE@ somehow.
> 
> Looking at automake.in, it's not obvious to me where a list is failed to
> be sorted. Those -local targets aren't generated by automake itself, so
> far as I can see. --thanks, karl.

Code like the above is generated by Automake when -hook or -local
targets are defined within the "true" side of an Automake conditional,
but not in the "false" side, for example:

  % cat >configure.ac <<'EOF'
AC_INIT([test], [0])
AM_INIT_AUTOMAKE([foreign])
AM_CONDITIONAL([NATIVE], [:])
AC_CONFIG_FILES([Makefile])
AC_OUTPUT
EOF
  % cat >Makefile.am <<'EOF'
if NATIVE
install-data-local: ;
install-exec-local: ;
uninstall-local: ;
dist-hook: ;
all-local: ;
endif
EOF
  % autoreconf -is
  % grep NATIVE_FALSE Makefile.in
@NATIVE_FALSE@dist-hook:
@NATIVE_FALSE@all-local:
@NATIVE_FALSE@install-data-local:
@NATIVE_FALSE@install-exec-local:
@NATIVE_FALSE@uninstall-local:

With an older Automake (1.12.6), the order of these generated targets is
easily seen to be inconsistent.  But with a current version of Automake
(1.16.5) the order seems to be always the same.  So I wonder if this is
reporting an issue that was long ago already fixed in Automake...

Cheers,
  Nick

Re: Getting long SOURCES lines with subdirs shorter

[Nick Bowler]

On 2023-12-01 15:37, Jan Engelhardt wrote:
> On Friday 2023-12-01 21:13, Mike Frysinger wrote:
>> On 17 Jul 2023 16:51, Karl Berry wrote:
>>> Hi Jan,
>>>
>>> Current automake likely won't have anything in store already,
>>>
>>> Not that I know of.
>>>
>>> a_SOURCES = $(addprefix aprog/,main.c foo.c bar.c baz.c)
>>>
>>> I've often wanted this myself. I'd certainly welcome a patch for it.
>>>
>>> Please work from automake trunk. None of the various branches are kept
>>> to date. (Sad but that's the reality.)
>>
>> prob stating the obvious, but $(addprefix) is a GNUism, so if we wanted to
>> use it, it'd required feature probing at configure time, and that always
>> complicates things :(
> 
> No-no, the idea was to make $(addprefix) an automakeism that is resolved 
> before
> GNU make (or any other make) is ever invoked.

I suggest inventing a new syntax if this approach is taken, one that
doesn't overload real-world make syntax, since some people do use
Automake with GNU-make-specific rules and whatnot.  We already have
things like %reldir% which are expanded by Automake so maybe using
percent signs as a marker for "things expanded by automake" would
be a good starting point for this.

I do sometimes wish Automake had more built-in macro facilities.
One can do things like generate includeable snippets or preprocess
Makefile.am with, say, m4, but that adds a bunch of additional
complexity which is not always worthwhile.

Cheers,
  Nick

bug#67539: GNU Automake 1.16.5 FAILS one test objc-megademo

[Nick Bowler]

On 2023-11-30 21:46, Karl Berry wrote:

Hi Dennis,

 libtool: compile: unable to infer tagged configuration

Thanks for the report.

As you surmise, apparently this needs to be reported to
libtool. (Although afaik libtool is currently unmaintained, so I don't
know when or if anything will get fixed.) At least, I have no idea what
to do about, or even work around, that error :(.


This could still be (partially) an Automake issue, since the problem 
appears partially related to how Automake is calling libtool from the 
build rules (it is missing the --tag option), and I think those are 
coming from Automake.


However it looks like there is no tag defined for Objective C[1] 
(presumably it would be --tag=OBJC).  Adding this option does appear to 
make things "work" in the sense that libtool just complains about the 
unknown tag but then proceeds to actually do stuff, rather than exiting 
outright with a fatal error.


Interestingly the libtool manual also says "If [libtool] can't infer a 
tag, then it defaults to the configuration for the C language", which is 
clearly not the case (it seems what actually happens is that if libtool 
can't infer a tag then it exits with a fatal error).  So I wonder if

this is actually a regression in libtool.

[1] https://www.gnu.org/software/libtool/manual/libtool.html#Tags

Cheers,
  Nick

[bug#67516] [PATCH] aclocal: handle both # and dnl for serial number lines

[Nick Bowler]

On 2023-11-28 16:06, Karl Berry wrote:
> Hi Ross - you sent a change to automake-patches back in February 2017.
> Sorry for the absurdly delayed reply.
> 
> https://lists.gnu.org/archive/html/automake-patches/2017-02/msg1.html
> 
> Instead of only accepting comment lines that start with # for serial
> lines, also accept dnl.
> ...
> -my $serial_line_rx = '^#\s*serial\s+(\S*)';
> +my $serial_line_rx = '^(?:#|dnl)\s*serial\s+(\S*)';
> 
> While there's no technical problem with the patch, I'm not sure
> supporting dnl is desirable. It seems better for the serial line to
> always end up in whatever file is generated, for the sake of debugging
> and documentation, instead of being eliminated at m4-time. If there was
> a specific case where it was better to eliminate it, please describe ...

This shouldn't make any difference.  In all "normal" uses of aclocal
everything in these files is going to be diverted to KILL anyway so
neither form should result in any output text going into configure.

And with respect to how aclocal imports macro files into a project
(either as separate files w/ --install, or directly into aclocal.m4),
it just copies files verbatim so it doesn't matter what syntax is used.

But by the same token there would seem to be no technical reason to
prefer dnl over #...

Cheers,
  Nick

Re: Detect --disable-dependency-tracking in Makefile.am

[Nick Bowler]

On 2023-09-30, Nick Bowler  wrote:
> Two suggestions, one relying on Automake internals and one not:
>
> Suggestion 1)
> internal ... Automake conditional called AMDEP.
[...]
> Suggestion 2)
[...]
> AM_CONDITIONAL([NO_DEPS], [test x"$enable_dependency_tracking" = x"no"])

> Note that these approaches are different in the case where dependency
> tracking is disabled because it is not supported by the user's tools,
> rather than by explicit request.  This may or may not matter for your
> use case.

Nevermind this last point, these suggestions are functionally identical;
the AMDEP conditional also only handles the "explicitly disabled" case
and is defined almost exactly the same way as NO_DEPS above (test is
just reversed).

So there is no reason to use #1 except to save a line in configure.ac.

Cheers,
  Nick

Re: Detect --disable-dependency-tracking in Makefile.am

[Nick Bowler]

On 2023-09-29, Dave Hart  wrote:
> I'm guessing someone has trod this ground before.  I'd appreciate
> pointers to examples of how others have detected
> --disable-dependency-tracking to change their build behavior.

Two suggestions, one relying on Automake internals and one not:

Suggestion 1) It is technically undocumented, but longstanding Automake
behaviour is that dependency tracking is internally implemented using an
Automake conditional called AMDEP.  So you can literally just write in
Makefile.am:

  if AMDEP
  # stuff to do when dependency tracking is available
  else
  # stuff to do when dependency tracking is unavailable or disabled
  endif

Suggestion 2) All explicit --enable-foo/--disable-foo arguments to
a configure script are available in shell variables; in the case of
--disable-dependency-tracking you can do something like this in
configure.ac:

  AM_CONDITIONAL([NO_DEPS], [test x"$enable_dependency_tracking" = x"no"])

then in Makefile.am:

  if NO_DEPS
  # stuff to do when dependency tracking is disabled
  else
  # stuff to do otherwise
  endif

Note that these approaches are different in the case where dependency
tracking is disabled because it is not supported by the user's tools,
rather than by explicit request.  This may or may not matter for your
use case.

Hope that helps,
  Nick

[bug#65600] [PATCH] Allow parameters AM_PROG_LEX

[Nick Bowler]

On 31/08/2023, Karl Berry  wrote:
> Hi Bogdan,
>
> In reference to:
> https://lists.gnu.org/archive/html/automake/2023-07/msg7.html.
>
> Thanks much!
>
> Since it's Autoconf 2.70 that started using the parameter, I've
> bumped the required value.
>
> I don't think we should require the brand-new ac 2.70 just for this
> minor feature.  And I think it should be feasible not to: if the
> autoconf version is >= 2.70, your new code runs; if <2.70, just ignore
> any given argument.  That way people can use the new automake macro with
> an older autoconf.

One way to maintain compatibility while ensuring the new options actually
are supported would be to just add the AC_PREREQ([2.70]) only in the cases
which actually trigger different behaviour in new Autoconf (i.e., the ones
that match yywrap and noyywrap options).  Keep the old prereq unchanged
outside of the case.

That way, the minimum autoconf version requirement is only triggered
for autoconf inputs that actually attempt to use the new feature (by
explicitly passing yywrap or noyywrap options to AM_PROG_LEX).

To implement a "fall back to old behaviour with old autoconf, even if
the user explicitly selected the new behaviour", then you can just not
touch AC_PREREQ at all.  There is no need for any version tests: old
versions of autoconf simply ignore any supplied arguments to
AC_PROG_LEX and always use the old behaviour.

Finally, one way to check whether AC_PROG_LEX accepts an argument is to
just use m4 to look for argument references in its definition, something
like:

  m4_bmatch(m4_defn([AC_PROG_LEX]), [[$][1@*]],
[m4_errprintn([new lex macro])],
[m4_errprintn([old lex macro])])

then you don't need to test autoconf version numbers at all.

Cheers,
  Nick

bug#64743: Speed up GNU make's internal processing

[Nick Bowler]

On 20/07/2023, Bruno Haible  wrote:
> Karl Berry wrote:
>> I just hope those weird-looking %:: rules do not cause trouble with
>> other makes. I guess we'll find out.
>
> I tested the default 'make' of various OSes, before submitting the patch.
> Whether some other, rarely-used 'make' implementation has problems with it,
> we'll find out.

FWIW I tried the makes on FreeBSD 9, HP-UX 11 and ULTRIX 4.5 and there was
no obvious negative effect with the snippet shown in the original post.

I expect all the non-GNU implementations are interpreting these lines
as ordinary target rules for a funnily-named files with funnily-named
prerequisites.  So it's not quite right to say it has "no effect", but
I think it shouldn't cause any particular problem for packages using
portable file names.

Cheers,
  Nick

bug#19614: Split packaging invocation to catch errors

[Nick Bowler]

On 2023-07-17, Karl Berry  wrote:
> Hi Dimitrios, Bogdan - back on this bug from 2015 (sorry):
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=19614
>
> Bogdan sent a patch that splits the tar and compress into separate
> invocations.  It seems basically good to me, but the dist-formats test
> fails because it builds multiple archive formats (.tar.gz, .tar.bz2,
> etc.) in parallel, and so removing $(distdir).tar (and .err) files are
> subject to a race condition.

>  dist-gzip: distdir
> - tardir=$(distdir) && $(am__tar) | eval GZIP= gzip $(GZIP_ENV) -c
>>$(distdir).tar.gz
> + tardir=$(distdir) && $(am__tar) > $(distdir).tar 2>$(distdir).tarerr
>
> So my question is, will it suffice in this limited case to just put $$
> into the filenames? It seems like it should be ok to me, but I'm not
> sure I have enough imagination to know why that would fail. I can't see
> figuring out how to run mktemp here.

With the tar file generation as a separate command, it should be
straightforward to avoid this problem by just moving the tar generation
and error checking commands into a separate rule.  Then changing all the
various dist-xyz commands to depend on that instead of distdir.  Example:

  $(distdir).tar: distdir
commands to tar it

  dist-gzip: $(distdir).tar
commands to gzip it

and so on.  Then there should be no race with parallel "make dist" as
the tar file will only be generated once.

Additional context/suggestion (not directly relevant to this bug report):

Note that "make dist" (parallel or othewrise) with multiple compressors
enabled only works because "make dist" itself does a recursive make
invocation and overrides am__post_remove_distdir.  Otherwise the cleanup
commands would conflict too.  It's quite hacky, and as a result it
currently doesn't work to explicitly call multiple dist targets in a
single make invocation, e.g.,

  make dist-zip dist-gzip # fails

Using a separate rule to generate the tar file should let us fix this
problem pretty easily too, should just be a matter of:

  - do not add "rm -f $(distdir).tar" to am__post_remove_distdir
  - arrange for make dist and make clean to delete $(distdir).tar
  - add the rule .INTERMEDIATE: $(distdir).tar

The last item instructs GNU make to delete the temporary tar file in
normal cases once all the rules that depend on it have been run.

Explicit deletion in "make dist" and "make clean" is then mostly for the
benefit of other implementations.  On other makes, the tar file will be
left behind in some cases (e.g., if the user runs make dist-zip).  This
makes a change from current behaviour but probably not a real problem.

Cheers,
  Nick

Re: if vs. ifdef in Makefile.am

[Nick Bowler]

On 2023-03-01, Jan Engelhardt  wrote:
> You can utilize the same mechanism behind automake's `make V=1`:
>
> NDEBUG = 0
> my_CPPFLAGS_0 =
> my_CPPFLAGS_1 = -NDEBUG
> my_CFLAGS_0 = -O3
> my_CFLAGS_1 =
> AM_CPPFLAGS = ${my_CPPFLAGS_${NDEBUG}}
> AM_CFLAGS = ${my_CFLAGS_${NDEBUG}}

This syntax is not standard or portable; Automake's silent-rules stuff
is backed by configure tests to ensure the syntax is supported by make.

That being said, with most make implementations it will be "fine", in
that make implementations tend to accept almost any line noise between
the brackets of a variable expansion without complaint, simply expanding
it to the empty string.  Even the original make from V7 UNIX works this
way.  So the flags will be missing but the build should still complete
as these flags are presumably not critical.

One exception is HP-UX, which doesn't match the brackets properly so
you will end up with a stray } in CPPFLAGS and CFLAGS, most likely
causing the build to fail.  You can work around this particular problem
by exploting make's multiple equivalent forms for variable expansion,
for example $(my_CPPFLAGS_${NDEBUG}) should be quite "portable" in
the sense that it will either work or you get the empty string.

But... Autoconf already has the AC_HEADER_ASSERT macro which adds a
--disable-assert configure option that sets NDEBUG... why not just use
that?  The other flags (-O3, -fsanitize=address) will need to be backed
by configure tests anyway as not all C compilers support these options,
so why not just add an --enable-debug or similar to control all this?

Cheers,
  Nick

Re: rhel8 test failure confirmation?

[Nick Bowler]

On 2023-03-01, Karl Berry  wrote:
> Does anyone have access to an RHEL 8-based machine? Alma Linux, Rocky
> Linux, original RHEL, or even (sort of) CentOS 8? It would be nice if
> someone could run a make check there (from automake dev).
>   git clone -q git://git.savannah.gnu.org/automake.git
>   cd automake
>   ./bootstrap
>   ./configure && make >
>   make -j8 VERBOSE=1 check keep_testdirs=yes >
> (choose whatever -j value you like)

  % cat /etc/redhat-release
  Red Hat Enterprise Linux release 8.0 (Ootpa)

  % uname -r
  4.18.0-305.10.2.el8_4.x86_64

I ran it twice, the first time out of my user directory (NFS), with no
failures, the second out of /tmp (XFS), with one failure:

  FAIL: t/remake-aclocal-version-mismatch.sh
  [...]
  
  Testsuite summary for GNU Automake 1.16i
  
  # TOTAL: 2935
  # PASS:  2808
  # SKIP:  87
  # XFAIL: 39
  # FAIL:  1
  # XPASS: 0
  # ERROR: 0
  
  See ./test-suite.log
  Please report to bug-autom...@gnu.org
  

Cheers,
  Nick

bug#61278: Automake warns about LDFLAGS for defined library

[Nick Bowler]

On 2023-02-04, Reuben Thomas via Bug reports for Automake
 wrote:
> When automake is run, it warns:
>
> liba2ps/Makefile.am:63: warning: variable 'libnowarnings_a_LDFLAGS' is
> defined but no program or
> liba2ps/Makefile.am:63: library has 'libnowarnings_a' as canonical name
> (possible typo)
>
> I don't understand this, because libnowarnings.a is defined as the
> canonical name of a library.

Indeed, this message does not seem very accurate.

What Automake is trying to tell you is that LDFLAGS is meaningless
on a static library.  This message could definitely be improved!

> If in the first Makefile.am I change the convenience library
> "libnowarnings.a" to a libtool library "libnowarnings.la", and add it
> to noinst_LTLIBRARIES, and rename the other variables accordingly,
> then again I get no warnings.

Unlike with a static library, LDFLAGS is used with libtool libraries, so
it is good that there is no warning.

Cheers,
  Nick

[bug#60807] [PATCH v2] tests: reuse am_cv_filesystem_timestamp_resolution

[Nick Bowler]

On 14/01/2023, Zack Weinberg  wrote:
> On Sat, Jan 14, 2023, at 7:18 PM, Mike Frysinger wrote:
>> Rather than assume such coarse delays, re-use existing logic for
>> probing the current filesystem resolution.  This speeds up the
>> testsuite significantly.>
[...]
> No objection to this patch in itself, but I want to make sure you're
> aware that the "existing logic for probing the current filesystem
> resolution" has a bug where, if you start running the script at just
> the wrong time, it will erroneously detect a finer timestamp resolution
> than the system actually supports.   For instance, if we can sleep for
> 0.1 second, the filesystem's timestamp resolution is 2s, and the sleep
> loop happens to start executing at 0h00m59.9s, then it'll tick over to
> 0h01m00.0s and conftest.file.a and conftest.file.b will have distinct
> timestamps.  This happens to me quite reliably: whenever I try to
> run the Automake test suite inside AFS, I'll get a couple of spurious
> test failures because of this bug.

I don't know how exactly this delay is used in the test suite, but if
the goal is just to "reliably wait until newly-created files have
different timestamps from prior-created files, without waiting
substantially longer than required", I've had good results with a
pattern like this in my own test suites (instead of sleeping for a
predetermined amount of time):

  cat >uptodate.mk <<'EOF'
  old: new
false
EOF
  touch old; touch new
  while make -f uptodate.mk >/dev/null 2>&1; do
touch new
  done

Cheers,
  Nick

bug#55025: Automake should allow one to enable POSIX make behavior

[Nick Bowler]

On 14/01/2023, Mike Frysinger  wrote:
[...]
>> I tried several other implementations and they follow the POSIX
>> behaviour by default, adding -e only when errors are not suppressed.
>
> this is exactly my point.  if i'm developing a project with automake
> and i'm using gnu make, i can easily produce bad code that is not
> portable.  but the insidious part is that it doesn't fail for me, it
> fails for my users who are not using gnu make, but using a
> POSIX-compliant implementation that respects the `set -e` behavior.

But as I demonstrated, adding .POSIX doesn't actually make GNU make
work the same as these "POSIX-by-default" implementations.  It just
gives it a different, GNU-specific behaviour, since GNU make apparently
enables -e regardless of whether or not errors are suppressed.

Does changing the behaviour in this way really improve anything for a
maintainer that is only testing with GNU make?

Without putting .POSIX in the Makefile, GNU is at least consistent with
NetBSD here (which never uses -e) and probably also other modern BSD
derivatives, as I think many have these days adopted NetBSD make.

Cheers,
  Nick

bug#55025: Automake should allow one to enable POSIX make behavior

[Nick Bowler]

On 2023-01-13, Mike Frysinger  wrote:
> On 13 Jan 2023 16:01, Karl Berry wrote:
>> I am doubtful about blithely defining .POSIX unconditionally. I feel
>> sure that will break existing Makefiles. I don't think we should do that.
>>
>> Detecting .POSIX in an existing Makefile.am and moving it to the front
>> sounds desirable, since that is clearly what the developer intended.
>>
>> Another (not mutually exclusive) possibility is to add an Automake
>> option (say, "make-posix") that outputs the .POSIX line. --thanks, karl.
>
> i'd be amenable to a `no-make-posix` option, but imo forcing people to
> opt-in to the right behavior is the wrong mental model.

Does adding .POSIX: to a Makefile actually solve any real-world
portability problem with Automake generated makefiles, or is all of
this just hypothetical?

Because we shouldn't go changing default behaviour to solve imaginary
problems ...

> i'm also a bit skeptical of the idea that we're breaking makefiles.

... and adding this to a Makefile really does change the behaviour
significantly on at least one popular make implementation (GNU make).

I imagine a lot of people who are unfamiliar with traditional UNIX
implementations would be very surprised to see gmake suddenly start
running their rules using /bin/sh -e, which is probably the most
obvious effect that the .POSIX special target has on this particular
implementation.

And it doesn't even seem to work to get GNU make to follow the POSIX
behaviour, as POSIX says -e is NOT used whenever errors are being
suppressed (that is, when running commands that start with a -, or
if the .IGNORE: special target is present in the makefile, or if
make is run with the -i option).  GNU make appears to just always
run the shell with -e if you include the .POSIX special target.

Even then, portable makefiles can't rely on the shell being run
with -e even normally, as NetBSD make does not do this.

I tried several other implementations and they follow the POSIX
behaviour by default, adding -e only when errors are not suppressed.

> if an implementation is POSIX compliant, then it already behaves as
> POSIX defines make.  as it stands now, we, and our users, are
> generating files that target an undefined standard that no one agrees
> on, cannot be tested or asserted, and maybe they happen to work on the
> developer's desktop, but not on random users of their own.  by
> defining .POSIX, we turn the state from "usually works, but sometimes
> fails" to "either works or fails, but it's the same everywhere".

Unfortunately, reality is messy.  GNU's behaviour means that adding
.POSIX: actually means that with this specific behaviour (whether or
not the shell is run with "-e"), you now have to consider three
possibilities instead of just two.

Consider these two Makefiles, identical except for the presence of
the .POSIX special target:

  % cat >noposix.mk <<'EOF'
  craziness:
@-false; echo hello
@false; echo hello
EOF

  % cat >posix.mk <<'EOF'
  .POSIX:
  craziness:
@-false; echo hello
@false; echo hello
EOF

GNU:

  % gmake -f noposix.mk
  hello
  hello

  % gmake -f posix.mk
  gmake: [posix.mk:3: craziness] Error 1 (ignored)
  gmake: *** [posix.mk:4: craziness] Error 1

NetBSD:

  % make -f noposix.mk
  hello
  hello

  % make -f posix.mk
  hello
  hello

HP-UX 11:

  (this is the actual POSIX-specified behaviour which I also observed on
  multiple other systems):

  % make -f noposix.mk
  hello
  *** Error exit code 1

  Stop.

  % make -f posix.mk
  hello
  *** Error exit code 1

Cheers,
  Nick

bug#60607: Making dvi target do nothing

[Nick Bowler]

On 2023-01-06, Reuben Thomas via Bug reports for Automake
 wrote:
> I'm using automake 1.16.5. The advice about how to disable the "dvi"
> target doesn't seem to work.
>
> In the manual it says:
>
>To make ‘dvi’ into a do-nothing target, see the example for
> ‘EMPTY_AUTOMAKE_TARGETS’ in *note Third-Party Makefiles::.
>
> If I have:
>
> EMPTY_AUTOMAKE_TARGETS = dvi
> .PHONY: $(EMPTY_AUTOMAKE_TARGETS)
> $(EMPTY_AUTOMAKE_TARGETS):
>
> in my Makefile.am, then "TEXI2DVI" is still run by "make distcheck".

This example in the manual is talking about writing a custom
Makefile, *without* using Automake, that you want to recurse
into using Automake's SUBDIRS feature.

> If I instead have:
>
> dvi:
>
> then it is not.

This is right way to override the dvi target that would
otherwise be generated by Automake.

Cheers,
  Nick

[bug#60535] [PATCH] depend2: switch echo|sed to automatic vars

[Nick Bowler]

On 2023-01-04, Mike Frysinger  wrote:
[...]
> dmake is one implementation that fails, and your suggestion doesn't work
> :/.
> $ dmake foo/bar.o
> dmake:  Error: -- Incomplete macro expression [)' b='$(at_f:.o=)'; test
> x"$$a.o" = x"$(@F)" || a=$$b;\
> echo $$a]

It might also be worth considering that all of these issues go away
if we can arrange the rules to use plain $(*F) instead of doing suffix
substitution on $(@F).

I think this code is used in both suffix and target rules, and $(*F) is
only valid in the former, but perhaps Automake can just substitute the
appropriate literal string into generated target rules?

I'm not familiar with how Automake actually uses this code to generate
rules, so maybe this is not a workable idea.

Cheers,
  Nick

[bug#60535] [PATCH] depend2: switch echo|sed to automatic vars

[Nick Bowler]

On 2023-01-04, Mike Frysinger  wrote:
> On 04 Jan 2023 21:10, Nick Bowler wrote:
[...]
>> maybe something like:
>>
>>   % cat >Makefile <<'EOF'
>> at_f = $(@F)
>> foo/bar.o:
>>  a='$(@F:.o=)' b='$(at_f:.o=)'; test x"$$a.o" = x"$(@F)" || a=$$b;\
>>echo $$a
>> EOF
>
> this is interesting.  we actually have some uses of this already in
> the tree.  no one seems to have complained though afaict.

Note that the NetBSD bug affects only substitutions on file/directory
variants like $(@F:a=b), there is no parse problem (that I know of)
with substitutions on the normal forms like $(@:a=b).

> automake-1.13+ has:
> lib/am/subdirs.am:AM_RECURSIVE_TARGETS +=
> $(am__recursive_targets:-recursive=)
>
> automake-1.9+ has:
> lib/am/texibuild.am:  for f in $@ $@-[0-9] $@-[0-9][0-9]
> $(@:.info=).i[0-9] $(@:.info=).i[0-9][0-9]; do \
>
> dmake is one implementation that fails, and your suggestion doesn't work
> :/.
> $ dmake foo/bar.o
> dmake:  Error: -- Incomplete macro expression [)' b='$(at_f:.o=)'; test
> x"$$a.o" = x"$(@F)" || a=$$b;\
> echo $$a]

Huh, I somehow have never noticed this problem with dmake.  So the
right-hand side cannot be empty on this one.

> how portable is $() ?  that seems to work in GNU make & dmake.

To expand to the empty string?  I expect that's probably fine, but not
sure about using a make variable on the RHS of a suffix substitution.
At least, this does not appear to work on ULTRIX (the substituted string
is output literally, not expanded at all).

I think my trick can be salvaged to work on all of these crazy
implementations by substituting no-op shell syntax instead:

  % cat >Makefile <<'EOF'
at_f = $(@F)
foo/bar.o:
a='$(@F:.o='')' b='$(at_f:.o='')'; \
  test x"$$a.o" = x"$(@F)" || a=$$b; \
  echo $$a
EOF

Cheers,
  Nick

[bug#60535] [PATCH] depend2: switch echo|sed to automatic vars

[Nick Bowler]

On 2023-01-03, Mike Frysinger  wrote:
> The echo|sed is used to split the dirname & filename so it can insert
> $(DEPDIR) in the middle, and then chop the trailing object suffix.  In
> the generic case, %OBJ% is $@, so we can leverage the POSIX vars $(@D)
> and $(@F) to do the pathname splitting and insert $(DEPDIR) in between.
> For chopping the object suffix, we can use the $(xxx:...=...) syntax.
> This avoids invoking sed completely.

I think this is a generally good idea.

> These dirname/filename vars have been in POSIX since at least 2004.

Except for one minor detail, $(@F) and $(@D) are highly portable.  I
expect they were in the very first POSIX.2 specs as they predate the
earliest standards; I believe they first appeared in UNIX System V (ca.
1983) and were later added to BSD in 4.3BSD-Reno (ca. 1990).

The only potential problem I am aware of with these features is one
implementation (dmake) which does not follow the POSIX rule of excluding
the trailing slash from the directory part, and (a bit more troublesome)
will expand $(@D) to the empty string if the directory part is empty,
instead of . as required by POSIX.  Usually this is not a major issue,
as writing something like ./$(@D)/$(@F) is usually sufficient to avoid
any problems with empty $(@D).

It's worth noting that $(?F), $(?D), $(%F) and $(%D) were never added
to BSD and as a result are missing from many BSD derivatives.  However,
these variables are of limited utility and are unlikely to be missed.

The $(x:a=b) suffix substitutions are of identical vintage, and as
far as I know every make supporting file/directory variables also,
in principle, supports suffix substitution too.  However, there are
many bugs in real-world implementations of this feature.

Of particular relevance to this patch: older versions of NetBSD make,
while they do support suffix substitution, fail to correctly parse
$(@F:.o=):

  % cat >Makefile <<'EOF'
foo/bar.o:
echo $(@F:.o=)
EOF
  % make
  echo bar.o.o=)
  Syntax error: ")" unexpected
  *** Error code 2

NetBSD make was widely adopted by other systems, so this particular
issue may actually be found elsewhere.  This bug was present as recently
as NetBSD 7.2 (ca. 2018).  Furthermore, old versions of FreeBSD (before
they ditched their own make in favour of NetBSD's version) also fail to
correctly handle $(@F:.o=), but in a different way.

The good news is that these problems can be worked around pretty easily
simply by using another variable:

  % cat >Makefile <<'EOF'
at_f = $(@F)
foo/bar.o:
echo $(at_f:.o=)
EOF
  % make
  echo bar
  bar

But this workaround trips over a different suffix substitution bug on
ULTRIX, which would otherwise work just fine with plain $(@F:.o=).
Maybe ULTRIX is not worth worrying about these days, but nevertheless,
it is not too hard to detect and work around both problems in pure
shell, which should still work to avoid the additional forks/sed,
maybe something like:

  % cat >Makefile <<'EOF'
at_f = $(@F)
foo/bar.o:
a='$(@F:.o=)' b='$(at_f:.o=)'; test x"$$a.o" = x"$(@F)" || a=$$b;\
  echo $$a
EOF

Cheers,
  Nick

Re: Old .Po file references old directory, how to start fresh?

[Nick Bowler]

On 2022-08-04, Travis Pressler via Discussion list for automake
 wrote:
> I'm learning how to make an autotools project and have created a test
> project to work with. I ran make with a directory `nested` and then deleted
> it and deleted the reference to it in my `Makefile.am`.
>
> Now I'm running ./configure && make and I get the following:
>
> *** No rule to make target 'nested/main.c', needed by 'main.o'. Stop.
>
> How can I run `make` so that it doesn't reference this old nested
> directory?

Sounds like just some stale dependencies left over from a prior version.

Running "make distclean" should delete all the automatically generated
dependency information and allow the package to be rebuilt normally.

Enabling the Automake subdir-objects feature probably would avoid the
specific scenario that led to your stale dependency problem.

Hope that helps,
  Nick

Re: Problem with build

[Nick Bowler]

Hi,

On 2022-08-01, aotto  wrote:
> but in ONE library I dont want to have a static library build because it
> is only used as dlopen (by tcl)…
[...]
> pkglib_LTLIBRARIES = libtclmkkernel.la
[...]
> question what I have to-do to avoid a "static" library "libtclmkkernel.a"

Since this seems to be a libtool question, I have added the libtool list
to Cc.

The following compilation option[1] seems appropriate:

  -shared

  Even if Libtool was configured with --enable-static, the object file
  Libtool builds will not be suitable for static linking.  Libtool
  will signal an error if it was configured with --disable-shared,
  or if the host does not support shared libraries.

And the following link option[2]:

  -shared
  If output-file is a program, then link it against any uninstalled
  shared libtool libraries (this is the default behavior). If output-
  file is a library, then only create a shared library. In the later
  case, libtool will signal an error if it was configured with
  --disable-shared, or if the host does not support shared libraries.

So, if you add -shared to libtclmkkernel_la_CFLAGS and also to
libtclmkkernel_la_LDFLAGS, I'd expect this to work as you expect
(I've not tried it).

[1] https://www.gnu.org/software/libtool/manual/libtool.html#Compile-mode
[2] https://www.gnu.org/software/libtool/manual/libtool.html#Link-mode

Hope that helps,
  Nick

Re: How to speed up 'automake'

[Nick Bowler]

On 2022-05-02, Karl Berry  wrote:
> - @echo '# dummy' >$@-t && $(am__mv) $@-t $@
> + @: >>$@
>
> 1) does it actually speed anything up?

The answer seems to be a resounding "yes".  I tried one of my packages
on an old slow PC, and changing this one line in Makefile.in cuts almost
5 seconds off of the depfiles generation step in config.status.

(All .deps directories manually deleted between runs as otherwise the
rule commands will not be executed).

  Before (x5):
  % time config.status Makefile depfiles
  real  0m15.320s
  real  0m15.210s
  real  0m15.210s
  real  0m15.210s
  real  0m15.220s

  After (x5):
  % time config.status Makefile depfiles
  real  0m10.650s
  real  0m10.550s
  real  0m10.550s
  real  0m10.550s
  real  0m10.650s

That 5 seconds is a relatively small part of total configure runtime but
it is noticeable.

So if make implementations have no problem including empty files (I tried
a few and all seem OK with it) then it seems like a win.

> 2) without the mv I fear we are no longer noticing write failure

I think it's OK.  All shells that I know of set a failure status when
redirection fails, at least for simple commands like that.

One possible gotcha is that redirections on the : command are not always
reliably performed by older shells.

There might not be any real world problem because configure sets SHELL
in the Makefile to one that probably does not exhibit any problem.  If
it matters, performing the redirection with "exec" instead of ":" should
work in every shell and have pretty much identical performance.

Cheers,
  Nick

bug#53780: Automake failing with conditional AC_CONFIG_FILES

[Nick Bowler]

On 2022-02-20, Mike Frysinger  wrote:
> can you link to the project/source ?  the snippet you posted isn't
> complete, and adding a few more lines doesn't cause automake crash for me.

Here is a minimal reproducer:

  % cat >configure.ac <<'EOF'
  AC_INIT([test], [0])
  AM_INIT_AUTOMAKE([foreign])
  AC_CONFIG_FILES([x:y z])
  AC_OUTPUT
EOF

  % touch x.am
  % touch z.am
  % autoreconf -is
  configure.ac:2: installing './install-sh'
  configure.ac:2: installing './missing'
  automake-1.16: error: x:y in @other_input_files
  [...]

Critically, both x.am and z.am files are required to exist to
reproduce the crash.

The "x" output should be ignored by Automake (since its input
filename, "y", does not end in ".in") but it apparently slightly
confused by the existence of the (should not be used) x.am in the
source tree.

I think this crash will, in practice, only ever happen due to
serious mistakes in configure.ac (i.e., I believe the reporter
has used AC_CONFIG_FILES with the wrong filenames).

Cheers,
  Nick

Re: portability of xargs

[Nick Bowler]

On 2022-02-14, Mike Frysinger  wrote:
> context: https://bugs.gnu.org/53340
>
> how portable is xargs ?  like, beyond POSIX, as autoconf & automake both
> support non-POSIX compliant systems.  i want to use it in its simplest
> form: `echo $var | xargs rm -f`.

As far as I can tell xargs was introduced in the original System V UNIX
(ca. 1983).  This utility subsequently made its way back into V10 UNIX
(ca. 1989) and subsequently 4.3BSD-Reno (ca. 1990) and from there to
basically everywhere.  The original implementation from System V
supports the "-x", "-l", "-i", "-t", "-e", "-s", "-n" and "-p" options.
Of these, POSIX only chose to standardize "-x", "-t", "-s", "-n" and
"-p" suggesting possible incompatibilities with other options.

HP-UX 11 xargs expects the last filename to be followed by a white-space
character, or it will be ignored:

  gnu% printf 'no blank at the end' | xargs printf '[%s]'; echo
  [no][blank][at][the][end]

  hpux11% printf 'no blank at the end' | xargs printf '[%s]'; echo
  [no][blank][at][the]

The HP-UX 11 behaviour is also observed on Ultrix 4.5, but not on
4.3BSD-Reno.  Since xargs input typically ends with a newline, this is
not a serious practical problem.

Cheers,
 Nick

bug#53780: Automake failing with conditional AC_CONFIG_FILES

[Nick Bowler]

On 2022-02-04, Valio Valtokari  wrote:
> Hello,
>
> I have a project that supports multiple platforms (windows and linux as of
> right now). To implement testing functionality I use a library that I
> haven't configured for windows in my project yet. As such, my configure.ac
> has these lines:
>
> AC_CONFIG_FILES([Makefile:src/Makefile])

Automake probably shouldn't crash but this line is a very unusual way to
tell configure to prepare the toplevel Makefile... it says the input to
generate it is src/Makefile (as opposed to the default Makefile.in).
Basically, this tells configure to copy src/Makefile to Makefile (and
apply substitutions).

Automake will not recognize this as something it should do anything with
(it only looks for matching .am files if it sees an input file that ends
with ".in").

I can only reproduce the crash if there is a file named "Makefile.am" in
the both the toplevel directory and in the "tests" directory.  The
conditional is irrelevant: this will crash just the same:

  AC_CONFIG_FILES([Makefile:src/Makefile tests/Makefile])

Given that, without knowing anything else about your project, I suspect
you probably meant to write:

  AC_CONFIG_FILES([Makefile src/Makefile])

to have configure and/or automake prepare two different Makefiles.

Hope that helps,
  Nick

Re: adding a command line option for ACLOCAL_PATH-type search paths

[Nick Bowler]

On 19/01/2022, Mike Frysinger  wrote:
> the ACLOCAL_PATH functionality is useful (adding search dirs after -I),
> but a bit unwieldy as an env var.  any reason we can't add a command line
> option for this ?  call it --aclocal-path ?  or --extra-system-acdir ?
> or some other other boring name ?
>
> for context, when cross-compiling, autotools (i.e. automake) tend to be
> installed in the system (i.e. /usr/), while all the libraries & macros
> being built against are found in a separate sysroot (e.g. ~/sysroot/).
> we want to insert that ~/sysroot/usr/share/aclocal path after the set
> of -I flags from the package, but before /usr/share/aclocal.
> -mike

FWIW another option besides the env var is to create a third
directory, and put a file called "dirlist" in it with two lines:

  /path/to/sysroot/usr/share/aclocal
  /usr/share/aclocal

Then you can use the --system-acdir=/path/to/that/directory option
for aclocal to have it search both places.

Cheers,
  Nick

Re: Automake for RISC-V

[Nick Bowler]

On 20/11/2021, Billa Surendra  wrote:
> I have RISC-V native compiler on target image, but when I am compiling
> automake on target image it needs automake on target. This is the main
> problem.

Automake should not be required to install automake if you are using
a released version and have not modified the build system.

> In the same way I am trying to install texinfo on target image it
> also need automake on target.

Likewise, texinfo should also not require automake to install.

Cheers,
  Nick

Re: Automake for RISC-V

[Nick Bowler]

Hi Billa,

On 18/11/2021, Billa Surendra  wrote:
> Dear All,
>
> I have cross-compiled Automake-1.16.2 package with RISC-V cross compiler,
> but when I am executing binaries on RISC-V target OS image its gives errors
> like "not found".

Automake is written in Perl so it does not really get "compiled" in the
usual sense.

[...]
> $   ./configure  --prefix=/usr --host=riscv64-unknown-linux-gnu
> $ make -j8
> $  make DESTDIR=$risc-v_rootfs/ install
[...]
> *Error message (on risc-v rootfs):*
>
> ./aclocal
> -/bin/sh: ./aclocal: not found
>
> ./aclocal-1.16
> -/bin/sh: ./aclocal-1.16: not found

My first guess is that perl is not installed on the host (risc-v)
system.  Specifically, these files begin with #!/usr/bin/perl (or
similar - depends on configure tests) and that program is not
available when you run them.

However, I took a quick look at Automake's configure script and it
appears it detects perl only on the build system and then installs
that filename into the installed scripts.  So I think it will not
work out of the box unless a supported perl version is installed
at the same location on both the build and host machines.

Cheers,
  Nick

Re: `make dist` fails with current git

[Nick Bowler]

On 2021-10-13, Zack Weinberg  wrote:
> On Wed, Oct 13, 2021, at 2:11 PM, Nick Bowler wrote:
>> I think this happened because your CI system has done a shallow clone.
>> So the changelog generation failed because the git log is incomplete.
>
> I did a --single-branch clone, but not a shallow one.  Shouldn't the trunk
> be self-contained?

I think --single-branch should be fine: it seems to work for me...

But with e.g., a "--depth 1" clone I get the exact same error you
reported (because the commit IDs mentioned in .git-log-fix are not
available).

A possible workaround (only if you don't care about the resulting
changelog being correct) would be to truncate .git-log-fix before
running make dist.

Cheers,
  Nick

Re: `make dist` fails with current git

[Nick Bowler]

On 13/10/2021, Zack Weinberg  wrote:
> On Wed, Oct 13, 2021, at 11:54 AM, Bob Friesenhahn wrote:
>> On Wed, 13 Oct 2021, Zack Weinberg wrote:
>>>
>>> Looks like some kind of problem with automatic ChangeLog generation?
>>
>> To me this appears to be the result of skipping an important step in
>> what should be the process.  It seems that there should be a
>> requirement that 'make distcheck' succeed prior to a commit.  Then
>> this issue would not have happened.
>
> Well, yes, but how do I _fix_ it? :-)
>
> <.< I _may_ be investigating the possibility of setting up CI for automake
> so that problems like this are at least noticed in a timely fashion.  But if
> the tree is broken to begin with, it's troublesome...

I think this happened because your CI system has done a shallow clone.
So the changelog generation failed because the git log is incomplete.

I have no trouble running "make dist" from a normal git checkout.

Cheers,
  Nick

Re: generated lex/yacc sources?

[Nick Bowler]

On 21/09/2021, Karl Berry  wrote:
> Suppose I want to generate a lex or yacc input file from another file,
> e.g., a CWEB literate program. Is there a way to tell Automake about
> this so that the ultimately-generated parser/lexer [.ch] files are saved
> in srcdir, as happens when [.ly] are direct sources, listed in *_SOURCES?
>
> I should know the answer to this, but sadly, I don't. I couldn't find
> any hints in the manual or sources or online, although that probably
> only indicates insufficient searching.

I think all that should be needed is to list the .l (or .y) file in
_SOURCES normally, then just write a suitable make rule to update
it from the literate sources.  Automake doesn't "know" about it but
make should do the right thing.  For example, this seems to work OK:

  % cat >Makefile.am <<'EOF'
bin_PROGRAMS = main

main_SOURCES = main.l

# for simplicity, keep distributed stuff in srcdir
$(srcdir)/main.l: $(srcdir)/main.x
cp $(srcdir)/main.x $@

EXTRA_DIST = main.x
MAINTAINERCLEANFILES = main.l
EOF

Cheers,
  Nick

bug#49901: Bug in build c-ares [too many loops]

[Nick Bowler]

On 17/08/2021, Brad House via Bug reports for Automake
 wrote:
> This ended up being the fix to needing to run autoreconf -fi multiple
> times:
> https://github.com/c-ares/c-ares/commit/e73fb47

Probably an OK workaround.  I suspect the effect of this change is to get
aclocal to pull in the ax_require_defined.m4 file on the first pass which
avoids the "too many loops" problem but not the include order problem
(which may not actually matter).

> After that, there is also the issue that the newer ax_code_coverage.m4
> that is automatically pulled in is not API compatible as it removes
> @CODE_COVERAGE_RULES@ which isn't too friendly, plus adds a slew of
> additional .m4 dependencies that had to be imported.

You can avoid things being automatically imported by not enabling
aclocal --install mode by default (remove this setting from
ACLOCAL_AMFLAGS in your Makefile.am files).  The default behaviour
is to use the files distributed with your package in preference to
newer versions that happen to be installed on user's systems.

I expect this also would avoid the "too many loops" problem.

> And then finally, there is actually a bug in the latest
> ax_code_coverage.m4 that quotes AC_MSG_ERROR() which causes older
> autoconf/automake versions to barf.
> https://github.com/c-ares/c-ares/commit/3883d99b05a845e4b40fc25c43ca83db70f4f20f#diff-fe38622a6c742be1bb93e6e65a0728f659114b3dc8d88172cb8844fa963418f9

The original quotation looks reasonable, so this change seems
questionable to me.  Probably there are real quotation bugs
somewhere else.

> I'm not sure where to report the last 2 issues.

For issues with the autoconf-archive macros, they have their own mailing
lists:

  https://savannah.gnu.org/mail/?group=autoconf-archive

Cheers,
  Nick

bug#49901: Bug in build c-ares [too many loops]

[Nick Bowler]

On 17/08/2021, Brad House via Bug reports for Automake
 wrote:
> On 8/17/21 1:06 PM, Nick Bowler wrote:
>> On 2021-08-17, Brad House via Bug reports for Automake
>>  wrote:
>>> I'm one of the maintainers of the c-ares project at
>>> https://github.com/c-ares/c-ares and have had a couple of users report
>>> this issue.
>> I took a quick glance and I found this:
>>
>>https://github.com/c-ares/c-ares/blob/main/m4/zz60-xc-ovr.m4
>>
>> which is redefining autoconf-supplied macros (AC_CONFIG_MACRO_DIR) and
>> is almost certainly the cause of your problem.
[...]
> Thanks for taking a glance at this, unfortunately it didn't correct the
> issue.

Right.  It seems aclocal is being tripped up by AX_CXX_COMPILE_STDCXX_11
and its dependencies from autoconf archive, likely by the newer versions
that are being copied in by aclocal --install.

Here is a small test case that reproduces the problem, emulating
AX_CXX_COMPILE_STDCXX_11 from Autoconf Archive v2021.02.19:

  % cat >configure.ac <<'EOF'
  AC_INIT([test], [0])
  FOO
  AC_OUTPUT
EOF

  % mkdir m4src
  % cat >m4src/00_bar.m4 <<'EOF'
  AC_DEFUN([BAR])
EOF
  % cat >m4src/10_foo.m4 <<'EOF'
  PROBLEM # This line is a problem!
  AC_DEFUN([FOO], [BAR])
EOF
  % cat >m4src/20_problem.m4 <<'EOF'
  AC_DEFUN([PROBLEM])
EOF

  % rm -f m4/*.m4
  % aclocal --system-acdir=$PWD/m4src -I m4 --install
  aclocal: installing 'm4/00_bar.m4' from '/tmp/x/m4src/00_bar.m4'
  aclocal: installing 'm4/10_foo.m4' from '/tmp/x/m4src/10_foo.m4'
  aclocal: installing 'm4/20_problem.m4' from '/tmp/x/m4src/20_problem.m4'
  aclocal: error: too many loops
  [...]

The issue is that due to how aclocal works, on the first pass it cannot
recognize that PROBLEM is a macro in 10_foo.m4 (which corresponds to
ax_cxx_compile_stdcxx_11.m4), because files are included in search
order and when 10_foo.m4 is processed, 20_problem.m4 is not included
yet.  So aclocal --install will copy only 00_bar.m4 and 10_foo.m4 into
the package.

On the second pass (after 00_bar.m4 and 10_foo.m4 are copied into
the package), the order in which these files are included has changed.
Now 20_problem.m4 is included first and this time PROBLEM is actually
recognized as a macro expansion.  So aclocal dutifully installs
20_problem.m4 into the package m4 directory.

Then aclocal bails with "too many loops" because a third pass would be
required.  If you manually run aclocal --install again, it will succeed
(since no files remain to be copied) but with the original ordering: PROBLEM
will not be recognized as a macro expansion (and the generated aclocal.m4 will
not include 20_problem.m4)

This sort of behaviour is the main reason why the aclocal documentation
suggests these files not contain anything other than macro definitions[1],
so that they are not sensitive to include order.

It is probably possible to improve aclocal to not bail out in this
scenario but autoconf-archive should still be fixed, perhaps by
moving the expansion of AX_REQUIRE_DEFINED into the expansion of
AX_CXX_COMPILE_STDCXX_11.  Or by using the Autoconf-provided
m4_pattern_forbid machinery which looks for unexpanded macros
in the ouptut to achieve a similar result.

[1] https://www.gnu.org/software/automake/manual/automake.html#Extending-aclocal

Cheers,
  Nick

bug#49901: Bug in build c-ares [too many loops]

[Nick Bowler]

On 2021-08-17, Brad House via Bug reports for Automake
 wrote:
> I'm one of the maintainers of the c-ares project at
> https://github.com/c-ares/c-ares and have had a couple of users report
> this issue.

I took a quick glance and I found this:

  https://github.com/c-ares/c-ares/blob/main/m4/zz60-xc-ovr.m4

which is redefining autoconf-supplied macros (AC_CONFIG_MACRO_DIR) and
is almost certainly the cause of your problem.  I suggest simply not
doing that, and aclocal will probably work as expected.  Just delete
this file and delete the XC_OVR_ZZ60 line from configure.ac and I
expect your aclocal failure will go away.

The likely reason why the issue is intermittent is because aclocal
will not see this definition on its first pass of a clean workspace,
and the "too many loops" error happens because the state changes in
the middle of the aclocal run, once the redefinition is incorporated
into aclocal.m4 (and things might even appear to work again after
that point, until you delete aclocal.m4 and go again).

If you really need something like this to support bootstrapping with
Autoconf 2.57 (which is getting close to 20 years old now!), I suggest
you find another solution that doesn't involve redefining the internals
on newer Autoconf versions.

Cheers,
  Nick

[RFC PATCH] Improve diagnostics wrt. mutliple AM_INIT_AUTOMAKE calls.

[Nick Bowler]

If you expand AM_INIT_AUTOMAKE more than once with options, automake
currently barfs on the resulting m4 traces with a less-than-helpful
assertion failure ("global options already processed").  This also
prompts users to report bugs against Automake, when the actual
problem is with their autoconf input.

Avoid the problem by detecting the scenario at M4 time, which is the
best position to provide an accurate description of the error.  I cannot
imagine there is any good reason to expand AM_INIT_AUTOMAKE more than
once so we can just reject it outright.  In a typical flow this means
the error will now be reported early by aclocal, and hopefully this
explains better to the user what is wrong.  For example, with the
configure.ac from bug #50046:

  % aclocal
  configure.ac:12: error: AM_INIT_AUTOMAKE expanded multiple times
  /tmp/am/share/aclocal-1.16/init.m4:29: AM_INIT_AUTOMAKE is expanded from...
  configure.ac:10: the top level
  /tmp/am/share/aclocal-1.16/init.m4:29: AM_INIT_AUTOMAKE is expanded from...
  configure.ac:12: the top level
  autom4te-2.69: /usr/bin/m4 failed with exit status: 1
  aclocal: error: autom4te failed with exit status: 1

This addresses .

* m4/init.m4: Make attempts to expand AM_INIT_AUTOMAKE more than
  once a fatal error at m4 time.
---

NOTE: this implementation depends on the undocumented m4sugar macro
m4_expansion_stack in order to display the location of both macro calls
(this is what m4_fatal uses internally to show the current location).

This seems like generally useful functionality.  I wonder if we should
consier describing it in the Autoconf manual.

 m4/init.m4 | 4 
 1 file changed, 4 insertions(+)

diff --git a/m4/init.m4 b/m4/init.m4
index c5807d211..b4686b140 100644
--- a/m4/init.m4
+++ b/m4/init.m4
@@ -28,6 +28,10 @@ m4_defn([AC_PROG_CC])
 # release and drop the old call support.
 AC_DEFUN([AM_INIT_AUTOMAKE],
 [AC_PREREQ([2.65])dnl
+m4_ifdef([_$0_ALREADY_INIT],
+  [m4_fatal([$0 expanded multiple times
+]m4_defn([_$0_ALREADY_INIT]))],
+  [m4_define([_$0_ALREADY_INIT], m4_expansion_stack)])dnl
 dnl Autoconf wants to disallow AM_ names.  We explicitly allow
 dnl the ones we care about.
 m4_pattern_allow([^AM_[A-Z]+FLAGS$])dnl
-- 
2.31.1

bug#50046: Probably a bug

[Nick Bowler]

On 2021-08-15, Karl Berry  wrote:
> two expansions of AM_INIT_AUTOMAKE in your configure.ac.
> ...
> diagnostic message was certainly not very useful!
>
> Fully agreed.
>
> Nick, any chance you could easily write a patch to generate a decent
> error message for this? I am tied up with other projects for a while.

I'll see what I can come up with.

Cheers,
  Nick

bug#50046: Probably a bug

[Nick Bowler]

On 2021-08-14, José Hipólito Moyano  wrote:
> I attached the project files in the email. Maybe they were filtered :-)
> Here they go again (maybe the C code won't compile XD, I was just
> starting)
[...]
>> automake: error: global options already processed

Your error is presumably caused by the two expansions of
AM_INIT_AUTOMAKE in your configure.ac.  There should be just one,
so instead of

  AM_INIT_AUTOMAKE([foreign 1.16])
  # To enable non recursive automake
  AM_INIT_AUTOMAKE([subdir-objects])

try:

  AM_INIT_AUTOMAKE([foreign subdir-objects 1.16])

Strictly speaking I would say there is an Automake bug here: the
diagnostic message was certainly not very useful!

Hope that helps,
  Nick

Re: How to prevent distribution of `texinfo.tex`

[Nick Bowler]

On 2021-06-23, Werner LEMBERG  wrote:
 Yeah, it would be nice to have a means to control that.
>>
>> Yes it is really not a good solution in this case.  The file is
>> detected at "automake" time and the rule to distribute texinfo.tex
>> is baked into the generated Makefile.in.  That then gets bundled up
>> into the tarball.
>
> Yep.  The size of `texinfo.tex` (374kByte) is not significant today,
> but if there is no Texinfo documentation in a project it is completely
> pointless to include it.
>
> I vote for removing this file from the list of mandatory files.

It's not mandatory.  It only gets included when the file is present in
your development workspace (presumably by some mistake?) when you run
automake.  I don't know how you ended up with it there in the first
place but simply delete texinfo.tex from your workspace and re-run
automake: now it won't be included in the distribution (problem solved?).

I'm afraid I'm not really understanding why this is an issue.

Removing anything from this list will just break any projects that
depend on the current behaviour when they switch to a new version
of Automake, probably in subtle and hard-to-notice ways.

Cheers,
  Nick

Re: How to prevent distribution of `texinfo.tex`

[Nick Bowler]

On 23/06/2021, Peter Johansson  wrote:
>
> On 24/6/21 3:02 am, Werner LEMBERG wrote:
>>> As far as I know there is no way to disable this behaviour, although
>>> I agree the automagic file inclusion can be a bit funky.
>> Yeah, it would be nice to have a means to control that.
>
> There is the dist hook, which can be used to remove files from the
> distribution, but seems dangerous to exclude files from the tarball that
> are mentioned in the Makefile.

Yes it is really not a good solution in this case.  The file is detected
at "automake" time and the rule to distribute texinfo.tex is baked into
the generated Makefile.in.  That then gets bundled up into the tarball.

If you simply delete texinfo.tex in a dist-hook rule, users will not
be able to run "make dist" from the tarball.  This is one of the things
tested by "distcheck" so that should hopefully catch it.

My suggestion, if accidental inclusion of texinfo.tex is really a
serious problem, is to use distcheck-hook and simply fail the check
if texinfo.tex is present in distdir.

Cheers,
  Nick

Re: How to prevent distribution of `texinfo.tex`

[Nick Bowler]

On 2021-06-23, Werner LEMBERG  wrote:
> The file `texinfo.tex` is in the list of files (given by `automake
> --help`) that gets always distributed.  How can I disable this?  I
> don't have texinfo documentation.

The texinfo.tex file (and others listed along with it) is included in
the distribution only if the file is present in your project workspace
when you run "automake" to (re)generate Makefile.in.

As far as I know there is no way to disable this behaviour, although I
agree the automagic file inclusion can be a bit funky.

Cheers,
  Nick

Re: parallel build issues

[Nick Bowler]

On 2021-06-21, Werner LEMBERG  wrote:
>
>> The problem is not related to the snippet you posted.  The
>> concurrent recursive make invocations are being spawned from
>> somewhere else in your build system.
>
> The `Makefile.am` file one level higher is as follows.
>
>   ACLOCAL_AMFLAGS = -I gnulib/m4 -I m4
>
>   SUBDIRS = gnulib/src \
> lib \
> frontend \
> doc
>   EXTRA_DIST = bootstrap \
>bootstrap.conf \
>FTL.TXT \
>gnulib/m4/gnulib-cache.m4 \
>GPLv2.TXT \
>README \
>TODO \
>.version
>
>   BUILT_SOURCES = .version
>   .version:
> echo $(VERSION) > $@-t && mv $@-t $@
>
>   dist-hook:
> echo $(VERSION) > $(distdir)/VERSION.TXT
>
> Looks pretty standard to me, but maybe I'm wrong.

Nothing shown here is going to cause this problem.  But with recursive
build problems it is insufficient to just look at just one makefile:
the problematic make invocations could be coming from anywhere in your
project.

For example, perhaps you have the same "frontend" directory listed also
in SUBDIRS for some other unrelated makefile?  That is probably the
simplest way this situation could happen.

Or perhaps the parent directory's makefile was itself being processed
by concurrent recursive invocations, which then results in independent
recursive invocations in the subdirectories.

If you can't find anything by a code inspection GNU make has some debug
features which may help visualize what is going on.

Cheers,
  Nick

Re: parallel build issues

[Nick Bowler]

On 2021-06-21, Warren Young  wrote:
> On Jun 21, 2021, at 11:49 AM, Werner LEMBERG  wrote:
>>
>>  bin_PROGRAMS += ttfautohintGUI
>
> Is Automake smart enough to realize what you’ve done there?

This is not a problem.  Automake interprets this assignment syntax
(and outputs a single assignment to bin_PROGRAMS in Makefile.in).

Cheers,
  Nick

Re: parallel build issues

[Nick Bowler]

On 2021-06-21, Werner LEMBERG  wrote:
> I have a `Makefile.am` in a `frontend` subdirectory that looks like
> the following (abridged).
[...]
> Running the generated `Makefile` with `make -j12`, I get this:
>
>   ...
>   Making all in frontend
>   make[2]: Entering directory '.../frontend'
>   ...
>   make  all-am
>   make[3]: Entering directory '.../frontend'
> CXX  info.o
> CXX  main.o
> CXX  ttfautohintGUI-ddlineedit.o
> CXX  ttfautohintGUI-info.o
> CXX  ttfautohintGUI-main.o
> CXX  ttfautohintGUI-maingui.o
> CXX  ttfautohintGUI-ttlineedit.o
>   make  ttfautohint
> CXX  ttfautohintGUI-ddlineedit.moc.o
> CXX  ttfautohintGUI-maingui.moc.o
> CXX  ttfautohintGUI-static-plugins.o
> CXX  ttfautohintGUI-ttlineedit.moc.o
>   make[4]: Entering directory '.../frontend'

Here one of your make rules has started a recursive build in the
"frontend" directory.

> CXX  info.o
>   make  ttfautohintGUI
>   make[4]: Entering directory '.../frontend'

Here *another* one of your make rules has *also* started a recursive
build in "frontend", concurrently with the one that was started
previously.

> CXXLDttfautohintGUI
> CXX  main.o
>   mv: cannot stat '.deps/info.Tpo': No such file or directory

Those independent make invocations are simultaneously running the same
compilation rules and have no knowledge of what the other make processes
are doing.  Failure is the inevitable outcome.

>   make[4]: *** [Makefile:1401: info.o] Error 1
>   make[4]: *** Waiting for unfinished jobs
> CXXLDttfautohint
>   make[4]: Leaving directory '.../frontend'
>   ...
>
> What am I doing wrong?  I would be glad for any pointers.

The problem is not related to the snippet you posted.  The concurrent
recursive make invocations are being spawned from somewhere else in
your build system.

Hope that helps,
  Nick

Re: config.sub/config.guess using nonportable $(...) substitutions

[Nick Bowler]

On 09/03/2021, Warren Young  wrote:
> On Mar 9, 2021, at 1:26 PM, Paul Eggert  wrote:
>>
>>> 1) There is no actual benefit to using $(...) over `...`.
>>
>> I disagree with that statement on technical grounds (not merely cosmetic
>> grounds), as I've run into real problems in using `...` along with " and
>> \,
>
> Me too, plus nesting.  The difference is most definitely not cosmetic.

I think what Karl means is that it is usually very easy to portably work
around the problems of nested and/or quoted `...` substitutions (usually
by just using a variable).

In other words, the difference between a script using $(...) and an
equivalent, more portable script using `...` is only one of appearance.

Regardless, there are no quoted or nested substitutions whatsoever in
config.sub.  I see exactly one nested substitution in config.guess, and
just a handful of quoted ones.  None appear particularly challenging to
write portably.

> Autoconf came out in 1991, so it’s the equivalent of supporting Version 6
> Unix (1975) in the original release, which it probably didn’t do, given that
> the Bourne shell didn’t even exist at that point.
>
> Are the malcontents not expecting heroic levels of backwards compatibility
> that Autoconf never has delivered?

No, I'm just expecting that things are not broken gratuitously in core
portability tools because someone does not like the appearance of the
more portable syntax.

I _especially_ don't expect this kind of breakage when upgrading from one
Automake point release to another (1.16.1 to 1.16.3).

Cheers,
  Nick

Re: config.sub/config.guess using nonportable $(...) substitutions

[Nick Bowler]

On 2021-03-08, Tim Rice  wrote:
> On Mon, 8 Mar 2021, Nick Bowler wrote:
[...]
>> These scripts using $(...) are incorporated into the recently-released
>> Automake 1.16.3, which means they get copied into packages bootstrapped
>> with this version.  So now, if I create a package using the latest bits,
>> configuring with heirloom-sh fails:
>>
>>   % CONFIG_SHELL=/bin/jsh jsh ./configure CONFIG_SHELL=/bin/jsh
>>   configure: error: cannot run /bin/jsh ./config.sub
>
> But why would you use CONFIG_SHELL= to specify a less capable shell?
> It is there to specify a more capable shell in case it is not already
> detected.

It is simply a proxy to test Solaris /bin/sh behaviour using a modern
GNU/Linux system.  This is much easier and faster than actually testing
on old Solaris systems and, more importantly, anyone can download and
install this shell as it is free software and reasonably portable.

Obviously I can successfully run my scripts on GNU/Linux using a modern
shell such as GNU Bash.  But that's not the point: Autoconf and friends
are first and foremost portability tools.  For me the goal is that this
should be working anywhere that anyone might reasonably want to run it.

But right now, it seems these portability tools are actually *causing*
portability problems, rather than solving them.  From my point of view
this is a not so great situation.

Cheers,
  Nick

config.sub/config.guess using nonportable $(...) substitutions

[Nick Bowler]

Hi,

I noticed that config.sub (and config.guess) scripts were very recently
changed to use the POSIX $(...) form for command substitutions.

This change is, I fear, ill-advised.  The POSIX construction is
widely understood to be nonportable as it is not supported by
traditional Bourne shells such as, for example, Solaris 10 /bin/sh.
This specific portability problem is discussed in the Autoconf manual
for portable shell programming[1].

These scripts using $(...) are incorporated into the recently-released
Automake 1.16.3, which means they get copied into packages bootstrapped
with this version.  So now, if I create a package using the latest bits,
configuring with heirloom-sh fails:

  % CONFIG_SHELL=/bin/jsh jsh ./configure CONFIG_SHELL=/bin/jsh
  configure: error: cannot run /bin/jsh ./config.sub

  % jsh config.sub x86_64-pc-linux-gnu
  config.sub: syntax error at line 53: `me=$' unexpected

(The heirloom-sh is essentially Solaris /bin/sh but runs on GNU/Linux systems).

What was the motivation for this change?  Backquotes work fine and are
more portable.  Can we just revert it so the script works again with
traditional shells?  Surely these scripts should be maximally portable,
I would think?

[1] 
https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.70/autoconf.html#index-_0024_0028commands_0029

Cheers,
  Nick

Re: RFC: Bump minimum Perl to 5.18.0 for next major release of both Autoconf and Automake

[Nick Bowler]

On 2021-02-18, Karl Berry  wrote:
> I think the right thresholds are 5.10 for absolute minimum and 5.16
> for 'we aren't going to test with anything older than this'
>
> I appreciate the effort to increase compatibility with old versions.
>
> I imagine you could provide Digest::SHA "internally", or test for it as
> Nick suggested, but I know how much of a pain it is to avoid/check for
> use of things that have seemingly been around forever. (Comes up all the
> time in the TeX world.)

Just to clarify, I was not suggesting that any kind of test is needed
before going ahead and using this module.  If there is a good reason
to use the module in Autoconf, as far as I'm concerned we should just
go ahead and use it.

I was just pointing out that requiring this module in Autoconf does not,
by itself, imply requiring perl 5.10, as the module may be available on
older installations too.

The reason for failures due to a missing module like this will be
obvious immediately.  A configure test may be _nice_ but probably
just extra work that is not really needed.

Cheers,
  Nick

Re: RFC: Bump minimum Perl to 5.18.0 for next major release of both Autoconf and Automake

[Nick Bowler]

Hi Zack,

On 2021-02-17, Zack Weinberg  wrote:
> On Fri, Jan 29, 2021 at 5:54 PM Karl Berry  wrote:
>> But, I think it would be wise to give users a way to override the
>> requirement, of course with the caveat "don't blame us if it doesn't
>> work", unless there are true requirements such that nothing at all would
>> work without 5.18.0 -- which seems unlikely (and undesirable, IMHO).
>> 2013 is not that long ago, in autotime.
>
> This is a reasonable suggestion but Perl makes it difficult.
[...]
> What we could do is something like this instead:
>
>use 5.008;  # absolute minimum requirement
>use if $] >= 5.016, feature => ':5.16';  # enable a number of
> desirable features from newer perls
>
> + documentation that we're only _testing_ with the newer perls.

FWIW, I just checked and I do currently build an Autotest testsuite
on a system where "perl" is perl 5.8.3, which works on autoconf-2.69.

So I suppose if Autoconf required a newer version, and I required a
newer version of Autoconf, then this is a problem.  But due to the
nature of Autoconf this is exclusively my problem and does not impact
downstream users at all.  So I'd just solve the problem (perhaps by
running autom4te on an updated setup) and wouldn't be bothered if
things are broken for a reason.

Only testing with new(ish) perl versions is not at all a problem IMO.
Interoperability is always "best effort": nobody can test every possible
system configuration.  As long as we don't claim to support systems
that are never ever tested, people who care about particular systems
just have to speak up when things stop working.

> I did some more research on perl's version history (notes at end) and
> I think the right thresholds are 5.10 for absolute minimum and 5.16
> for 'we aren't going to test with anything older than this'.  5.10 is
> the oldest perl that shipped Digest::SHA, which I have a specific need
> for in autom4te;

... on the topic of of reasons to break things, the perl 5.8 installation
in question does seem to have Digest::SHA available to it.  So for this
dependency I would suggest Autoconf should be following the Autoconf
philosophy and "you must have the Digest::SHA perl module" is different
from "you must have perl version 5.10 or newer".

> it is also the oldest perl to support `state` variables and the `//`
> operator, both of which could be quite useful.

However these new syntactic constructs are obviously unavailable.
I think "//" is not a great reason (by itself) to break compatibility
but "state" could be.

Cheers,
  Nick

Re: DIST_COMMON

[Nick Bowler]

On 2021-02-17, Leo Butler  wrote:
> I cannot find DIST_COMMON documented in the automake manual[*]. Is this
> intended or an oversight?

Most likely intentional, this looks pretty internal to the "make dist"
machinery and not meant to be used directly by package authors.

> Looking at the automake perl script doesn't really enlighten me,
> either.
>
> I would like to know:
>
> -what does DIST_COMMON contain by default?

Looks to me like it is set to the list of files that Automake will
package by "make dist" that aren't otherwise explicitly listed in
Makefile.am.

So, loosely speaking, it should contain all[1] the files that Autoconf
used to produce "configure" plus all the files that Automake used to
produce "Makefile.in", plus the outputs of those processes, plus a
few other files that get automatically distributed such as the various
GNU-standard files like ChangeLog.  Probably some other things too.

[1] the ones Automake knows about, anyway.

> -is it possible to set it or otherwise over-ride it in Makefile.am?

Techncally the answer to this question is "yes".

Automake allows pretty much anything that it generates to be overridden
by Makefile.am, including DIST_COMMON: just include an assignment to
DIST_COMMON in Makefile.am, setting it to whatever you desire, and it
will suppress the assignment generated by Automake.

However, while this is possible, overriding Automake-internal definitions
is not generally recommended.  If you need to tweak this, consider using
a dist-hook rule instead.

Cheers,
  Nick

bug#45756: Prepending '+' to the recipe line when linking with GCC's -flto=jobserver

[Nick Bowler]

On 2021-01-09, R. Diez  wrote:
> [Resending in hopes it will attach to the new bug. --karl]
>
>> [...]
>> At any rate, it would be extremely helpful to have a minimal-as-possible
>> runnable (automake-able) example showing the case where the + needs to
>> be prepended. rdiez, can you create such a mini-project?
>  > [...]
>
> I normally use Autoconf, and I do not understand very much the separation
> between Autoconf and Automake. I do not know who is responsible for the
> generation of the makefile rules to link the executable. Either Autoconf or
> Automake must decide that GCC is not just used for compiling each object
> file, but also for linking, and that rule is not visible in the makefile.am
> file.
>
> Of course, such a linking rule does not user $(MAKE), and there is no '+'
> prefix, so the GNU Make jobserver file descriptors will not be passed to
> child processes. This is documented in the GNU Make manual.
>
> You do not need any special demo project for this. Just take any existing
> Automake project written in C or C++, and use these compilation flags in
> configure.ac :
>
> AM_CFLAGS="-flto=jobserver"
> AM_CXXFLAGS="-flto=jobserver"
>
> If you run the makefile with "make -j 2", GCC will receive environment
> variable MAKEFLAGS with a setting like "--jobserver-fds=xxx", but GNU Make
> will
> close the file descriptors mentioned there before executing the rule and
> running GCC. This issue is not visible in GCC yet due to this bug I
> reported:
>
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94330
>
> I am not sure how I can demonstrate this in a project, there is not actually
> much to demonstrate.
>
> This is not just an issue while linking. Like I said, any stage, including
> compilation of a single object file, could use the GNU Make jobs server. So
> there should be a global option in Autoconf or Automake to prepend a '+' to
> all generated rules.

This issue has come up from time to time.  I think I wrote something on
it recently.  think everyone can agree that a solution to this problem
is desirable.

However simply prepending "+" to commands is not practical for Automake
to do because "+" has way more effects than just keeping the jobserver
fds open.  In particular, it will completely break "make -n".

A configure option to allow the user to enable this (rather than an
automake option) would probably be a simple and acceptable way to
get things at leas working, even if it's not an "ideal" solution.

Another possibility is for a "+"-prefixed command to check MAKEFLAGS
to see if options like -n that suppress command execution were used
(Automake already has to do this sort of thing in some rules).

Since I believe the jobserver feature is exclusive to GNU make, I
imagine it would also be possible to make use of GNU make substitution
features to only add the "+" when make options that suppress command
execution are omitted.  This could probably be done in a manner that is
interoperable with other make implementations and would likely perform
better than shell tests inside commands.

Finally this issue could also probably be solved by changing GNU make
itself: providing another mechanism to keep jobserver fds open in rules.

Cheers,
  Nick

Re: bug#45756: Prepending '+' to the recipe line when linking with GCC's -flto=jobserver

[Nick Bowler]

On 2021-01-09, R. Diez  wrote:
> [Resending in hopes it will attach to the new bug. --karl]
>
>> [...]
>> At any rate, it would be extremely helpful to have a minimal-as-possible
>> runnable (automake-able) example showing the case where the + needs to
>> be prepended. rdiez, can you create such a mini-project?
>  > [...]
>
> I normally use Autoconf, and I do not understand very much the separation
> between Autoconf and Automake. I do not know who is responsible for the
> generation of the makefile rules to link the executable. Either Autoconf or
> Automake must decide that GCC is not just used for compiling each object
> file, but also for linking, and that rule is not visible in the makefile.am
> file.
>
> Of course, such a linking rule does not user $(MAKE), and there is no '+'
> prefix, so the GNU Make jobserver file descriptors will not be passed to
> child processes. This is documented in the GNU Make manual.
>
> You do not need any special demo project for this. Just take any existing
> Automake project written in C or C++, and use these compilation flags in
> configure.ac :
>
> AM_CFLAGS="-flto=jobserver"
> AM_CXXFLAGS="-flto=jobserver"
>
> If you run the makefile with "make -j 2", GCC will receive environment
> variable MAKEFLAGS with a setting like "--jobserver-fds=xxx", but GNU Make
> will
> close the file descriptors mentioned there before executing the rule and
> running GCC. This issue is not visible in GCC yet due to this bug I
> reported:
>
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94330
>
> I am not sure how I can demonstrate this in a project, there is not actually
> much to demonstrate.
>
> This is not just an issue while linking. Like I said, any stage, including
> compilation of a single object file, could use the GNU Make jobs server. So
> there should be a global option in Autoconf or Automake to prepend a '+' to
> all generated rules.

This issue has come up from time to time.  I think I wrote something on
it recently.  think everyone can agree that a solution to this problem
is desirable.

However simply prepending "+" to commands is not practical for Automake
to do because "+" has way more effects than just keeping the jobserver
fds open.  In particular, it will completely break "make -n".

A configure option to allow the user to enable this (rather than an
automake option) would probably be a simple and acceptable way to
get things at leas working, even if it's not an "ideal" solution.

Another possibility is for a "+"-prefixed command to check MAKEFLAGS
to see if options like -n that suppress command execution were used
(Automake already has to do this sort of thing in some rules).

Since I believe the jobserver feature is exclusive to GNU make, I
imagine it would also be possible to make use of GNU make substitution
features to only add the "+" when make options that suppress command
execution are omitted.  This could probably be done in a manner that is
interoperable with other make implementations and would likely perform
better than shell tests inside commands.

Finally this issue could also probably be solved by changing GNU make
itself: providing another mechanism to keep jobserver fds open in rules.

Cheers,
  Nick

Re: Automake's file locking

[Nick Bowler]

On 2021-02-03, Bob Friesenhahn  wrote:
> GNU make does have a way to declare that a target (or multiple
> targets) is not safe for parallel use.  This is done via a
> '.NOTPARALLEL: target' type declaration.

According to the manual[1], prerequisites on .NOTPARALLEL target are
ignored and this will simply disable parallel builds completely for
the entire Makefile.  I did a quick test and the manual seems to be
accurate about this.

Order-only prerequisites can be used to prevent GNU make from running
specific rules in parallel.  These are more difficult (but not impossible)
to declare in an interoperable way.

[1] https://www.gnu.org/software/make/manual/make.html#index-_002eNOTPARALLEL

Cheers,
  Nick

Re: Automake's file locking (was Re: Autoconf/Automake is not using version from AC_INIT)

[Nick Bowler]

On 2021-01-28, Zack Weinberg  wrote:
> There is a potential way forward here.  The *only* place in all of
> Autoconf and Automake where XFile::lock is used, is by autom4te, to
> take an exclusive lock on the entire contents of autom4te.cache.
> For this, open-file locks are overkill; we could instead use the
> battle-tested technique used by Emacs: symlink sentinels.  (See
> https://git.savannah.gnu.org/cgit/emacs.git/tree/src/filelock.c .)
>
> The main reason I can think of, not to do this, is that it would make
> the locking strategy incompatible with that used by older autom4te;
> this could come up, for instance, if you’ve got your source directory
> on NFS and you’re building on two different clients in two different
> build directories.  On the other hand, this kind of version skew is
> going to cause problems anyway when they fight over who gets to write
> generated scripts to the source directory, so maybe it would be ok to
> declare “don’t do that” and move on.  What do others think?

I think it's reasonable to expect concurrent builds running on different
hosts to work if and only if they are in different build directories and
no rules modify anything in srcdir.  Otherwise "don't do that."

If I understand correctly the issue at hand is multiple concurrent
rebuild rules, from a single parallel make implementation, are each
invoking autom4te concurrently and since file locking didn't work,
they clobber each other and things go wrong.

I believe mkdir is the most portable mechanism to achieve "test and set"
type semantics at the filesystem level.  I believe this works everywhere,
even on old versions of NFS that don't support O_EXCL, and on filesystems
like FAT that don't support any kind of link.

The challenge with alternate filesystem locking methods compared to
proper file locks is that you need a way to recover when your program
dies before it can clean up its lock files or directories.

Could the issue be fixed by just serializing the rebuild rules within
make?  This might be way easier to do.  For example, we can easily
do it in NetBSD make:

  all: recover-rule1 recover-rule2
  clean:
rm -f recover-rule1 recover-rule2

  recover-rule1 recover-rule2:
@echo start $@; sleep 5; :>$@; echo end $@

  .ORDER: recover-rule1 recover-rule2

Heirloom make has a very similar mechanism that does not guarantee
relative order:

  .MUTEX: recover-rule1 recover-rule2

Both of these will ensure the two rules are not run concurrently by a
single parallel make invocation.

GNU make has order-only prerequisites.  Unlike the prior methods, this
is trickier to do without breaking other makes, but I have used a method
like this one with success:

  # goal here is to get rule1_seq set to empty string on non-GNU makes
  features = $(.FEATURES) # workaround problem with old FreeBSD make
  orderonly = $(findstring order-only,$(features))
  rule1_seq = $(orderonly:order-only=|recover-rule1)

  recover-rule2: $(rule1_seq)

I don't have experience with parallel builds using other makes.

Cheers,
  Nick

Re: Future plans for Autotools

[Nick Bowler]

As always, thanks for all your effort Zack!

I wanted to share some of my thoughts on Autoconf and friends.  Maybe I
wrote too much.

For me the most important requirement of the GNU build system is that
it must be as straightforward as possible for novice users to build free
software packages from source code, with or without local modification.

This is what empowers users with the benefits of free software.  If
users are unable to build or modify the software that they use, they
are unable to take advantage of those benefits.

For me, every other consideration is secondary.

The interface consistency prescribed by the GNU coding standards goes
a long way: you learn the steps for one package and can apply that
knowledge to almost any other package.

The trend towards requiring everyone to build from VCS snapshots
and requiring zillions of specific versions of various build tools
is concerning.  Unfortunately I think many developers don't really
care about the user experience when it comes to building their software
releases from source.

This brings me to another important strength of the GNU Build System: if
I prepare a package today I want to be confident that people will still
be able to build it 5, 10, 20 or more years from now.

Now obviously we can't predict the future but we can look to past
experience: just today, I unpacked GNU Bison 1.25 (ca. 1996) on a modern
GNU/Linux system, running on a processor architecture and distribution
that didn't even exist back then, and it builds *out of the box*.

Typical issues encountered with old GNU packages are usually very minor
if you have any problems at all.  For a more complex example, I tried
building glib-1.2.10 (ca. 2001).  I had to update config.sub/config.guess
to the latest, set CC='gcc -std=gnu89' (because the code does not work with
C99 inline) and edit one line of code to disable use of an obsolete GNU C
extension (both compilation problems are due to not following the Autoconf
philosophy and using version checks instead of feature checks, oops!)

My general experience with CMake is that you probably can't build any
old packages because whatever version of CMake you have available simply
doesn't understand the package's build scripts, and the version which
could understand them just doesn't work on your system because you have
a newer processor or something.

I don't have enough experience with Meson to say.  Mainstream free
software packages have only very recently started using it.  On the
GNU side, glib-2.60 (ca. 2019) converted to meson and I am able to
build it.  If possible, I will have to try again in 2039.  I bet the
autoconf-based glib-1.2.10 tarball from 2001 will still mostly work,
and so will the 1996 version of GNU Bison.

Cheers,
  Nick

Re: INSTALL_DATA += -p

[Nick Bowler]

On 2020-11-03, Thien-Thi Nguyen  wrote:
> I'd like to make sure that timestamps are preserved on "make
> install".

In general, preserving timestamps while copying files cannot be done
reliably and when it is possible, it is difficult to do in a portable
fashion.  But it seems preservation is not really required.

> I found the variable ‘INSTALL_DATA’ but cannot do the
> above (subject line) addition to Makefile.am, because Automake
> interprets INSTALL as a primary and bails out since that is not
> defined as such.

These INSTALL... variables come from configure (via AC_PROG_INSTALL); if
you want to change them you can alter them in your configure script.

It is important to note, however, that Automake's supplied install-sh
script currently does not implement "-p" or any other option to copy
timestamps.

> The background reason is that i am installing .scm and .go files
> (the latter compiled from the former) and the .go files need to
> have a "later" timestamp than the .scm files for Guile to DTRT.

"Later" here means "greater than?"  Or is "greater than or equal to"
acceptable?

> I suppose a workaround is to use an installation hook to simply
> touch(1) the .go files.

An install hook would be my recommendation.

You should touch both files, as some "touch" implementations
truncate timestamps (this means a touched file could potentially
get an earlier timestamp than an untouched one).

Touching both files only ensures "greater than or equal to" timestamps.
If the timestamps must also be different then this is a bit trickier
(call touch in a loop until the timestamp changes).

Cheers,
  Nick

Re: [PATCH] Add --jobserver/jobserver option for GCC -flto=jobserver

[Nick Bowler]

On 2020-10-28, Zack Weinberg  wrote:
> On Wed, Oct 28, 2020 at 2:16 PM Nick Bowler  wrote:
>> On 2020-10-28, H.J. Lu  wrote:
>> > GCC introduced some time ago option -flto=jobserver in order to use the
>> > GNU Make jobserver when parallelising LTO builds.  It is actually a
>> > similar "recursive make".  When doing a recursive make, you need to
>> > place a '+' character at the beginning of the recipe line in order to
>> > let GNU Make pass the jobserver file descriptors to the child
>> > processes.
>> >
>> > Add the --jobserver option to add a '+' character to the recipe line in
>> > program.am and ltlibrary.am.
> ...
>> Surely this needs to be a configure-time option, perhaps combined
>> with some sort of configure test, since otherwise users won't get
>> this choice, right?  As an automake option the choice made by whomever
>> prepares the distribution will get baked into distributed Makefile.in
>> files...
>
> I was going to say something very similar: there shouldn't be an
> option at all.  The decision of whether or not to put + at the
> beginning of the recipe line should be made _when make is run_, based
> on whether -flto=jobserver actually appears in $(LDFLAGS) or wherever.

Since this check cannot be done inside a rule, testing LDFLAGS at make
time is probably impractical to do portably.

However I think I misunderstood the impact of this option on first
reading.

I am not aware of any portability concerns with including "+" in
commands, if this is literally the only difference it should be OK
to just always include it (after ensuring that "make -n" is respected
portably in the command).

Cheers,
  Nick

Re: [PATCH] Add --jobserver/jobserver option for GCC -flto=jobserver

[Nick Bowler]

On 2020-10-28, H.J. Lu  wrote:
> GCC introduced some time ago option -flto=jobserver in order to use the
> GNU Make jobserver when parallelising LTO builds.  It is actually a
> similar "recursive make".  When doing a recursive make, you need to
> place a '+' character at the beginning of the recipe line in order to
> let GNU Make pass the jobserver file descriptors to the child processes.
>
> Add the --jobserver option to add a '+' character to the recipe line in
> program.am and ltlibrary.am.
[...]
> @@ -8100,6 +8106,7 @@ Operation modes:
>--versionprint version number, then exit
>-v, --verboseverbosely list files processed
>--no-force   only update Makefile.in's that are out of date
> +  --jobserver  enable GNU make jobserver
>-W, --warnings=CATEGORY  report the warnings falling in CATEGORY
>
>  Dependency tracking:
> @@ -8176,6 +8183,7 @@ sub parse_arguments ()
>   'include-deps'  => sub { $ignore_deps = 0; },
>   'i|ignore-deps' => sub { $ignore_deps = 1; },
>   'no-force'  => sub { $force_generation = 0; },
> + 'jobserver' => sub { $enable_jobserver = '+ '; },
>   'f|force-missing'  => \$force_missing,
>   'a|add-missing' => \$add_missing,
>   'c|copy'=> \$copy_missing,
> diff --git a/doc/automake.texi b/doc/automake.texi
> index 17bc2dae6..d09b1d94a 100644
> --- a/doc/automake.texi
> +++ b/doc/automake.texi
> @@ -2719,6 +2719,11 @@ This enables the dependency tracking feature.  This
> feature is enabled
>  by default.  This option is provided for historical reasons only and
>  probably should not be used.
>
> +@item --jobserver
> +@opindex --jobserver
> +This enables the GNU make jobserver feature support for GCC
> -flto=jobserver
> +option.

Surely this needs to be a configure-time option, perhaps combined
with some sort of configure test, since otherwise users won't get
this choice, right?  As an automake option the choice made by whomever
prepares the distribution will get baked into distributed Makefile.in
files...

[...]
> diff --git a/lib/am/program.am b/lib/am/program.am
> index 9a243b399..4df9cbeee 100644
> --- a/lib/am/program.am
> +++ b/lib/am/program.am
> @@ -21,4 +21,4 @@
>  ## Or maybe not... sadly, incremental linkers are rarer than losing
>  ## systems.
>   @rm -f %PROGRAM%%EXEEXT%
> - %VERBOSE%$(%XLINK%) $(%XPROGRAM%_OBJECTS) $(%XPROGRAM%_LDADD) $(LIBS)
> + %JOBSERVER%%VERBOSE%$(%XLINK%) $(%XPROGRAM%_OBJECTS) $(%XPROGRAM%_LDADD)

This will break "make -n", won't it?  Some sort of test of $(MAKEFLAGS)
is probably required in this command.

Cheers,
  Nick

Re: configure: error: cannot find install-sh, install.sh, or shtool in "." "./.." "./../.."

[Nick Bowler]

On 2020-08-01, TomK  wrote:
> Thanks very much Karl.  Appreciate this feedback. You've answered alot
> of the lingering questions I had around this topic.  Much appreciated!
>
> Just for some continued open discussion, included some basic answers.
> Not meant to sway to one side or the other, just to understand reasoning
> behind what I see used.
>
> On 7/30/2020 5:05 PM, Karl Berry wrote:
[...]
>> I don't agree with "deprecated". Left quotes must continue to work
>> forever and there is every reason to use them, in shell code that must
>> be maximally portable.
>
> access.redhat.com/solutions/715363
>
> for i in `find /usr/include -type f `
>
> fails on very large results.  $() has a higher results limit.

This is the least of your problems with this construct.  It will also
perform word splitting and pattern expansion on the filenames, which is
almost certainly not intended or desired so if you need to do something
like this in a configure script it is probably better to find a portable
construct which does not have these problems (I suspect such a solution
will not use command substitution at all).

Even if $() was portable it does not improve things very much; consider
the following:

  % mkdir /tmp/cwd
  % echo hello >/tmp/cwd/totally_unrelated_file
  % cd /tmp/cwd
  % mkdir /tmp/test
  % mkdir /tmp/test/not_even_a_file
  % echo hello >'/tmp/test/* *'
  % for i in $(find /tmp/test -type f); do printf '%s\n' "$i"; done
  /tmp/test/* *
  /tmp/test/not_even_a_file
  totally_unrelated_file

Oops...

Re: configure: error: cannot find install-sh, install.sh, or shtool in "." "./.." "./../.."

[Nick Bowler]

On 25/07/2020, TomK  wrote:
> Out of curiosity and a bit on another topic.  Is the syntax written
> like this for compatibility reasons with other shells?  Or because it
> could get in the way of the parsers Automake uses?

Autoconf is primarily a portability tool and thus a key goal is for
generated configure scripts to run in pretty much any unix-like
environment that anyone could possibly want to run them on.

Although these days configure expects support for things like shell
functions and comments so you probably won't have much luck on the
original 1979 Bourne shell, but there are a lot of different ksh88
derivatives/workalikes still floating around which were the basis
for POSIX standardization and the default shell on basically every
unix-like environment you're likely to encounter outside of museums.

> Code snippet:
[...]
> In particular:
>
>
> "x$host_alias" != x;
>
>
> I know adding 'x' or another character prevents failures when a variable
> is empty but that's been deprecated for sometime.

I don't know of any shell that gets this wrong with empty variables.
But metacharacters are a problem.  For example, if we write:

  test "$a" != "hello"

then at least some shells, (e.g., heirloom-sh), fail if a="!" or
a="(", etc, even though it is valid according to POSIX:

  $ a='('
  $ test "$a" != "hello"
  test: argument expected
  $ echo $?
  1

Prefixing with a benign (usually "x") string neatly avoids this problem:

  $ test x"$a" != x"hello"
  $ echo $?
  0

It's simple enough to do that usually this pattern is applied everywhere
even in cases where it is not strictly necessary -- I don't need to know
whether or not host_alias can legitimately be set to '!'.

> [] is deprecated in favor of [[]]
> `` is deprecated in favor of $()

[[ ]] is not POSIX compliant, and won't work at all in many modern
shells including dash.

$() is POSIX but unfortunately not widely portable in practice, e.g.,
again in heirloom-sh:

  $ a=$(echo hello)
  syntax error: `a=$' unexpected

Cheers,
  Nick

bug#40699: "dist Hook" documentation in manual is incorrect or unclear about write permissions

[Nick Bowler]

On 18/04/2020, Vincent Lefevre  wrote:
> On 2020-04-18 15:04:08 -0600, Karl Berry wrote:
[...]
>> Also, not that I wrote any of this, but it seems to me that the
>> pervasive assumption is that the automake user in fact owns the file
>> trees in question. Thus rm -rf should work even if it's mode zero.
>
> No, it doesn't. At least not with
>
>   rm (GNU coreutils) 8.30
>
> i.e. it does not change the permissions in order to make the removal
> work recursively (the -f just means that it ignores nonexistent files
> and that it will never prompt for confirmation).
>
>> Of course it won't work on a read-only source tree, but, well, it's just
>> an example ... --thanks, karl.
>
> It's just an example, but it should be correct (at least if there are
> no warnings about things that have been dropped to make it simpler).
> A user may want to copy-paste this example, and he will get something
> incorrect.

I guess the note about dist-hooks when packaging from a read-only srcdir
was added because of my bug report[1].

I have never noticed read-only _directories_ in the distdir before (I
have assumed they are always created read-write so that Automake can
put distributed files in them) but I also don't ever put directory
names in EXTRA_DIST which is the case specifically called out in the
documentation.

I think it's kind of weird that distribution permissions can depend on
how the package tarball was unpacked so since then I always use
dist-hook to fix up permissions in $(distdir) before anything else.

[1] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=10878

Cheers,
  Nick

Re: [PATCH] up_to_date_p: treat equal mtime as outdated.

[Nick Bowler]

On 2020-04-13, Paul Eggert  wrote:
> I just checked, and GNU Make uses high-resolution file timestamps when
> available, and considers a file to be up-to-date if it has exactly the same
> timestamp as its dependency. I suspect that this is because Makefile rules
> like
> this:
>
> a: b
>   cp -p b a
>
> would otherwise cause needless work if one ran 'make; make'.

If the Autoconf manual's portability notes are to be believed, then
such a rule is really not a good idea because on filesystems suporting
subsecond timestamp precision some "cp -p" implementations can create
the destination file with a strictly older timestamp than the source.

It's worth pointing out that even on filesystems supporting high-
precision timestamps, actual mtime updates usually have significantly
lower resolution (typically on the order of a millisecond or so).
Sufficiently fast make rules can frequently create many files with
exactly the same mtime.  Therefore considering equal timestamps to be
"up-to-date" is really the only reasonable option.

In cases where it is required to create two files with distinct mtimes,
the only real portable option I've found is to touch one of the files
in a loop and compare timestamps until they are different.

Cheers,
  Nick

Re: Installing something nonstandard in $(libdir)

[Nick Bowler]

On 2020-02-07, Tom Tromey  wrote:
>> "Zack" == Zack Weinberg  writes:
>
> Zack> Makefile.am:158: error: 'libfoo$(SOEXT).1' is not a standard library
> name
> Zack> Makefile.am:158: did you mean 'libfoo$(SOEXT).a'?
>
> Zack> and lib_DATA is the obvious alternative but that doesn't work either:
>
> Zack> Makefile.am:145: error: 'libdir' is not a legitimate directory for
> 'DATA'
>
> Zack> So, the question is, is there a lib_SOMETHING variable that I can use
> Zack> to install to $(libdir) arbitrary stuff that automake doesn't
> Zack> understand?  If not, is there some other option?
>
> I believe you can work around the checks by providing your own install
> directory variable, like:
>
> myexeclibdir = $(libdir)
> myexeclib_DATA = ...
>
> The "exec" is in the name to ensure that "make install-exec" installs
> these files, see (info "(automake) The Two Parts of Install") for this
> detail.

Nice!

The install-exec versus install-data was actually why I didn't suggest a
similar trick, I had no idea that simply putting "exec" in the directory
variable name has this effect.

Learn something every day!

Cheers,
  Nick

Re: Installing something nonstandard in $(libdir)

[Nick Bowler]

Hi Zack,

On 2/6/20, Zack Weinberg  wrote:
> For reasons too complicated to get into here, I have been
> experimenting with building shared libraries in an autoconf+automake
> build *without* using libtool.  [Please do not try to talk me out of
> this.]  I have something that works correctly on ELF-based operating
> systems with GCC, *except* for installation, where automake is
> refusing to do what I want.
[...]
> So, the question is, is there a lib_SOMETHING variable that I can use
> to install to $(libdir) arbitrary stuff that automake doesn't
> understand?  If not, is there some other option?

You can use an install-exec-hook to install your libraries.

Hope that helps,
  Nick

Re: Supporting build rules for grouped targets

[Nick Bowler]

On 2020-01-20, Markus Elfring  wrote:
> Variants of the make software support build rules for grouped targets.
>
> Examples:
> * 
> https://www.gnu.org/software/make/manual/html_node/Multiple-Targets.html#Rules-with-Grouped-Targets
> * 
> https://docs.oracle.com/cd/E86824_01/html/E54763/make-1s.html#REFMAN1make-1s-usag
>
> How can feature checks be achieved for such functionality around safer
> management of desired dependencies?

I don't understand exactly what you are asking, but the Automake manual
has a section[1] on how to write portable makefile rules for programs
that produce multiple output files.

I normally use the dedicated witness file and deletion-recovery rules
without locks which is fairly simple and sufficient for most cases.

[1] https://www.gnu.org/software/automake/manual/automake.html#Multiple-Outputs

Cheers,
  Nick

Re: How to install data / lib in more than 1 place?

[Nick Bowler]

On 12/11/19, Georg-Johann Lay  wrote:
>> On Tue, 10 Dec 2019, Georg-Johann Lay wrote:
[...]
>>> Will this also work with same file names? Like
>>>
>>> avrfoo_LIBRARIES = libfoo.a
>>>
>>> avrbar_LIBRARIES = libfoo.a
>>>
>>> or would that confuse the tools?
[...]
> It appears to work though, and even if the libs are built multiple
> times, I could reduce the number of Makefile.am's from ~1200 to ~250.

Another option is to use an install hook to copy a file installed in one
location into all the other locations.  Using a hook would also enable you
to e.g., link the files instead of copying them when that is possible.

Cheers,
  Nick

Re: Make -j to Compiler Question

[Nick Bowler]

Hello Nick,

On 2019-09-21, Nicholas Krause  wrote:
> I'm currently looking on and continuing the palleraling of gcc. There
> was a discussion about if its possible to link to make -j to split the
> tasks if possible. If so how and what is the easiest way to get this
> info into

Assuming you're talking about interacting with the GNU make jobserver, I
suggest asking this question on the GNU make list[1], after reading the
relevant documentation on the jobserver[2].

There are other make implemetations (e.g. NetBSD make) which support
parallelism but probably use a different method.

[1] https://lists.gnu.org/mailman/listinfo/help-make
[2] https://www.gnu.org/software/make/manual/html_node/POSIX-Jobserver.html

Cheers,
  Nick

Re: BUILT_SOURCES called on `make dist` even if the built sources should not be included in the dist

[Nick Bowler]

Hi Jerry,

On 9/17/19, Jerry Lundström  wrote:
> This problem seems to have been introduced in v1.16 with:
>
> - "./configure && make dist" no longer fails when a distributed file
> depends on one from BUILT_SOURCES.
>
> And what I can see in the Makefile output is that $(BUILT_SOURCES) has
> been added to distdir.
>
> I can't really see how this change got approved, isn't the point of
> BUILT_SOURCES to be sources built when building!?  Including them into
> distributions seems wrong.

I'm not sure exactly what the problem you are having is, because ...

[...]
> Here is an example:
>
> ```
> bin_PROGRAMS = test
>
> EXTRA_DIST = ext/TEST
>
> test_SOURCES = main.c
> test_LDADD = ext/built.o
>
> BUILT_SOURCES = ext/built.o
>
> ext/built.o:
>   echo "int sdkjfhskjhfskjd(void){ return 0; }" > ext/built.c
>   gcc -c ext/built.c -o ext/built.o
>
> CLEANFILES = ext/built.c ext/built.o
> ```

... I just ran this example with Automake 1.16.1 and neither ext/built.c
nor ext/built.o are included in the distribution tar file generated by
'make dist'.

So it seems to be working exactly as you wanted it to work.

Cheers,
  Nick

bug#35848: Should automake use gmake by default if exists?

[Nick Bowler]

Hello,

On 5/22/19, libor.buk...@oracle.com  wrote:
> On 5/21/19 5:37 PM, Nick Bowler wrote:
>> On 5/21/19, libor.buk...@oracle.com  wrote:
>>> automake expects GNU make to support dependency tracking.
>>>
>>> On Solaris it works well if MAKE variable is set to gmake during the
>>> configuration, otherwise, it fails with the following error.
>>>
>>> config.status: error: Something went wrong bootstrapping makefile
>>> fragments for automatic dependency tracking.  Try re-running configure
>>> with the '--disable-dependency-tracking' option to at least be able to
>>> build the package (albeit without support for automatic dependency
>>> tracking).
>>> See `config.log' for more details
[...]
> In general, the dependency tracking works on Solaris. However, some
> packages (e.g., jq, flex, graphviz) expect GNU make since Makefile.am
> files are not compatible with Solaris make (conditional assignment
> operator, ...). If it is the case, one would expect a hint to use GNU
> make, therefore, the update of the error message could be the best way
> to go.

Oh, now this problem makes sense.

Recent versions of Automake (1.16+) use a make rule to generate the
dependency stubs.  So if the package uses GNU extensions in Makefile.am
then the default "make" might not be able to execute that rule, leading
to this failure to generate the stubs by config.status.

In this case, since those packages require GNU make to work, it would
probably be ideal (short of making their makefiles portable...) if those
packages added a check to their configure scripts that $MAKE supports
whatever extensions are required to build the package.  This would enable
much more accurate error messages (e.g., "$MAKE does not support conditional
assignment required by this package, please try a different make").

But improving this error message is probably a good idea anyway because
I agree "Something went wrong" gives no hint to the user as to what the
problem is.

Cheers,
  Nick

bug#35848: Should automake use gmake by default if exists?

[Nick Bowler]

On 5/21/19, libor.buk...@oracle.com  wrote:
> automake expects GNU make to support dependency tracking.
>
> On Solaris it works well if MAKE variable is set to gmake during the
> configuration, otherwise, it fails with the following error.
>
> config.status: error: Something went wrong bootstrapping makefile fragments
>  for automatic dependency tracking.  Try re-running configure with the
>  '--disable-dependency-tracking' option to at least be able to build
>  the package (albeit without support for automatic dependency
> tracking).
> See `config.log' for more details
>
> My suggestion is to use gmake by default if the command exists,
> otherwise fallback to make command. It could improve the overall user
> experience since the user would not need to search a solution
> (./configure MAKE=gmake) every-time this error is encountered.

I doubt this will actually help users because the user probably
won't know they need to explicitly run "gmake", and if the package
is configured with MAKE=gmake and they run plain "make" it is likely
to also fail.

The only "fancy" feature that Automake's dependency tracking should
require of the make implementation is an "include" functionality, for
example any make which implements the "include" directive from POSIX
should suffice.  I'm pretty sure Solaris make has such a function, so
I'd expect automatic dependency tracking to work with Solaris make.

If it is not working, then there might be a real bug to fix in the
dependency tracking feature in Automake.

> Note that we used automake 1.16.1 on Solaris 11.4 but the issue is also
> reproducible with older automake versions.

This doesn't sound accurate because the error you encountered does not
exist in automake versions before 1.16.  Older versions may have a
/different/ failure, of course...

Cheers,
  Nick

bug#35762: (No Subject)

[Nick Bowler]

On 2019-05-16, howaboutsyne...@protonmail.com
 wrote:
> On Thursday, May 16, 2019 8:12 PM, Eric Blake  wrote:
[...]
>> : >"$log_file"
> Very cool! Thank you very much! I'll use that in my local automake
> rebuild.  Where can I find more information about ":" ? it seems kinda
> tough to search for.

The relevant standards are a good place to start:

  
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#colon

Cheers,
  Nick

bug#35526: misleading documentation about flag variable ordering

[Nick Bowler]

On 5/1/19, Daniel Kahn Gillmor  wrote:
> https://www.gnu.org/software/automake/manual/automake.html#Flag-Variables-Ordering
> says:
>
> The reason ‘$(CPPFLAGS)’ appears after ‘$(AM_CPPFLAGS)’ or
> ‘$(mumble_CPPFLAGS)’ in the compile command is that users should
> always have the last say.
>
> The implication here is that for any of the *FLAGS variables, later
> options are inherently more effective than earlier options.
>
> This is not the case universally.  For example -Wl,--as-needed is more
> effective if it shows up earlier in the command line.  (see
> https://bugs.debian.org/347650 for a libtool-related problem for that
> kind of ordering/re-ordering)

It's still mostly true, and it really is more about turning off options
that may be turned on by the package but the user does not want.  Even
in the case of --as-needed, if the package had AM_LDFLAGS = -Wl,--as-needed
the user could (in principle) cancel it with LDFLAGS = -Wl,--no-as-needed,
since the user flags appear later.  If these were reversed then the
package option would override anything the user tried to do.

But it is true that this is not universal.  Not every compiler option
has a way to cancel its effect later on the command line.  In the case
of CPPFLAGS, -I options are an example where later options are in some
sense "less effective" than earlier options.

But on the other hand, -D options are commonly placed in CPPFLAGS and
in that case their effects can be canceled and/or changed by later
options.

Since all the user-provided flags are lumped together in a single
variable they have to go _somewhere_ as a unit, and convention puts them
after the corresponding package-set flags, so compilation commands look
something like:

  cc \
   

and linking (when the C compiler is used for linking) looks like:

  cc \
   

(Note that the convention puts LDFLAGS before libraries so things
like LDFLAGS=-Wl,--as-needed _should_ work as expected, libtool
issues notwithstanding).

Obviously this scheme is not perfect in every possible scenario, but
the only way to handle every possible scenario is probably to have the
user to write all the compiler commands manually, in which case there
is no point in having a build system.

> It would be good if the documentation could avoid implying something
> that isn't the case, because that dubious suggestion casts doubt on the
> rest of the documentation (or on the user's understanding of the rest of
> the buildsystem)

The manual can always be improved.  Do you have a suggestion for how to
write this paragraph to make it better?

Cheers,
  Nick

Re: Help with -Werror

[Nick Bowler]

Hello,

On 2019-04-24, Phillip Susi  wrote:
> It seems like every time I go back to try to do somoe work on the parted
> sources I run into a failure to compile due to some silly warning or
> other and -Werror being enabled.  This time it is from a generated
> source file made by gperf.  Is this set by default these days in
> automake?  Because I can not figure out how it is being used.
> Makefile.am does not seem to have anything to turn it on.  Makefile sets
> CFLAGS_WERROR=-Werror, but I can see nothing that references
> CFLAGS_WERROR anywhere.  How does this variable end up being passed to
> gcc?
>
> I am tempted to just disable -Werror completely, but at the very least
> it should be disabled for BUILT_SOURCES since you can't really fix the
> warnings there.  Any advice on how to do this?

Automake does not add -Werror to the default C compiler flags, and it
does not do anything with a CFLAGS_WERROR variable.

If that is happening for a particular package, then it is because the
package authors did something to make it happen.  So probably any
questions about it happening in parted should be taken up with the
parted maintainer(s)...

For the most part, -Werror is a developer tool which will only cause
problems for users, so my strong recommendation is that it should
never appear in package releases, but not everybody subscribes to
that philosophy...

Cheers,
  Nick

Re: Is it possible to set the permission bits used by the default install target in a Makefile.am?

[Nick Bowler]

Hello Craig,

On 2019-03-13, Craig Sanders  wrote:
> Is it possible to set the permission bits used by the default install
> target in a Makefile.am?
>
> To help try and illustrate what I mean, I present a code snippet from one
> of my Makefie.am files.
>
>>> Begin code snippet >>
>
> gimpdir = ${prefix}
>
> gimp_SCRIPTS = scaleAndSetSize.py \
>ScaleAndSetSizeClass.py
>
> .PHONY: install
> install:
>
> mkdir -p ${prefix}
> ${INSTALL} -m 544 scaleAndSetSize.py ${prefix}
> ${INSTALL} -m 444 ScaleAndSetSizeClass.py ${prefix}
>
> << End code snippet <<
>
> My problem with this code snippet is - I don't like the fact that I have
> overridden the default install target to get the files installed with the
> permission bits set the way I want. Rather, I'd like to have the default
> install target do the install work for me, using permission bits that I
> would like to specify. Does anybody know if this is possible?

Automake uses INSTALL_SCRIPT to install scripts, which is normally provided
by AC_PROG_INSTALL from Autoconf (and is set to INSTALL).  You can set
this explicitly in Makefile.am to something different (or change the
value in configure).

However, that's probably a pain because you want different permissions
for different files.

One option would be to use both xxx_DATA and xxx_SCRIPTS, which are
installed by INSTALL_DATA and INSTALL_SCRIPT, respectively (this is the
only practical difference between xxx_DATA and xxx_SCRIPTS).  You can
then adjust those variables separately as desired.

Alternately you can use install-local[1] instead, to get more flexibility
but without replacing the standard "install" target.  Try to respect
DESTDIR as well, and prefer $(MKDIR_P) over open-coding mkdir -p.
For example (totally untested):

  544_scripts = scaleAndSetSize.py
  444_scripts = ScaleAndSetSizeClass.py

  install-local: install-my-scripts
  install-my-scripts:
$(MKDIR_P) "$(DESTDIR)$(gimpdir)"
$(INSTALL) -m 544 $(544_scripts) "$(DESTDIR)$(gimpdir)"
$(INSTALL) -m 444 $(444_scripts) "$(DESTDIR)$(gimpdir)"
  .PHONY: install-my-scripts

Consider a corresponding uninstall target as well:

  uninstall-local: uninstall-my-scripts
  uninstall-my-scripts:
test ! -d "$(DESTDIR)$(gimpdir)" && cd "$(DESTDIR)$(gimpdir)" && \
  rm -f $(544_scripts) $(444_scripts)
  .PHONY: uninstall-my-scripts

Something like that should be just as good as what you get from the
built-in "install" rule (be sure to test with 'make distcheck').

Hope that helps,
  Nick

Re: Parallel builds with some ordering constraints

[Nick Bowler]

On 12/29/18, Kip Warner  wrote:
> On Sat, 2018-12-29 at 16:10 -0500, Nick Bowler wrote:
[...]
>>   all_tests_except_start = test1.log test2.log test3.log test-
>> stop.log
>>   all_tests_except_stop = test-start.log test1.log test2.log
>> test3.log
>>
>>   $(all_tests_except_start): test-start.log
>>   test-stop.log: $(all_tests_except_stop)
>
> [snip]
>
>> Hope that helps,
>
> Almost! The problem is with the last rule you defined because a rule to
> generate test-stop.log would have already been generated by Automake
> and this would override it.

Huh.  That probably means the example in the manual is broken too.

Anyway, the solution is straightforward.  Rules in Makefile.am only
override Automake-supplied rules if they are spelled _exactly_ the same
way.  So you just need to change the spelling in the rule, usually by
using a variable like:

  test_stop_log = test-stop.log
  $(test_stop_log): blah blah blah

and the Automake-generated rule for test-stop.log should be emitted
normally.

Hope that helps,
  Nick

Re: Parallel builds with some ordering constraints

[Nick Bowler]

Hello,

On 2018-12-29, Kip Warner  wrote:
> Parallel builds work fine for my build tree of a system daemon I am
> developing. I have unit tests in the form of check_SCRIPTS and
> check_PROGRAMS.
>
> These unit tests, however, can only be partially parallelized because
> there needs to be some ordering constraints.

OK, I am assuming you are using the Automake parallel-tests feature.

> I have a unit test in check_SCRIPTS which starts the daemon via init.d.
> The daemon writes out a pid file. There is another unit test script to
> stop it via init.d.
[...]
> For ensuring the stop daemon script at least runs after the start
> script when make computes the dependency graph, I have a (working)
> hack. I simply make the stop daemon target shell script depend on the
> test log of the start script. This works, but it could break in the
> future with newer Automakes.
>
> But in any event, I still don't know what I can do for the binary
> check_PROGRAMS that test the daemon itself to constrain them to run
> between the former two?

So there's no problem with building the programs, the issue is just in
the execution order of the test cases?  You have one test case which
must run before all other test cases, and one test case which must run
after all other test cases.

The documented method to ensure ordering between two (or more) test
cases in the parallel test harness is to put explicit make prerequisites
between the log files[1], e.g. (totally untested):

  all_tests_except_start = test1.log test2.log test3.log test-stop.log
  all_tests_except_stop = test-start.log test1.log test2.log test3.log

  $(all_tests_except_start): test-start.log
  test-stop.log: $(all_tests_except_stop)

[1] 
https://www.gnu.org/software/automake/manual/automake.html#Parallel-Test-Harness

Hope that helps,
  Nick

Re: Recreate the config files

[Nick Bowler]

On 12/7/18, Deepa Ballari  wrote:
> I'm trying to add new options to newlib.I get all different sort of
> errors when I run autoconf,automake..
> How can I recreate the config files and sync with
> automake (GNU automake) 1.15, autoconf (GNU Autoconf) 2.69, libtool
> (GNU libtool) 2.4.6 ?
>
> List of errors:
> 1)newlib/libc$ autoconf
> configure.in:37: error: possibly undefined macro: AM_CONDITIONAL
>   If this token and others are legitimate, please use m4_pattern_allow.
>   See the Autoconf documentation.
> configure.in:71: error: possibly undefined macro: AC_LIBTOOL_WIN32_DLL
> configure.in:72: error: possibly undefined macro: AM_PROG_LIBTOOL

I expect the issue is that you are expected to run aclocal first.  You
can sometimes use autoreconf which knows how to run several of the GNU
build tools in the right sequence.

However, the newlib project may have additional requirements to
bootstrap their build system, as many projects use additional functions
outside of the basic GNU set.  This would presumably be documented
somewhere in the newlib documentation, (a lot of projects have a script
to do this, sometimes called 'bootstrap').

You will likely get better help on the newlib mailing list.

Cheers,
  Nick

bug#33594: TEXINFO_TEX file is not in dist

[Nick Bowler]

Hi Ben,

On 2018-12-02, Ben Elliston  wrote:
> When setting TEXINFO_TEX to a different location (eg,
> doc/texinfo.tex), the file is no longer distributed in the dist
> tarball. Shouldn't it be?

This actually is the documented behaviour[1]:

  ... if you set the TEXINFO_TEX variable (see below), it overrides
  the location of [texinfo.tex] and turns off its installation into
  the source [by automake --add-missing] as well as its distribution.

There's no rationale given but I guess that disabling distribution
when tweaking these things is more flexible: it is easier to add the
file to EXTRA_DIST if you want it distributed than it is to prevent
distribution otherwise.

[1] https://www.gnu.org/software/automake/manual/automake.html#Texinfo

Cheers,
  Nick

bug#33166: automake --add-missing does not install config.rpath when AM_ICONV is used

[Nick Bowler]

On 10/26/18, Hans-Bernhard Bröker  wrote:
> Am 26.10.2018 um 17:06 schrieb Stuart Caie:
>> Technically, AM_ICONV is not distributed with automake, and neither is
>> is config.rpath. However, automake recognises config.rpath as a special
>> file to distribute nonetheless.
>
> That's not really automake doing that.  That's done by gettextize and
> the autofoo code it adds to your project.

To be fair, Automake does have a special handling for config.rpath:
it will distribute a file with this name automatically if it is
present in the source tree, just like it does with files named
README, COPYING, etc.  But that's the extent of it.

(Aside: I've never liked this particular behaviour of Automake.
Especially in "foreign" mode, it's easy to miss adding files to
version control and it's possible nobody notices until someone
other than the original author runs "make dist", gets no error
whatsoever, and publishes a tarball without any README).

bug#33085: using myprog_CXXFLAGS with VPATH prevents Makefile from finding sources

[Nick Bowler]

Hello,

On 10/18/18, Julien COURTAT  wrote:
> Here's my bug report, I'm building my software out of the source tree, this
> is called parallel build (very nice feature).
> The way to reproduce the issue is very simple, set myprog_CXXFLAGS = -DANY
> and VPATH=@MYDIR@ and the corresponding Makefile won't find my source file.
> Replace myprog_CXXFLAGS by AM_CXXFLAGS and it works well.
> Autoconf 2.69, automake 1.13.4
[...]
> Makefile.am :
> bin_PROGRAMS=myprog
> myprog_SOURCES  = Main.cpp
> myprog_CXXFLAGS = -DANY
> AM_CPPFLAGS  = -I@MYPROG_PATH@
> VPATH=@MYPROG_PATH@

Don't set VPATH like this.  It supersedes the definition supplied by
Automake and is almost certainly the cause of your problems.

The GNU build system has built-in support for out-of-tree builds like
this.  What you do is cd to a new directory, then run configure from
there.  For example:

  mkdir build && cd build && /path/to/configure [blah blah] && make

Then everything should Just Work™ (provided you don't override VPATH
like the above).  You can use $(top_srcdir), $(srcdir), $(top_builddir)
and $(builddir) to refer to files in particular locations (e.g. for
include paths).

See the Automake manual, §2.2.6 "Parallel Build Trees (a.k.a. VPATH
Builds)"[1] for more information.

[1] https://www.gnu.org/software/automake/manual/automake.html#VPATH-Builds

Hope that helps,
  Nick

bug#32269: Expanding variables

[Nick Bowler]

On 7/25/18, Philip Prindeville  wrote:
> Since automake/autoconf are responsible for generating Makefiles, we
> could add something like:
>
> val.%:
>   @$(if $(filter undefined,$(origin $*)),\
>   echo "$* undefined" >&2, \
>   echo '$(subst ','"'"',$($*))' \
>   )
>
> var.%:
>   @$(if $(filter undefined,$(origin $*)),\
>   echo "$* undefined" >&2, \
>   echo "$*='"'$(subst ','"'\"'\"'"',$($*))'"'" \
>   )
>
> toward the bottom.

I'm not exactly sure what the purpose of this is, but it doesn't
look like portable code, so probably not something Automake should
be generating.

But don't let that stop you.  You can add basically anything you'd
like, including the above rules, to your project's Makefile.am and
Automake will copy it through to the output.

Cheers,
  Nick

Re: PATCH: silence Shellcheck warning

[Nick Bowler]

On 7/21/18, Ben Elliston  wrote:
> This patch silences a warning from Shellcheck about using old-style
> `...` command substitutions.
[...]
> commit 4d35c7aae97234bf055519075ef03cd4090a1dfc
> Author: Ben Elliston 
> Date:   Sun Jul 22 08:22:44 2018 +1000
>
> * missing: Use $(..) command substitution syntax.

Several shells do not support $(...) substitutions so portable scripts
like 'missing' must always use backticks.  I would suggest that this
linter warning is spurious if not outright counterproductive.

Cheers,
  Nick

bug#31728: Automake and AM_WITH_DMALLOC; endorsing proprietary software?

[Nick Bowler]

Hi,

I have some doubts about the automake-provided macro AM_WITH_DMALLOC.
This appears to be a development tool which could be useful to help
programmers debug their packages.  It has a very brief description
in the Automake manual in section 6.4.1 "Public Macros"[1], including
a link to the dmalloc website.

The trouble is that dmalloc appears to be non-free: the license does
not seem to permit distribution for a fee (see below).  This macro in
Automake doesn't seem to exist for any real portability purpose, but
rather it only adds options that help extend program functionality
with this library.

Perhaps I am mistaken, but I wonder if this macro has a place in a
GNU package like Automake.  The manual entry may encourage developers
to use this macro/library.  For an example of this, GNU make 4.2.1
did make use of the AM_WITH_DMALLOC macro, so in turn that package's
configure --help output suggests this tool for debugging.

The license text found in the latest dmalloc 5.5.2 release is this:

  Copyright 2000 by Gray Watson

  Permission to use, copy, modify, and distribute this software for
  any purpose and without fee is hereby granted, provided that the
  above copyright notice and this permission notice appear in all
  copies, and that the name of Gray Watson not be used in advertising
  or publicity pertaining to distribution of the document or software
  without specific, written prior permission.

  Gray Watson makes no representations about the suitability of the
  software described herein for any purpose.  It is provided "as is"
  without express or implied warranty.

The permission grant is almost identical to the ISC license except that
"and/or" is changed to "and" and, more importantly, the words "with or
without fee" are changed to "and without fee".

[1] 
https://www.gnu.org/software/automake/manual/automake.html#index-AM_005fWITH_005fDMALLOC

Thoughts?

Cheers,
  Nick

Re: _SOURCES files in sub-directories lead to make distdir failure

[Nick Bowler]

Hi,

On 1/24/18, netfab  wrote:
> Into that project, there's a subdirectory to build a library using
> libtool-2.4.6. The source code of this library is organized into
> sub-directories, like this :
>>  mylib/makefile.am
>>  mylib/aaa.cpp
>>  mylib/aaa.h
>>  mylib/foo/bbb.cpp
>>  mylib/foo/bbb.h
>>  mylib/bar/ccc.cpp
>>  mylib/bar/ccc.h

Looks fine so far.

> The makefile.am for this lib contains :
>> libmyLIB_la_SOURCES = \
>>  aaa.cpp aaa.h \
>>  foo/bbb.cpp foo/bbb.h \
>>  bar/ccc.cpp bar/ccc.h

This looks fine too.

> I'm initializing automake with :
>> AM_INIT_AUTOMAKE([subdir-objects])

Also fine.

> When building the whole project, it works fine.
> However, when running :
>> make distcheck
>
>
> Is fails like following, and I don't see how to fix this :
>> make[5]: Entering directory '/path/to/project/build/src/lib/mylib'
>> make[5]: *** No rule to make target 'foo/bbb.h ', needed by 'distdir'. Stop.
>
> Any advice ? Thanks.

The thing that distcheck is testing here is that the package can be
built with separate source and build directories.  It appears that this
functionality is broken in your package, and distcheck is notifying you
of this fact.

Since you are hitting this problem now, you have probably only been
testing in-tree builds until this moment.

Unfortunately your provided snippets are not complete working code, and
I expect the error involves part of the code you have not shown us.

Cheers,
  Nick

Re: [PATCH] "make dist" did not depend on $(BUILT_SOURCES)

[Nick Bowler]

On 2017-11-28 18:13 -0800, Jim Meyering wrote:
> On Tue, Nov 28, 2017 at 12:45 PM, Nick Bowler <nbow...@draconx.ca> wrote:
> > The Automake manual unequivocally states that BUILT_SOURCES files are
> > generated only when running 'make all', 'make check' or 'make install'.
> >
> > So if they are going to be generated on 'make dist' as well, then the
> > manual needs a corresponding update.
> 
> Hi Nick,
> Thanks for the suggestion, but I do not think it is desired. "make
> dist" is already defined as building everything that goes into the
> distribution tarball, and that implies it must also build anything
> (e.g., from BUILT_SOURCES) that happens to be required to do that.

I agree that it *should* but not that it *must*, because BUILT_SOURCES
explicitly (by design) bypasses the usual prerequisite mechanisms.

If you use normal prerequisites instead of BUILT_SOURCS everything
works just fine wrt. distribution, of course, and is the approach I
would personally recommend in all cases.

> Perhaps more importantly, this is an implementation detail that feels
> like it should not be made part of the contract that the documentation
> provides ...

But now with the change applied, as it stands the documentation is
simply wrong.  For example, this passage from the manual (§9.4 Built
Sources):

  "... BUILT_SOURCES is honored only by ‘make all’, ‘make check’ and
  ‘make install’."

is no longer true.  This error can be corrected without explicitly
documenting the new behaviour, for example by making the list of
targets non-exhaustive in nature.

Perhaps something like:

  ... BUILT_SOURCES is honored only by certain targets, including ‘make
  all’, ‘make check’ and ‘make install’.

Although not mentioning distribution at all means that someone reading
this section is left to figure out for themselves if and how these two
Automake features work together...

> ... in case some day automake tightens up "make dist" so it builds
> only those BUILT_SOURCES files that are actually required to build
> the tarball components.

There is need to worry about this ever happening, because computing
such a subset of BUILT_SOURCES is impossible in general.

Cheers,
  Nick

Re: [PATCH] "make dist" did not depend on $(BUILT_SOURCES)

[Nick Bowler]

Hi Jim,

On 2017-11-28 11:21 -0800, Jim Meyering wrote:
> Date: Thu, 20 Mar 2014 12:31:32 -0700
> Subject: [PATCH] "make dist" did not depend on $(BUILT_SOURCES)
> 
> * lib/am/distdir.am (distdir-am): New intermediate target.
> Interpose this target between $(distdir) and its dependency
> on $(DISTFILES), so that we can ensure $(BUILT_SOURCES) are
> all created before we begin creating $(DISTFILES).
[...]
>  NEWS   |  3 +++
>  lib/am/distdir.am  |  7 --
>  t/dist-vs-built-sources.sh | 56 
> ++
>  t/list-of-tests.mk |  1 +
>  4 files changed, 65 insertions(+), 2 deletions(-)
>  create mode 100644 t/dist-vs-built-sources.sh

The Automake manual unequivocally states that BUILT_SOURCES files are
generated only when running 'make all', 'make check' or 'make install'.

So if they are going to be generated on 'make dist' as well, then the
manual needs a corresponding update.

Otherwise this looks like a useful improvement.

Cheers,
  Nick

Re: Finding #includes from a yacc .y.

[Nick Bowler]

Hi Ralph,

On 2017-11-20, Ralph Corderoy  wrote:
>> It seems wrong for foo.y to have to `#include
>> "path/from/root/to/bar.h" since that means it has to alter if they
>> move around the hierarchy.  Is there another way?
>
> I can be more precise having dug into this project a bit.
> Currently, it has
>
> sbr_libmh_a_CPPFLAGS = ${AM_CPPFLAGS} -I./sbr

This relative include path refers to the current working directory
of the compiler, which is normally the build directory and is thus
essentially equivalent to -I$(builddir)/sbr ...

> Would it be wrong or a misuse of top_srcdir to change that to
>
> sbr_libmh_a_CPPFLAGS = ${AM_CPPFLAGS} -I$(top_srcdir)/sbr

... so if your headers are in the source directory, as is typical,
then something like this is perfectly sensible.

Cheers,
  Nick

Re: warning: TEST_LDFLAGS' is defined but no program or library has 'TEST' as canonical name

[Nick Bowler]

Hi,

On 2017-11-20, Thomas Martitz  wrote:
> here's some quite annoying warning. I'm trying to define a variable
> TEST_LDFLAGS that multiple programs use. There is no program named TEST.
> The same works fine with TEST_CFLAGS (i.e. no warning is displayed).
>
> Here's the warning:
>
> Makefile.am:4: warning: variable 'TEST_LDFLAGS' is defined but no program
> or
> Makefile.am:4: library has 'TEST' as canonical name (possible typo)

I'm surprised there is no warning with CFLAGS; it appears this warning
is issued for mumble_SOURCES, LIBADD, LDADD, LDFLAGS and DEPENDENCIES
only.

> Here's the Makefile.am
>
> TEST_CFLAGS = -g
> TEST_LDFLAGS = -Wl,-z,defs
>
> bin_PROGRAMS = test
>
> test_CFLAGS = $(TEST_CFLAGS)
> test_LDFLAGS = $(TEST_LDFLAGS)
>
> Is this known? Is there a workaround? Can I ignore the warning?

If you were to later add a program called TEST, then the results could
be surprising.  But you can certainly ignore the warning if you'd like.

Alternately you can perhaps use a different name that does not conflict
with the Automake naming structure.  Perhaps LDFLAGS_FOR_TEST?

You can disable the warning outright with -Wno-syntax (but this might
disable more than you'd like).

Finally, this warning is not issued for variables substituted by configure.

Cheers,
  Nick

Re: Per-Object Flags for Autotool C++ library?

[Nick Bowler]

On 11/4/17, Jeffrey Walton <noloa...@gmail.com> wrote:
> On Sat, Nov 4, 2017 at 3:56 PM, Nick Bowler <nbow...@draconx.ca> wrote:
>>> EXTRA_libcryptopp_la_DEPENDENCIES listing the objects worked for Linux
>>> and OS X, but not Solaris. For Solaris I needed to drop the leading
>>> `EXTRA`, and use just `libcryptopp_la_DEPENDENCIES`.
>>
>> Is that just because you happen to be running an antique version of
>> Automake on the Solaris machine?
>
> Well, I'm not sure. Is this considered old:
>
> $ automake --version
> automake (GNU automake) 1.11.2

Well, it's coming up on its 6th birthday :)

> One of our driving principles is "things just work". We don't want
> library users inconvenienced or installing extra software. They should
> be able to sit down at their computer, run configure, and everything
> should work as expected.
>
> If something does not work as expected then it becomes our problem.
> We are expected to find workarounds so library users are not
> inconvenienced.

Library users shouldn't be running Automake at all, because when you
distribute a package all of the generated files are included (and do
not depend on Automake).

e.g., if you create your package with the latest versions then it
should "just work" on Solaris.

Cheers,
  Nick

Re: Per-Object Flags for Autotool C++ library?

[Nick Bowler]

Hello,

> EXTRA_libcryptopp_la_DEPENDENCIES listing the objects worked for Linux
> and OS X, but not Solaris. For Solaris I needed to drop the leading
> `EXTRA`, and use just `libcryptopp_la_DEPENDENCIES`.

Is that just because you happen to be running an antique version of
Automake on the Solaris machine?

Cheers,
  Nick

Re: Per-Object Flags for Autotool C++ library?

[Nick Bowler]

On 11/3/17, Jeffrey Walton  wrote:
> On Thu, Nov 2, 2017 at 6:04 PM, Jeffrey Walton  wrote:
>> I'm working on adding Autotools to a C++ library and test program. My
>> Automake.am has:
>>
>> 
>> ...
>
> I believe I applied Nick and Mathieu correctly. The project is
> available at https://github.com/noloader/cryptopp-autotools . It
> includes the six Git commands to duplicate the issue.
>
> The new issue is, the compile stops after about 4 files are compiled.
> Here's the pastebin of `make V=1`: https://pastebin.com/nCYN2RHh. The
> error is also shown below.
>
> The linker is invoked for reasons unknown to me at the moment. Most of
> the objects are missing. I even deleted the project's directory and
> re-cloned to ensure they were not old artifacts hanging around.

For whatever reason it appears that the generated makefile has missing
prerequisites for libcryptopp.la.  I would expect everything listed in
LIBADD to end up as a prerequisite of the library.  This might require
some investigation to find out why that apparently did not happen in
your case.

Adding everything to EXTRA_libcryptopp_la_DEPENDENCIES might help as
a workaround, e.g.,

  EXTRA_libcryptopp_la_DEPENDENCIES = $(libcryptopp_la_LIBADD)

But this (or equivalent) should have happened automatically.

> I have no idea why a C compiler is being invoked in some places. I
> took great care to ensure Autoconf knew this was a C++ project, and
> not a C project. That's another problem I've been searching for an
> answer for.

It seems it decided to link the library using the C compiler because no
source files are specified for the library.  There may be (or should be)
a way to force it one way or the other, but an obvious workaround is to
specify at least one C++ source file in libcryptopp_la_SOURCES (could be
one of the real files or just a stub).  The _SOURCES objects will appear
earlier on the linker command line than any of the _LIBADD objects.

Cheers,
  Nick

Re: Per-Object Flags for Autotool C++ library?

[Nick Bowler]

Hi Jeffrey,

On 11/2/17, Jeffrey Walton  wrote:
> I'm working on adding Autotools to a C++ library and test program. My
> Automake.am has:
>
> lib_LTLIBRARIES = \
>libcryptopp.la
>
> libcryptopp_la_SOURCES = \
>cryptolib.cpp \
>cpu.cpp \
>integer.cpp \
>
>...
>
> cpu.cpp needs additional flags to enable ISAs on IA-32, Aarch64 and
> Power7/Power8. According to
> https://www.gnu.org/software/automake/manual/html_node/Per_002dObject-Flags.html,
> I need to add an additional library:
>
> CPU_FLAG = -msse2 -msse3 -mssse3
> libcpu_a_SOURCES = cpu.cpp
> libcpu_a_CXXFLAGS = $(CXXFLAGS) $(CPU_FLAG)

Note that you should not include $(CXXFLAGS) here.  CXXFLAGS is always
included (so with this it will duplicated on the command line, which
might be undesired by the user).

> Now that the objects are built we need to add libcpu.a back into
> libcryptopp.la in the exact position it would have been in if I could
> have specified per-object flags. The Automake manual gives an example
> of linking a program with disjoint libraries, but not adding the
> extraneous library back to the main (primary?) library at a particular
> position.
>
> The "in the exact position" is important.

Not too familiar with C++ stuff but I would be a bit concerned that
it might not be possible at all to force a particular link order for
the objects in the static version of the library.

Nevertheless for the shared library case you can probably achieve this
using several dummy libraries.  Something like this should work OK
(totally untested):

lib_LTLIBRARIES = libfoo.la
EXTRA_LTLIBRARIES = libdummy1.la libdummy2.la libdummy3.la

libdummy1_la_SOURCES = a.cpp b.cpp

libdummy2_la_SOURCES = c.cpp d.cpp
libdummy2_la_CXXFLAGS = -mstuff

libdummy3_la_SOURCES = e.cpp f.cpp

libfoo_la_SOURCES =
libfoo_la_LIBADD = $(libdummy1_la_OBJECTS) \
   $(libdummy2_la_OBJECTS) \
   $(libdummy3_la_OBJECTS)

and then the linking order should be a, b, c, d, e, f -- with c and d
compiled using your special flags.

Cheers,
  Nick

Re: Parallel build sometimes resulting in "fatal error: config.h: No such file or directory"

[Nick Bowler]

Hi Simon,

On 10/16/17, Simon Sobisch  wrote:
[...]
> Running without `make -j` always work but using parallel builds sometime
> break with the mentioned error.
[...]
> ~~~
> gcc -O2 -pipe -finline-functions -fsigned-char -Wall -Wwrite-strings
> -Wmissing-prototypes -Wno-format-y2k -U_FORTIFY_SOURCE
> -Wl,-z,relro,-z,now,-O1  /home/simon/gnucobol/cobc/../cobc/cobc.c   -o
> ../cobc/cobc
> /home/simon/gnucobol/cobc/../cobc/cobc.c:26:10: fatal error: config.h:
> No such file or directory
>  #include "config.h"
>   ^~
> compilation terminated.

I took a quick look at your project.  The problem is likely this bit
from cobc/Makefile.am:

  COBC = $(top_builddir)/cobc/cobc$(EXEEXT)

  cobc.1: [...] $(COBC)

The problem is that this COBC macro does not match the Automake generated
target name.  When it gets pulled in as a prerequisite for cobc.1 and the
file does not already exist, the built-in GNU make rule applies, which
produces cobc from cobc.c.

This is the wrong rule, so the compilation fails.

The prerequisites in make rules typically should match the target names
exactly.  In this case, it should be:

  cobc.1: [...] cobc$(EXEEXT)

Hope that helps,
  Nick

bug#28838: Authoritative answer about licensing and copyright of provided scripts and templates

[Nick Bowler]

On 2017-10-14, Joël Krähemann  wrote:
> I need some authoritative answer about copyright notices to be used
> for scripts and templates. The files generated by autoscan, autoconf,
> automake or alike.
>
> I need this information in order to proceed with a submission on
> savannah.gnu.org,
> https://savannah.gnu.org/task/index.php?14667

Since you posted this to the Automake list I will try to answer the
Automake part of this question.  Note that I am not a legal expert
and this is not legal advice (you won't find that on a volunteer
mailing list).

Automake works by combining code from your project (such as the code you
write in Makefile.am) which is subject to your copyright, together with
code provided by Automake (configure tests, makefile snippets, etc.)
which is subject to the FSF copyright.

So in a free software project, the resulting Makefile.in will need
compatible free software licenses for your parts and for the FSF-
copyrighted parts in order to be free.

Most of the code in Automake is released under the GPL version 2 or
later.  But the configure tests and generated Makefile.in fragments
use a simple permissive license (not the GPL), consisting of a warranty
disclaimer and a notice preservation requirement.  This is compatible
with most, if not all, free software licenses (just preserve the notice).

It should be sufficient to put your own copyright and license notices in
the Automake input files.  These will be copied into the Automake output
along with the FSF copyright and license notices.

Hope that helps,
  Nick

Re: No rule to make target 'bzr.mk', needed by 'all-am'

[Nick Bowler]

On 9/29/17, Sascha Manns  wrote:
> Am Freitag, den 29.09.2017, 16:26 +0200 schrieb Sascha Manns:
>> i have a project what provides a file called "bzr.mk". This isnt
>> generated and should just installed in $(datadir)/bzrmk.
>> [...]
>> bzrmk_DATA = bzr.mk
>>
>> But while building the package i'm getting:
>> Making all in src
>> make[3]: Entering directory '/build/bzrmk-1.2.1/src'
>> make[3]: *** No rule to make target 'bzr.mk', needed by 'all-
>> am'.  Stop.
>> make[3]: Leaving directory '/build/bzrmk-1.2.1/src'
>> Makefile:464: recipe for target 'all-recursive' failed
>
> I found it out. I'll did the from a tarball, generated with make dist.
> Now i changed the src/Makefile.am, and included the bzr.mk in
> EXTRA_DIST.

That'll work.  Alternately you can use the dist_ prefix[1], e.g.,

  dist_bzrmk_DATA = brz.mk

[1] 
https://www.gnu.org/software/automake/manual/automake.html#Fine_002dgrained-Distribution-Control

Cheers,
  Nick

Re: Automake Digest, Vol 175, Issue 3

[Nick Bowler]

On 2017-09-05, Kip Warner  wrote:
[...]
> Hey Thomas. Good question. It could well be that no hackery at all is
> required with this. Here is my Makefile.am:
>
> https://github.com/cartesiantheatre/narayan-designer/blob/master/Source/Makefile.am
>
> See parser_clobbered_source_full_paths as an example. This variant
> containing the full path is used in BUILT_SOURCES, nodist_..._SOURCES,
> CLEANFILES, and as a target.
>
> The parser_clobbered_source_files_only variant containing the file
> names only is used on line 150 as a workaround for where bisonc++(1)
> emits its files.
>
> If you can see a more elegant way of solving the same problem I'm
> trying to, I'm all ears.

If your only uses of the directoryless-filenames are in rules, then
just write the names including directories in the make variables,
then strip off the directory components inside the rule.  In rules
you can use the much more powerful shell constructs.

Example:

  % cat >Makefile <<'EOF'
  FOO = a b/c d/e/f

  my_rule:
for i in $(FOO); do \
  case $$i in */*) i=`expr "$$i" : '.*/\(.*\)'`; esac; \
  printf '%s\n' "$$i"; \
done
EOF
  % make my_rule
  a
  c
  f

If you assume a reasonably-POSIXish shell, you can use something like
$${i##*/} to strip directory parts instead (I think this form will fail
on at least Solaris /bin/sh).

Cheers,
  Nick

Re: Portable $addprefix

[Nick Bowler]

Hello,

On 8/24/17, Kip Warner  wrote:
> I'd like to transform the following variable in my Makefile.am from...
>
> files_only = a.foo b.foo c.foo d.foo ...
>
> Into...
>
> files_with_path = dir/a.foo dir/b.foo dir/c.foo dir/d.foo ...

I'm not aware of any truly portable way to do this directly in make.

But your example looks like a pretty static list (i.e., this list won't
be changed by the user after the package is generated), so the portable
way is to just generate both lists in advance, at the same time you run
automake (perhaps with a perl script that postprocesses Makefile.in).

If the list depends on configure results then another possibility is
to have configure generate both lists.

Finally, while not portable to all make implementations, expansions
like this:

  $(files_only:%:dir/%)

do work in multiple implementations other than GNU make.

Cheers,
  Nick

bug#27781: LIBOBJS should depend on LIBOBJDIR

[Nick Bowler]

On 8/23/17, Mathieu Lirzin  wrote:
> Michael Haubenwallner  writes:
>> Another thought about the final "$(LIBOBJS): .../.dirstamp" Makefile
>> line: If I remember correctly, there have been (non-GNU) make
>> implementations thatchoke on this rule when LIBOBJS is an empty
>> variable: Am I wrong here, or is GNU make required anyway these days?
>
> GNU make is not required for current Automake version.  I didn't find
> any reference of this issue with empty variable target in Autoconf
> "Portable Make programming" section [1] but if it is actually the case
> then it would be nice to find a solution for that.

POSIX requires that target rules specify at least one target.  At least
dmake will fail to parse the makefile if it contains an target rule with
no targets:

  % cat >Makefile <<'EOF'
FOO =
dummy: ;
$(FOO): ;
EOF
  % dmake
  dmake:  Makefile:  line 3:  Error: -- Missing targets or attributes in rule

If the target list is a macro which can reasonably be empty then one
solution is to add a bogus target name that won't be used anywhere
else, along the lines of:

  gobbledegook $(FOO): ;

Cheers,
  Nick

Re: Not installing to hard-coded locations vs polkit's fixed location

[Nick Bowler]

Hi,

On 2017-08-21, Mike Fleetwood  wrote:
> I'm working on adding installation of a polkit action file into
> GParted's build and install system, however the polkit daemon only
> recongises action files installed into the single location of
> /usr/share/polkit-1/action/.

There is a section about this issue in the Automake manual[1].

> Currently the Makefile.am contains this line:
> (larger fragment of the Makefile.am below)
> polkit_actiondir = $(datadir)/polkit-1/actions
[...]
> Are there any resolutions to this?
> I could:
> 1) Leave things as they are and document it as the builders
>responsibility, that when prefix defaults to /usr/local, or anything
>other than /usr, that the polkit action file will need manually
>installing into the correct location under a unique name so as not to
>overright any distro package provided copy.

It is pretty much fine as is.  If it matters, the installer can specify
polkit_actiondir when they install your package, for example:

  % make polkit_actiondir=/the/correct/location install

Just include a note about it in your README.

Things get a bit more complicated if you want the default install
location to be something probed at configure time.  This usually
involves some heuristics to get a reasonable user experience.

> 2) Set polkit_actiondir to /usr/share/polkit-1/action but that is
>against automake guidance and breaks 'make distcheck'.

This is generally a bad idea, as installing to hardcoded locations breaks
many things.  For example, it will prevent successful installation as an
unpriviliged user (unless the user knows to override polkit_actiondir).

> Are there any other solutions which are reasonable?

Another option (which might not be acceptable for you) is to do
nothing: i.e., don't install any files into the external location
by default.  The user can place files manually into the correct
locations as required.

[1] 
https://www.gnu.org/software/automake/manual/automake.html#Hard_002dCoded-Install-Paths

Cheers,
  Nick

Re: Integrating flexc++(1) and bisonc++(1) into Makefile.am

[Nick Bowler]

Hello,

On 7/12/17, Kip Warner  wrote:
> My challenge is replicating their functionality for flexc++(1) and
> bisonc++(1) in the absense of macros to make their usage easier in
> Automake
[...]
> In trying to integrate the two tools into my build environment, I've
> attempted the following in Makefile.am:
[...]
> BUILT_SOURCES = \
[...]
>   Source/ParserBase.h \
>   Source/Parser.h \
>   Source/Parser.ih \
>   Source/Parser.cpp
>
> myprogram_SOURCES = \
[...]
>   Source/Parser.cpp
[...]
> # Generate parser source from Backus-Naur grammar rules via bisonc++...
> Source/ParserBase.h:
> Source/Parser.h:
> Source/Parser.ih:
> Source/Parser.cpp: Source/Parser.ypp
>   $(BISONCPP) --target-directory=$(top_builddir)/Source $<
>
> FLEXCPP and BISONCPP are obtained via AC_PATH_PROG in configure.ac.
>
> This all works ok, but I suspect this is not an elegant solution and
> there are some very good suggestions from this mailing list.

There aren't really any "elegant" solutions.  Make handles this kind of
tool quite badly.  It is possible to get things to work but it is always
a tradeoff between flexibility of your build system and simplicity of
your rules.

If you are happy with this method then it is totally fine.  Do make
sure parallel builds work by testing them routinely (both clean and
incrementally) -- I think listing everything in BUILT_SOURCES like you
do probably "resolves" any parallelism problems here, (by reducing
opportunities for parallelism).

The Automake manual has section on writing portable make rules for tools
that produce multiple outputs[1], with a discussion of various approaches
and their limitations.  I generally prefer approaches using a dedicated
witness file.

Finally, consider whether you want to distribute the generated parser
sources.  That way your users don't need these tools installed just to
build your package.

[1] https://www.gnu.org/software/automake/manual/automake.html#Multiple-Outputs

Cheers,
  Nick