Re: 1.16.90 regression: configure now takes 7 seconds to start

[Zack Weinberg]

On Tue, Jun 11, 2024, at 7:00 PM, Karl Berry wrote:
> Maybe this is a silly question, but, is there a reason why this test
> needs to be performed in every single package that uses Automake?
>
> I was under the impression that the purpose of this test was
> merely to speed up running Automake's own test suite.
>
> There are other packages that have many tests that presumably also
> substantially benefit?

The timestamp resolution is actually used by core automake code, namely
the "ensuring generated files are newer than source" config.status step,
which is necessary to make tarball builds work reliably on systems that
don't have the `automake`, `autoconf`, `aclocal`, etc. tools already
installed.

I have an idea for how to streamline the whole mess, and will tinker
with it early next week.  Have to concentrate on the day job for the
remainder of this week.

zw

Re: Bug Resilience Program of German Sovereign Tech Fund

[Zack Weinberg]

On Thu, Jun 6, 2024, at 10:37 AM, Dan Kegel wrote:
> That's a really good idea.  Automake and Autotools in general underpin
> a fair amount of key open source software, but is taken for granted.
>
> Ideas for making the case for funding: identify...
> - how many commonly used Debian/Ubuntu/Alpine packages and/or GitHub
> projects rely on automake/Autotools
> - markers of risk (e.g. very out of date or buggy embedded Autotools files)
> - problems this has caused in the past
> - related government standards for software quality
> All of that might help make the case.

On the autoconf side of the fence, I do not have spare cycles to write a
proposal all by myself, but I would be interested in working with one or
more other people to write such a proposal, and in doing some of the
work.  In particular, I already have a mental list of things that
*ought* to be done in the near future for autoconf but I do not have the
capacity to do by myself on a volunteer basis.

I'm adding Sumana Harihareswara to the cc: list; she helped me secure the
funding for the release of autoconf 2.70 and might be interested in working
on this pitch as well.

zw

Re: GCC reporting piped input as a security feature

[Zack Weinberg]

On Tue, Apr 9, 2024, at 11:35 PM, Jacob Bachmeyer wrote:
> Jan Engelhardt wrote:
>> On Tuesday 2024-04-09 05:37, Jacob Bachmeyer wrote:
>>
 In principle it could be posible to output something different to
 describe this stramge situation explicitly.  For instance, output
 "via stdin" as a comment, or output `stdin/../filename' as the file
 name. (Programs that optimize the file name by deleting XXX/.../
 are likely not to check whether XXX is a real directory.)
...

How about `/dev/stdin/-` if no filename has been specified with #line or
whatever, and `/dev/stdin/[filename]` if one has, where [filename] is
the specified filename with all leading dots and slashes stripped,
falling back to `-` if empty? /dev/stdin can be relied on to either not
exist or not be a directory, so these shouldn't ever be openable.

zw

Re: GNU Coding Standards, automake, and the recent xz-utils backdoor

[Zack Weinberg]

On Mon, Apr 1, 2024, at 2:04 PM, Russ Allbery wrote:
> "Zack Weinberg"  writes:
>> It might indeed be worth thinking about ways to minimize the
>> difference between the tarball "make dist" produces and the tarball
>> "git archive" produces, starting from the same clean git checkout,
>> and also ways to identify and audit those differences.
>
> There is extensive ongoing discussion of this on debian-devel. There's
> no real consensus in that discussion, but I think one useful principle
> that's emerged that doesn't disrupt the world *too* much is that the
> release tarball should differ from the Git tag only in the form of
> added files. Any files that are present in both Git and in the release
> tarball should be byte-for-byte identical.

That dovetails nicely with something I was thinking about myself.
Obviously the result of "make dist" should be reproducible except for
signatures; to the extent it isn't already, those are bugs in automake.
But also, what if "make dist" produced *two* disjoint tarballs? One of
which is guaranteed to be byte-for-byte identical to an archive of the
VCS at the release tag (in some clearly documented fashion; AIUI, "git
archive" does *not* do what we want).  The other contains all the files
that "autoreconf -i" or "./bootstrap.sh" or whatever would create, but
nothing else.  Diffs could be provided for both tarballs, or only for
the VCS-archive tarball, whichever turns out to be more compact (I can
imagine the diff for the generated-files tarball turning out to be
comparable in size to the generated-files tarball itself).

This should make it much easier to find, and therefore audit, the pre-
generated files, and to validate that there's no overlap. It would add
an extra step for people who want to build from tarball, without having
to install autoconf (or whatever) first -- but an easier extra step
than, y'know, installing autoconf. :)  Conversely, people who want to
build from tarballs but *not* use the pre-generated configure, etc,
could now download the 'bare' tarball only.

("Couldn't those people just build from a git checkout?"  Not if they
don't have the tooling for it, not during early stages of a distribution
bootstrap, etc.  Also, the act of publishing a tarball that's a golden
copy of the VCS at the release tag is valuable for archival purposes.)

zw

Re: GNU Coding Standards, automake, and the recent xz-utils backdoor

[Zack Weinberg]

On Sun, Mar 31, 2024, at 3:17 AM, Jacob Bachmeyer wrote:
> Eric Gallager wrote:
>> Specifically, what caught my attention was how the release tarball
>> containing the backdoor didn't match the history of the project in its
>> git repository. That made me think about automake's `distcheck`
>> target, whose entire purpose is to make it easier to verify that a
>> distribution tarball can be rebuilt from itself and contains all the
>> things it ought to contain.
>
> The problem is that a release tarball is a freestanding object, with no 
> dependency on the repository from which it was produced.  In this case, 
> the attacker added a bogus "update" of build-to-host.m4 from gnulib to 
> the release tarball, but that file is not stored in the Git repository.  
> This would not have tripped "make distcheck" because the crocked tarball 
> can indeed be used to rebuild another crocked tarball.
>
> As Alexandre Oliva mentioned in his reply, there is not really any good 
> way to prevent this, since the attacker could also patch the generated 
> configure script more directly.

I have been thinking about this incident and this thread all weekend and
have seen a lot of people saying things like "this is more proof that tarballs
are a thing of the past and everyone should just build straight from git".
There are a bunch of reasons why one might disagree with this as a blanket
statement, but I do think there's a valid point here: the malicious xz
maintainer *might* have been caught earlier if they had committed the
build-to-host.m4 modification to xz's VCS.  (Or they might not have!
Witness the three (and counting) malicious patches that they barefacedly
submitted to *other* software and got accepted because the malice was
subtle enough to pass through code review.)

It might indeed be worth thinking about ways to minimize the difference
between the tarball "make dist" produces and the tarball "git archive"
produces, starting from the same clean git checkout, and also ways to
identify and audit those differences.

...
> Maybe the best revision to the GNU Coding Standards would be that 
> releases should, if at all possible, contain only text?  Any binary 
> files needed for testing can be generated during "make check" if 
> necessary

I don't think this is a good idea.  It's only a speed bump for someone
trying to smuggle malicious data into a package (think "base64 -d") and
it makes life substantially harder for honest authors of programs that
work with binary data, and authors of material whose "source code"
(as GPLv3 uses that term) *is* binary data.  Consider pngsuite, for
instance (http://www.schaik.com/pngsuite/) -- it would be a *ton* of
work to convert each of these test PNG files into GNU Poke scripts,
and probably the result would be *less* ergonomic for purposes of
improving the test suite.

I would like to suggest that a more useful policy would be "files
written to $prefix by 'make install' should not have any data
dependency on files labeled as part of the package's testsuite".
That doesn't constrain honest authors and it seems within the
scope of what the reproducible builds people could test for.
(Build the package, install to nonce prefix 1, unpack the tarball
again, delete the test suite, build again, install to prefix 2, compare.)
Of course a sufficiently determined malicious coder could detect
the reproducible-build test environment, but unlike "no binary data"
this is a substantial difficulty increment.

zw

bug#68808: subsecond mtime discovery code insufficient

[Zack Weinberg]

On Tue, Jan 30, 2024, at 6:32 PM, Karl Berry wrote:
> test "$[*]" != "X conftest.ts1 conftest.ts2" ||
> ..
> test "$[]2" = conftest.ts3 &&
> ..
> etc.
>
> I've never seen anything like that before. 

Oh! Those square brackets are M4 quotes to prevent M4 from expanding $* and $2 
itself.

zw

bug#68808: subsecond mtime discovery code insufficient

[Zack Weinberg]

On Tue, Jan 30, 2024, at 7:46 AM, Erik A Johnson wrote:
> I found the problem!  The default GNU make on macOS (even Sonoma) is
> 3.81 from 2006!  The configure script for GNU make versions <4.0 did
> not have the right test for subsecond resolution in the macOS stat()
> function, so 3.81 was built with resolution only to the second, not
> subsecond, thus needing at least one second sleep.

Okay, so that means the filesystem timestamp resolution test needs to
probe the behavior of `make` as well as all the other things it tests. I
will work on a patch for that as soon as I have a usable workstation
again, which will probably not be until next week at the earliest; if
anyone wants to beat me to it, feel free.

> (Why, then, does Apple continue to include 3.81 in the software 18
> years later? Beyond me.)

3.81 was the last version distributed under GPLv2; Apple won't touch
anything that's distributed under GPLv3.  Their copy of GNU M4 is also
vintage 2006, for the same reason.  I don't claim to understand why
their legal department thinks GPLv2 is fine but v3 has to be avoided,
but (like many other proprietary Unix vendors) they probably keep
shipping the old GNU tools instead of, like, switching to current BSD
tools, for the sake of compatibility with old scripts.

zw

bug#68808: subsecond mtime discovery code insufficient

[Zack Weinberg]

On Mon, Jan 29, 2024, at 6:00 PM, Karl Berry wrote:
> But, looking at that code, I belatedly see that shell arrays are being
> used. At least I think so.

It is absolutely not *supposed* to be using shell arrays. If it is, that was a 
mistake on my part.

zw

Unavailable due to hardware problems

[Zack Weinberg]

In case anyone is waiting for me to respond to questions or write a patch or 
anything, please be advised that my work computer has crashed hard and until 
replacement components arrive (early next week) I can't do much of anything.

[bug#68416] [PATCH] python: add 3.20 - 3.16 to the version search list

[Zack Weinberg]

On Mon, Jan 15, 2024, at 7:02 PM, Jacob Bachmeyer wrote:
> Frederic Berat wrote:
>> pythons="python python2 python3"
>> for i in {20..0};do pythons="$pythons python3.$i";done
>> for i in {7..0};do pythons="$pythons python2.$i";done
>>
>> m4_define_default([_AM_PYTHON_INTERPRETER_LIST], [$pythons])
>
> The {20..0} is a syntax unfamiliar to me, yet even Bash 3 has it...
> interesting.

Just responding to this specific thing: the POSIX shell spec doesn't
include any form of brace expansion.  ksh93 (-2008) does recognize
both {a,b,c} and {1..10}, but NetBSD /bin/sh and /bin/ksh don't.
The `seq` utility is not portable either.

zw

bug#68274: automake 1.16j nonnumerical version confuses scripts

[Zack Weinberg]

On Sun, Jan 14, 2024, at 1:49 AM, Mike Frysinger wrote:
> On 13 Jan 2024 15:58, Karl Berry wrote:
>> Another alternative: when this came up 30-odd years ago, rms changed the
>> GNU maintainers doc to suggest x.y.90, .91, etc. for pretests. Doing
>> that would at least have the benefit of following a recommendation, and
>> as a side effect, would also fix jami's assumption (poor practice though
>> it is, IMHO).
>> https://gnu.org/prep/maintain/html_node/Test-Releases.html#Test-Releases
>> 
>> Doing an ls -R on alpha (fp:/srv/data/ftp-mirror/alpha/gnu), it seems
>> (rough guess with some grep counting) the .90 convention is by far the
>> most common approach (a couple thousand), followed by the suffix letter
>> a la automake (~750 releases), followed by -rc (~360). -hexid and -date
>> are both trailing the field. Other random conventions also present.
>> 
>> It all feels like bikeshedding to me, so my inclination is to do
>> nothing.  If we do change, I think we should use .90.  --best, karl.
>
> using .90 is certainly better than single-letters.  if you're fine with
> it, then let's switch.

For what it's worth, I had planned to switch Autoconf, starting with the
next release, to use *some* version numbering scheme for beta releases
that sorts correctly according to things like strverscmp() and
dpkg --compare-versions.  The "append a letter to the version number
intended for the final release" convention makes these algorithms sort
the betas *after* the release, which is backwards.

My plan *was* to append letters to the version number for the *previous*
release, with a gap (e.g. 2.72{j,k,...} would be prereleases for 2.73),
which I think is what Automake is doing now) but I like .9x version numbers
better because it's more common (as you observed) and therefore more likely
to be understood at sight.  I'd actually forgotten that .9x versions were
an official GNU recommendation.

zw

[bug#68325] sanity.m4: a millisecond is too fast?

[Zack Weinberg]

On Mon, Jan 8, 2024, at 7:44 AM, Karl Berry wrote:
> The plethora of bug reports from the last pretest make me believe that
> many systems simply cannot (currently) reliably handle a millisecond
> difference in mtime.
...
> My current idea is to only try for a hundredth of a second.
...
> The whole idea of the am test command $sleep seems like a
> bogus workaround to me. On the other hand, failing a true fix, I think
> it needs to be added in many more places, with the advent of subsecond
> mtimes. Debugging all this is nowhere on my list :(.
>
> Anyway, if anyone has any ideas about how to better proceed, please
> speak up.

Nick Bowler had a promising suggestion, recorded in 
https://savannah.gnu.org/support/?110989 , but I am still traveling and won't 
be able to work on it until next week. In the meantime, your patch should at 
least give us some data on whether the problem really is timestamps.

zw

[bug#67841] [PATCH] Clarify error messages for misuse of m4_warn and --help for -W.

[Zack Weinberg]

On Sat, Dec 23, 2023, at 1:54 PM, 
> Without Jacob's suggested use of encode(), I see (Jacob, thanks for the
> explanations), but Zack, I'm not sure you intended this bug to track
> that. Feel free to reopen if so.

I decided I didn't want to write that code and verify it in a hurry. I'll come 
back to it next year. This bug was just to make sure automake and autoconf were 
back in sync on all the shared files.

[bug#67971] [PATCH] Sync shared files from Autoconf

[Zack Weinberg]

lib/Automake/ChannelDefs.pm and lib/Automake/Channels.pm were updated
in Autoconf to address .
Bring over the changes.

It *should* be impossible to reach report_bad_channel from code in Automake.

* lib/Automake/Channels.pm (msg): If the channel argument is invalid,
  don’t crash; report the mistake and use the ‘syntax’ channel.
  (report_bad_channel): New function for reporting invalid channels.

* lib/Automake/ChannelDefs.pm (usage): Clarify that the list of
  warning categories is exhaustive, and that “all”, “none”,
  “no-CATEGORY”, and “error” are not warning categories.
---
 lib/Automake/ChannelDefs.pm | 10 
 lib/Automake/Channels.pm| 47 -
 2 files changed, 52 insertions(+), 5 deletions(-)

diff --git a/lib/Automake/ChannelDefs.pm b/lib/Automake/ChannelDefs.pm
index 1c436645e..8b334ee93 100644
--- a/lib/Automake/ChannelDefs.pm
+++ b/lib/Automake/ChannelDefs.pm
@@ -197,7 +197,7 @@ Return the warning category descriptions.
 
 sub usage ()
 {
-  return "Warning categories include:
+  return "Warning categories are:
   cross  cross compilation issues
   gnuGNU coding standards (default in gnu and gnits modes)
   obsolete   obsolete features or constructions (default)
@@ -207,10 +207,12 @@ sub usage ()
   extra-portability  extra portability issues related to obscure tools
   syntax dubious syntactic constructs (default)
   unsupportedunsupported or incomplete features (default)
-  allall the warnings
-  no-CATEGORYturn off warnings in CATEGORY
+
+-W also understands:
+  allturn on all the warnings
   none   turn off all the warnings
-  error  treat warnings as errors";
+  no-CATEGORYturn off warnings in CATEGORY
+  error  treat all enabled warnings as errors";
 }
 
 =item C
diff --git a/lib/Automake/Channels.pm b/lib/Automake/Channels.pm
index b4563d36e..84e93d106 100644
--- a/lib/Automake/Channels.pm
+++ b/lib/Automake/Channels.pm
@@ -628,7 +628,13 @@ sub msg ($$;$%)
   $location = '';
 }
 
-  confess "unknown channel $channel" unless exists $channels{$channel};
+  if (!exists $channels{$channel})
+{
+  # This can happen as a result of e.g. m4_warn([nonsense], [message])
+  # so it should not crash.
+  report_bad_channel($channel, $location);
+  $channel = 'syntax';
+}
 
   my %opts = %{$channels{$channel}};
   _merge_options (%opts, %options);
@@ -662,6 +668,45 @@ sub msg ($$;$%)
 }
 }
 
+sub report_bad_channel ($$)
+{
+  my ($channel, $location) = @_;
+  my $message;
+  my $report_as = 'error';
+
+  # quotemeta is both too aggressive (e.g. it escapes '-') and
+  # too generous (it turns control characters into \ + themselves,
+  # not into symbolic escapes).
+  my $q_channel = $channel;
+  $q_channel =~ s/(?=[\"\$\'\@\`\\])/\\/g;
+  $q_channel =~ s/([^\x20-\x7e])/sprintf('\\x%02X', ord $1)/eg;
+  $q_channel = '"' . $q_channel . '"';
+
+  if ($channel eq '' || $channel eq 'all')
+{
+  # Prior to version 2.70, the Autoconf manual said it was valid to use
+  # "all" and the empty string as the category argument to m4_warn, so
+  # don't treat those cases as errors.
+  $report_as = 'obsolete';
+  $message = "use of $q_channel as a diagnostic category is obsolete\n";
+  $message .= "(see automake --help for a list of valid categories)";
+}
+  elsif ($channel eq 'none'
+ || ($channel =~ /^no-/ && exists $channels{substr($channel, 3)}))
+{
+  # Also recognize "none" and "no-[category]", as someone might have
+  # thought anything acceptable to -W is also acceptable to m4_warn.
+  # Note: m4_warn([error], [...]) does actually issue an error.
+  $message = "-W accepts $q_channel, but it is not a diagnostic category";
+}
+  else
+{
+  $message = "unknown diagnostic category " . $q_channel;
+}
+
+  msg $report_as, $location, $message;
+}
+
 
 =item C
 
-- 
2.41.0

What does (did?) it mean when aclocal exits with code 63?

[Zack Weinberg]

One of Autoconf's regression tests includes:

# If Autoconf is too old, or the user has turned caching off, skip:
AT_CHECK([aclocal || { ret=$?; test $ret -eq 63 && ret=77; exit $ret; }],
 [], [], [ignore])

The effect is to skip the rest of the test if aclocal's exit status is 63.
This code was written in 2006, I can't find any documentation on what
it means for aclocal to exit with code 63, and I also can't find any
code in the current automake git repo that looks like it would make
aclocal exit with code 63.  (*automake* can exit with code 63 under
some circumstances, but it really looks like aclocal never will.)

Does anyone remember what situation this might have been trying to detect?

Thanks,
zw

[bug#67841] [PATCH v2, committed] Clarify error messages for misuse of m4_warn and --help for -W.

[Zack Weinberg]

m4_warn([category], [message]) passes its arguments directly to
Autom4te::Channels::msg.  If the category argument is not a recognized
“channel”, that function will crash and emit a *Perl* stack trace,
which makes it look like there’s something wrong with autoconf or
autom4te, rather than something wrong with the script.

Making matters worse, in autoconf 2.69, the manual said you could
use “all” and the empty string as the first argument to m4_warn.
As far as I can tell, neither of those was ever a valid message
channel, and the manual was corrected in 2.70, but we still got
a bug report from someone who tried it.

This patch makes us issue a nice helpful user error, instead of a
confusing internal error, when Autom4te::Channels::msg is called with
a bogus channel argument.  If the bogus channel is “all” or the empty
string, that error is demoted to a -Wobsolete warning.  If it is one
of the other special tokens recognized by -W, we customize the error
message, since someone might’ve thought that “none” being acceptable
to -W meant it was also acceptable to m4_warn.  The --help output for
autoconf, autoheader, autom4te, and autoreconf is also adjusted to
clarify that not all of the tokens recognized by -W count as
warning categories.

(Oddly enough, the lack of filtration at the autom4te level means
m4_warn([error], […]) actually does issue an error.  There’s no other
way to get exactly that effect — m4_errprintn(…)  and m4_fatal(…)
both do something a little different — so I I propose to leave that
alone for now and promote it to a proper, documented feature, probably
spelled m4_error(…), post-2.72.)

Channels.pm and ChannelDefs.pm are shared with Automake.  I believe
there is nothing you can write in a Makefile.am that will cause
Automake::Channels::msg to be called with an arbitrary user-supplied
channel argument, so these changes should have no visible effect on
that side of the fence.

Addresses .

 * lib/Autom4te/Channels.pm (msg): If the channel argument is invalid,
   don’t crash; report the mistake and use the ‘syntax’ channel.
   (report_bad_channel): New function for reporting invalid channels.

 * lib/Autom4te/ChannelDefs.pm (usage): Clarify that the list of
   warning categories is exhaustive, and that “all”, “none”,
   “no-CATEGORY”, and “error” are not warning categories.

 * bin/autoconf.in, bin/autoheader.in, bin/autom4te.in,
   bin/autoreconf.in: Tweak --help text.

 * tests/m4sugar.at (m4@@_warn (bad categories)): New test.
---
 bin/autoconf.in |  1 +
 bin/autoheader.in   |  1 +
 bin/autom4te.in |  1 +
 bin/autoreconf.in   | 10 +++---
 lib/Autom4te/ChannelDefs.pm | 10 +++---
 lib/Autom4te/Channels.pm| 47 ++-
 tests/m4sugar.at| 63 +
 7 files changed, 122 insertions(+), 11 deletions(-)

diff --git a/bin/autoconf.in b/bin/autoconf.in
index 060a9a04..5f4a2e5c 100644
--- a/bin/autoconf.in
+++ b/bin/autoconf.in
@@ -64,6 +64,7 @@ Operation modes:
   -f, --force   consider all files obsolete
   -o, --output=FILE save output in FILE (stdout is the default)
   -W, --warnings=CATEGORY   report the warnings falling in CATEGORY
+(comma-separated list accepted)
 
 " . Autom4te::ChannelDefs::usage . "
 
diff --git a/bin/autoheader.in b/bin/autoheader.in
index 4afa5f13..f5af0df2 100644
--- a/bin/autoheader.in
+++ b/bin/autoheader.in
@@ -74,6 +74,7 @@ or else 'configure.in'.
   -d, --debug  don\'t remove temporary files
   -f, --force  consider all files obsolete
   -W, --warnings=CATEGORY  report the warnings falling in CATEGORY
+   (comma-separated list accepted)
 
 " . Autom4te::ChannelDefs::usage . "
 
diff --git a/bin/autom4te.in b/bin/autom4te.in
index 8957a6c9..2e4e7d52 100644
--- a/bin/autom4te.in
+++ b/bin/autom4te.in
@@ -160,6 +160,7 @@ Operation modes:
   -o, --output=FILEsave output in FILE (defaults to '-', stdout)
   -f, --force  don't rely on cached values
   -W, --warnings=CATEGORY  report the warnings falling in CATEGORY
+   (comma-separated list accepted)
   -l, --language=LANG  specify the set of M4 macros to use
   -C, --cache=DIRECTORYpreserve results for future runs in DIRECTORY
   --no-cache   disable the cache
diff --git a/bin/autoreconf.in b/bin/autoreconf.in
index c6077efe..285d0f49 100644
--- a/bin/autoreconf.in
+++ b/bin/autoreconf.in
@@ -84,20 +84,18 @@ Operation modes:
   --no-recursive   don't rebuild sub-packages
   -s, --symlinkwith -i, install symbolic links instead of copies
   -m, --make   when applicable, re-run ./configure && make
-  -W, --warnings=CATEGORY  report the warnings falling in CATEGORY [syntax]
+  -W, --warnings=CATEGORY  report the warnings falling in CATEGORY
+   (comma-separated list 

[bug#67841] [PATCH] Clarify error messages for misuse of m4_warn and --help for -W.

[Zack Weinberg]

On Fri, Dec 15, 2023, at 6:52 PM, Karl Berry wrote:
> Hi Zack,
>
> Since this touches code shared between Autoconf and Automake, I'm not
> checking it in yet and I'm requesting comments from both sides of the
> fence.
>
> Well, it seems good to me in principle, FWIW. I don't think this
> directly affects automake?

I doubt it, unless there's something you can put in a Makefile.am that will
cause Automake::Channels::msg to be called with a user-controlled $channel
argument.  (On the autoconf side of the fence, m4_warn([channel], [message])
does exactly that, hence the patch.)

zw

[bug#67841] [PATCH] Clarify error messages for misuse of m4_warn and --help for -W.

[Zack Weinberg]

On Fri, Dec 15, 2023, at 7:08 PM, Jacob Bachmeyer wrote:
> Zack Weinberg wrote:
>> [...]
>> Also, there’s a perl 2.14ism in one place (s///a) which I need
>> to figure out how to make 2.6-compatible before it can land.
...
>> +  $q_channel =~ s/([^\x20-\x7e])/"\\x".sprintf("%02x", ord($1))/aeg;
...
> If I am reading perlre correctly, you should be able to simply drop the 
> /a modifier because it has no effect on the pattern you have written, 
> since you are using an explicit character class and are *not* using the 
> /i modifier.

Thanks, you've made me realize that /a wasn't even what I wanted in the
first place.  What I thought /a would do is force s/// to act byte by
byte -- or, in the terms of perlunitut, force the target string to be
treated as a binary string.  That might be clearer with a concrete example:

$ perl -e '$_ = "\xE2\x88\x85"; s/([^\x20-\x7e])/sprintf("\\x%02x", 
ord($1))/eg; print "$_\n";'
\xe2\x88\x85
$ perl -e '$_ = "\N{EMPTY SET}"; s/([^\x20-\x7e])/sprintf("\\x%02x", 
ord($1))/eg; print "$_\n";'
\x2205

What change do I need to make to the second one-liner to make it also
print \xe2\x88\x85?  How do I express that in a way that is backward
compatible all the way to 5.6.0?  And finally, how do I ensure that
there is absolutely nothing I can put in the initial assignment to
$_ that will cause the rest of the one-liner to crash?  For example
over in the Python universe it's very easy to get Unicode conversion
to crash:

$ python3 -c 'print("\uDC00".encode("utf-8"))'
Traceback (most recent call last):
  File "", line 1, in 
UnicodeEncodeError: 'utf-8' codec can't encode character '\udc00' in position 
0: surrogates not allowed

Given that having non-ASCII here in the first place is pretty unlikely,
I am going to go ahead and land the patch with your suggested changes,
but I'd still appreciate any further advice you have.

zw

[bug#67841] [PATCH] Clarify error messages for misuse of m4_warn and --help for -W.

[Zack Weinberg]

In autoconf 2.69, the manual said that you could use “all” and the
empty string as the first argument to m4_warn.  As far as I can tell
this was never actually true, and the manual was corrected in 2.70,
but we still got a bug report from someone who tried it and got a
confusing internal error from the guts of autom4te.

This patch makes us issue a nice helpful user error, instead of a
confusing internal error, when Autom4te::Channels::msg is called with
a bogus channel argument.  If the bogus channel is “all” or the empty
string, that error is demoted to a -Wobsolete warning.  If it is one
of the other special tokens recognized by -W, we customize the error
message, since someone might’ve thought that “none” being acceptable
to -W meant it was also acceptable to m4_warn.  The --help output for
autoconf, autoheader, autom4te, and autoreconf is also adjusted to
clarify that not all of the tokens recognized by -W count as
warning categories.

(Oddly enough, m4_warn([error], […]) actually issues an error, because
the argument to m4_warn is passed straight through as the channel
argument to Autom4te::Channels::msg.  Since neither m4_errprintn(…)
nor m4_fatal(…)  does quite the same thing as m4_warn([error], […]),
I propose to leave that alone for now and promote it to a proper,
documented feature (probably spelled m4_error(…)) post-2.72.)

Since this touches code shared between Autoconf and Automake, I’m not
checking it in yet and I’m requesting comments from both sides of the
fence.  Also, there’s a perl 2.14ism in one place (s///a) which I need
to figure out how to make 2.6-compatible before it can land.

Addresses .

 * lib/Autom4te/Channels.pm (msg): If the channel argument is invalid,
   don’t crash; report the mistake and use the ‘syntax’ channel.
   (report_bad_channel): New function for reporting invalid channels.

 * lib/Autom4te/ChannelDefs.pm (usage): Clarify that the list of
   warning categories is exhaustive, and that “all”, “none”,
   “no-CATEGORY”, and “error” are not warning categories.

 * bin/autoconf.in, bin/autoheader.in, bin/autom4te.in,
   bin/autoreconf.in: Tweak --help text.

 * tests/m4sugar.at (m4@@_warn (bad categories)): New test.
---
 bin/autoconf.in |  1 +
 bin/autoheader.in   |  1 +
 bin/autom4te.in |  1 +
 bin/autoreconf.in   | 10 +++---
 lib/Autom4te/ChannelDefs.pm | 10 +++---
 lib/Autom4te/Channels.pm| 48 +++-
 tests/m4sugar.at| 63 +
 7 files changed, 123 insertions(+), 11 deletions(-)

diff --git a/bin/autoconf.in b/bin/autoconf.in
index 060a9a04..5f4a2e5c 100644
--- a/bin/autoconf.in
+++ b/bin/autoconf.in
@@ -64,6 +64,7 @@ Operation modes:
   -f, --force   consider all files obsolete
   -o, --output=FILE save output in FILE (stdout is the default)
   -W, --warnings=CATEGORY   report the warnings falling in CATEGORY
+(comma-separated list accepted)
 
 " . Autom4te::ChannelDefs::usage . "
 
diff --git a/bin/autoheader.in b/bin/autoheader.in
index 4afa5f13..f5af0df2 100644
--- a/bin/autoheader.in
+++ b/bin/autoheader.in
@@ -74,6 +74,7 @@ or else 'configure.in'.
   -d, --debug  don\'t remove temporary files
   -f, --force  consider all files obsolete
   -W, --warnings=CATEGORY  report the warnings falling in CATEGORY
+   (comma-separated list accepted)
 
 " . Autom4te::ChannelDefs::usage . "
 
diff --git a/bin/autom4te.in b/bin/autom4te.in
index 8957a6c9..2e4e7d52 100644
--- a/bin/autom4te.in
+++ b/bin/autom4te.in
@@ -160,6 +160,7 @@ Operation modes:
   -o, --output=FILEsave output in FILE (defaults to '-', stdout)
   -f, --force  don't rely on cached values
   -W, --warnings=CATEGORY  report the warnings falling in CATEGORY
+   (comma-separated list accepted)
   -l, --language=LANG  specify the set of M4 macros to use
   -C, --cache=DIRECTORYpreserve results for future runs in DIRECTORY
   --no-cache   disable the cache
diff --git a/bin/autoreconf.in b/bin/autoreconf.in
index c6077efe..285d0f49 100644
--- a/bin/autoreconf.in
+++ b/bin/autoreconf.in
@@ -84,20 +84,18 @@ Operation modes:
   --no-recursive   don't rebuild sub-packages
   -s, --symlinkwith -i, install symbolic links instead of copies
   -m, --make   when applicable, re-run ./configure && make
-  -W, --warnings=CATEGORY  report the warnings falling in CATEGORY [syntax]
+  -W, --warnings=CATEGORY  report the warnings falling in CATEGORY
+   (comma-separated list accepted)
 
 " . Autom4te::ChannelDefs::usage . "
 
-The environment variable 'WARNINGS' is honored.  Some subtools might
-support other warning types, using 'all' is encouraged.
-
 Library directories:
   -B, --prepend-include=DIR  prepend directory DIR to search path
   -I, --include=DIR   

[bug#67670] [PATCH] Revise tests for file mtime resolution again.

[Zack Weinberg]

In order for the Automake testsuite to be able to use sub-second delays to
control whether certain files are considered newer than others, five(!)
separate pieces of software all need to cooperate: automake itself,
autoconf’s internal “autom4te” utility, the Perl interpreter and its
libraries, the sleep(1) shell utility, and finally the filesystem hosting
the build directory.  The existing tests for this are a combination of
inadequate and incorrect.  This patch, in conjunction with a patch just
committed to Autoconf trunk

https://git.savannah.gnu.org/cgit/autoconf.git/commit/?id=39d96e6fff7ceae63b823872602caf4d255a38c8

should make everything much more robust, as follows:

- _AM_FILESYSTEM_TIMESTAMP_RESOLUTION is completely rewritten.
  It no longer looks for autom4te at all, because this macro is invoked
  unconditionally from AM_INIT, so *every* project that uses Automake
  would get this test that’s only relevant to Automake’s own testsuite.
  Also, it tries sleeping for as little as one millisecond (smaller delays
  consistently get rounded up to 1ms on my computer and I expect that’s
  universal), it should accurately detect FAT’s two-second resolution now,
  and it should not be tripped up anymore by running at precisely the
  moment that will make a 0.1s sleep cross a 1s boundary (this may sound
  unlikely but it used to cause a couple of test failures *every time* I
  ran the automake testsuite on a network filesystem that only supported
  1s resolution).

- In support of the above, the test for working ls -t moved from
   AM_SANITY_CHECK to _AM_FILESYSTEM_TIMESTAMP_RESOLUTION.  This allowed
   me to simplify the test for $srcdir/configure being older than a
   freshly created file.

- If automake is capable of reading high-resolution file modification
  timestamps from the operating system, it prints “Features: subsecond-mtime”
  as the second line of --version output.  (We can’t just assume this
  works for sufficiently new automake, because it depends on whether the
  Perl interpreter provides this capability, and that’s not a simple
  question of which version of Perl you have either.)

- The Autoconf patch mentioned above adds the same annotation to the
  output of autom4te --version.

- Finally, t/ax/test-defs.in looks for the “Features: subsecond-mtime”
  string from both automake and autom4te and resets the sleep time to
  one second if it’s not there.  There might be a better place to put this,
  somewhere it’ll execute every time the *overall testsuite* is invoked
  rather than once for each test, but I couldn’t find one.

Tested on x86-64-linux with development automake and development autoconf.

Previous discussion:

- https://lists.gnu.org/archive/html/automake/2023-03/msg0.html
- https://lists.gnu.org/archive/html/automake/2023-04/msg2.html
- https://lists.gnu.org/archive/html/automake/2023-12/msg5.html
- https://debbugs.gnu.org/cgi/bugreport.cgi?bug=64756

ChangeLog:

* m4/sanity.m4 (_AM_FILESYSTEM_TIMESTAMP_RESOLUTION): Rewrite for greater
  reliability.  Don’t probe autom4te at all here.  Check for working
  “ls -t” here.
  (AM_SANITY_CHECK): Do not cache the result.  Do not check for working
  “ls -t” check here.  Disentangle control flow in the loop probing
  the relative ages of build and source directory.

* lib/Automake/FileUtils.pm: Sync from autoconf.
* bin/automake.in (version): Include “Features: subsecond-mtime” in the
  output if $Automake::FileUtils::subsecond_mtime is true.
* configure.ac: Rename the substitution variable MODIFICATION_DELAY to
  MTIME_RESOLUTION.

* t/ax/test-defs.in: Require both $AUTOMAKE and $AUTOM4TE to report
  support for high-resolution timestamps before setting $sleep to
  delay for less than one second.
---
 bin/automake.in   |  17 ++--
 configure.ac  |   9 +-
 lib/Automake/FileUtils.pm |  71 +++-
 m4/sanity.m4  | 171 --
 t/ax/test-defs.in |  26 --
 5 files changed, 180 insertions(+), 114 deletions(-)

diff --git a/bin/automake.in b/bin/automake.in
index c0ffc1986..820d0d541 100644
--- a/bin/automake.in
+++ b/bin/automake.in
@@ -8271,16 +8271,19 @@ General help using GNU software: 
.
 
 sub version ()
 {
-  print 
+  print "automake (GNU $PACKAGE) $VERSION\n";
+  print "Features: subsecond-mtime\n"
+  if $Automake::FileUtils::subsecond_mtime;
+  print "\nCopyright (C) $RELEASE_YEAR Free Software Foundation, Inc.";
+  print '
+License GPLv2+: GNU GPL version 2 or later
+  
 This is free software: you are free to change and redistribute it.
 There is NO WARRANTY, to the extent permitted by law.
 
-Written by Tom Tromey 
-   and Alexandre Duret-Lutz .
-EOF
+Written by Tom Tromey 
+   and Alexandre Duret-Lutz .
+';
   # --version always returns 0 per GNU standards.
   exit 0;
 }
diff --git a/configure.ac b/configure.ac
index 

Re: [RFC PATCH]: autom4te: report subsecond timestamp support in --version

[Zack Weinberg]

On Tue, Dec 5, 2023, at 11:30 PM, Jacob Bachmeyer wrote:
> Zack Weinberg wrote:
>> $ autom4te --version
>> autom4te (GNU Autoconf) 2.71
>> Features: subsecond-timestamps
>>
>> Copyright (C) 2021 Free Software Foundation, Inc.
>> License GPLv3+/Autoconf: GNU GPL version 3 or later
>> <https://gnu.org/licenses/gpl.html>, 
>> <https://gnu.org/licenses/exceptions.html>
>> This is free software: you are free to change and redistribute it.
>> There is NO WARRANTY, to the extent permitted by law.
>>
>> Written by Akim Demaille.
>>
>> This preserves the effectiveness of sed '/Copyright/,$d' and also
>> leaves room for future additions to the "Features:" line.
>
> That looks like a good idea to me [...]

Karl also approved, so I have gone ahead and implemented this and
pushed it to Autoconf.  In the process I discovered that *both*
automake and autom4te need to advertise the capability for subsecond
timestamps; expect a patch for Automake shortly.

zw

[PATCH v2, committed] autom4te: report subsecond timestamp support in --version

[Zack Weinberg]

The Automake test suite wants this in order to know if it’s safe to
reduce the length of various delays for the purpose of ensuring files
in autom4te.cache are newer than the corresponding source files.  We
can also take advantage of this to speed up a couple of tests in our
own testsuite.

* lib/Autom4te/FileUtils.pm: Provide (but do not export) a flag
  $subsecond_mtime, indicating whether the ‘mtime’ function reports
  modification time with precision greater than one second.
  Reorganize commentary and import logic for clarity.  Add
  configuration for emacs’ perl-mode to the bottom of the file.
* bin/autom4te.in ($version): If $Autom4te::FileUtils::subsecond_mtime
  is true, print “Features: subsecond-mtime” as the second line
  of --version output.

* lib/autotest/general.m4: Move definitions of AS_MESSAGE_LOG_FD,
  AT_JOB_FIFO_IN_FD, and AT_JOB_FIFO_OUT_FD to top level and change
  the latter two to be defined using _AT_DEFINE_INIT.  This enables
  use of AS_MESSAGE_LOG_FD in AT_TESTS_PREPARE code.

* tests/local.at (AT_MTIME_DELAY): New utility that delays a test for
  an appropriate time to ensure all files created before its use are
  considered older than all files created afterward.  The delay will
  be as short as possible.
  (AT_TESTS_PREPARE): Calculate and log the delay used by AT_MTIME_DELAY.
  Only use a delay shorter than one second if the build filesystem,
  our autom4te, and the automake on the PATH all support this.
  (AT_CMP): En passant removal of unnecessary blank lines.

* tests/tools.at: Use AT_MTIME_DELAY, instead of sleeping for a
  hardcoded interval, everywhere the delay was to control file
  timestamps.
---
 bin/autom4te.in   |  6 ++-
 lib/Autom4te/FileUtils.pm | 71 +++-
 lib/autotest/general.m4   | 20 ++---
 tests/local.at| 86 +--
 tests/tools.at| 26 +---
 5 files changed, 164 insertions(+), 45 deletions(-)

diff --git a/bin/autom4te.in b/bin/autom4te.in
index 38a61ac9..8957a6c9 100644
--- a/bin/autom4te.in
+++ b/bin/autom4te.in
@@ -207,8 +207,10 @@ General help using GNU software: 
.
 
 # $VERSION
 # 
-$version = "autom4te (@PACKAGE_NAME@) @VERSION@
-Copyright (C) @RELEASE_YEAR@ Free Software Foundation, Inc.
+$version = "autom4te (@PACKAGE_NAME@) @VERSION@\n"
+  . ($Autom4te::FileUtils::subsecond_mtime
+ ? "Features: subsecond-mtime\n" : "")
+  . "\nCopyright (C) @RELEASE_YEAR@ Free Software Foundation, Inc.
 License GPLv3+/Autoconf: GNU GPL version 3 or later
 , 
 This is free software: you are free to change and redistribute it.
diff --git a/lib/Autom4te/FileUtils.pm b/lib/Autom4te/FileUtils.pm
index c1e8e8c3..06f87c31 100644
--- a/lib/Autom4te/FileUtils.pm
+++ b/lib/Autom4te/FileUtils.pm
@@ -38,24 +38,45 @@ use 5.006;
 use strict;
 use warnings FATAL => 'all';
 
-use Exporter;
+BEGIN
+{
+  require Exporter;
+  our @ISA = qw (Exporter);
+  our @EXPORT = qw (
+   _file 
+   _file
+_hint 
+   _has_case_matching_file _dir_cache
+   _dir_cache_file);
+}
+
+# Use sub-second resolution file timestamps if available, carry on
+# with one-second resolution timestamps if Time::HiRes is not available.
+#
+# Unfortunately, even if Time::HiRes is available, we don't get
+# timestamps to the full precision recorded by the operating system,
+# because Time::HiRes converts timestamps to floating-point, and the
+# rounding error is hundreds of nanoseconds for circa-2023 timestamps
+# in IEEE double precision.  But this is the best we can do without
+# dropping down to C.
+#
+# $subsecond_mtime is not exported, but is intended for external
+# consumption, as $Autom4te::FileUtils::subsecond_mtime.
+BEGIN
+{
+  our $subsecond_mtime = 0;
+  eval
+{
+  require Time::HiRes;
+  import Time::HiRes qw(stat);
+  $subsecond_mtime = 1;
+}
+}
+
 use IO::File;
-
-# use sub-second resolution timestamps if available,
-# carry on with one-second resolution timestamps if that is all we have
-BEGIN { eval { require Time::HiRes; import Time::HiRes qw(stat) } }
-
 use Autom4te::Channels;
 use Autom4te::ChannelDefs;
 
-our @ISA = qw (Exporter);
-our @EXPORT = qw (
- _file 
- _file
-  _hint 
- _has_case_matching_file _dir_cache
- _dir_cache_file);
-
 =over 4
 
 =item C
@@ -122,11 +143,6 @@ sub mtime ($)
 $atime,$mtime,$ctime,$blksize,$blocks) = stat ($file)
 or fatal "cannot stat $file: $!";
 
-  # Unfortunately Time::HiRes converts timestamps to floating-point, and the
-  # rounding error can be hundreds of nanoseconds for circa-2023 timestamps.
-  # Perhaps some day Perl will support accurate file timestamps.
-  # For now, do the best we can without going outside Perl.
-
   return $mtime;
 }
 

Re: rhel8 test failure confirmation?

[Zack Weinberg]

On Mon, Dec 4, 2023, at 7:14 PM, Jacob Bachmeyer wrote:
> Zack Weinberg wrote:
[snip everything addressed in the other thread]
> Yes, there was a bit of confusion here; not only is the FileUtils
> module synchronized between autom4te and automake

Thanks for reminding me that I need to make sure all those files are
actually in sync before I cut the final 2.72 release.

>>   require Time::HiRes;
>>   import Time::HiRes qw(stat);
>
> I believe that the import is not actually necessary

The previous line is a "require", not a "use", so I believe it _is_
necessary.  Have I misunderstood?

> ... should do no harm as long as any use of stat in the code
> is prepared to handle floating-point timestamps.

There's only one use, in 'sub mtime', and that's the place
where we actively want the floating-point timestamps.

zw

Re: [RFC PATCH]: autom4te: report subsecond timestamp support in --version

[Zack Weinberg]

On Mon, Dec 4, 2023, at 7:26 PM, Jacob Bachmeyer wrote:
> Now that I have seen the actual patch, yes, this test should be
> accurate.  The test in the main autom4te script will also work, even
> if there is a mismatch between the script and its library

Good.

> This appears to be misaligned with the GNU Coding Standards, which
> states:  "The first line is meant to be easy for a program to parse;
> the version number proper starts after the last space."
>
> Perhaps the best option would be to conditionally add a line "This
> autom4te supports subsecond timestamps." after the license notice?

I don't like putting anything after the license notice because it's
convenient to be able to pipe --version output to sed '/Copyright/,$d'
without losing anything relevant for troubleshooting.  So how about

$ autom4te --version
autom4te (GNU Autoconf) 2.71
Features: subsecond-timestamps

Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+/Autoconf: GNU GPL version 3 or later
, 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Akim Demaille.

This preserves the effectiveness of sed '/Copyright/,$d' and also
leaves room for future additions to the "Features:" line.

zw

[RFC PATCH]: autom4te: report subsecond timestamp support in --version

[Zack Weinberg]

The Automake test suite wants this in order to know if it’s safe to
reduce the length of various delays for the purpose of ensuring files
in autom4te.cache are newer than the corresponding source files.

* lib/Autom4te/FileUtils.pm: Provide (but do not export) a flag
  $subsecond_mtime, indicating whether the ‘mtime’ function reports
  modification time with precision greater than one second.
  Reorganize commentary and import logic for clarity.  Add
  configuration for emacs’ perl-mode to the bottom of the file.

* bin/autom4te.in ($version): If $Autom4te::FileUtils::subsecond_mtime
  is true, add the text “ (subsecond timestamps supported)” to the
  first line of --version output.
---
 bin/autom4te.in   |  6 ++--
 lib/Autom4te/FileUtils.pm | 71 ---
 2 files changed, 56 insertions(+), 21 deletions(-)

diff --git a/bin/autom4te.in b/bin/autom4te.in
index 38a61ac9..9a2e5f12 100644
--- a/bin/autom4te.in
+++ b/bin/autom4te.in
@@ -207,8 +207,10 @@ General help using GNU software: 
.
 
 # $VERSION
 # 
-$version = "autom4te (@PACKAGE_NAME@) @VERSION@
-Copyright (C) @RELEASE_YEAR@ Free Software Foundation, Inc.
+$version = "autom4te (@PACKAGE_NAME@) @VERSION@"
+  . ($Autom4te::FileUtils::subsecond_mtime
+ ? " (subsecond timestamps supported)\n" : "\n")
+  . "Copyright (C) @RELEASE_YEAR@ Free Software Foundation, Inc.
 License GPLv3+/Autoconf: GNU GPL version 3 or later
 , 
 This is free software: you are free to change and redistribute it.
diff --git a/lib/Autom4te/FileUtils.pm b/lib/Autom4te/FileUtils.pm
index c1e8e8c3..06f87c31 100644
--- a/lib/Autom4te/FileUtils.pm
+++ b/lib/Autom4te/FileUtils.pm
@@ -38,24 +38,45 @@ use 5.006;
 use strict;
 use warnings FATAL => 'all';
 
-use Exporter;
+BEGIN
+{
+  require Exporter;
+  our @ISA = qw (Exporter);
+  our @EXPORT = qw (
+   _file 
+   _file
+_hint 
+   _has_case_matching_file _dir_cache
+   _dir_cache_file);
+}
+
+# Use sub-second resolution file timestamps if available, carry on
+# with one-second resolution timestamps if Time::HiRes is not available.
+#
+# Unfortunately, even if Time::HiRes is available, we don't get
+# timestamps to the full precision recorded by the operating system,
+# because Time::HiRes converts timestamps to floating-point, and the
+# rounding error is hundreds of nanoseconds for circa-2023 timestamps
+# in IEEE double precision.  But this is the best we can do without
+# dropping down to C.
+#
+# $subsecond_mtime is not exported, but is intended for external
+# consumption, as $Autom4te::FileUtils::subsecond_mtime.
+BEGIN
+{
+  our $subsecond_mtime = 0;
+  eval
+{
+  require Time::HiRes;
+  import Time::HiRes qw(stat);
+  $subsecond_mtime = 1;
+}
+}
+
 use IO::File;
-
-# use sub-second resolution timestamps if available,
-# carry on with one-second resolution timestamps if that is all we have
-BEGIN { eval { require Time::HiRes; import Time::HiRes qw(stat) } }
-
 use Autom4te::Channels;
 use Autom4te::ChannelDefs;
 
-our @ISA = qw (Exporter);
-our @EXPORT = qw (
- _file 
- _file
-  _hint 
- _has_case_matching_file _dir_cache
- _dir_cache_file);
-
 =over 4
 
 =item C
@@ -122,11 +143,6 @@ sub mtime ($)
 $atime,$mtime,$ctime,$blksize,$blocks) = stat ($file)
 or fatal "cannot stat $file: $!";
 
-  # Unfortunately Time::HiRes converts timestamps to floating-point, and the
-  # rounding error can be hundreds of nanoseconds for circa-2023 timestamps.
-  # Perhaps some day Perl will support accurate file timestamps.
-  # For now, do the best we can without going outside Perl.
-
   return $mtime;
 }
 
@@ -394,3 +410,20 @@ sub set_dir_cache_file ($$)
 =cut
 
 1; # for require
+
+### Setup "GNU" style for perl-mode and cperl-mode.
+## Local Variables:
+## perl-indent-level: 2
+## perl-continued-statement-offset: 2
+## perl-continued-brace-offset: 0
+## perl-brace-offset: 0
+## perl-brace-imaginary-offset: 0
+## perl-label-offset: -2
+## cperl-indent-level: 2
+## cperl-brace-offset: 0
+## cperl-continued-brace-offset: 0
+## cperl-label-offset: -2
+## cperl-extra-newline-before-brace: t
+## cperl-merge-trailing-else: nil
+## cperl-continued-statement-offset: 2
+## End:
-- 
2.41.0

Re: rhel8 test failure confirmation?

[Zack Weinberg]

On Sun, Dec 3, 2023, at 4:49 PM, Karl Berry wrote:
>> There would not need to be much parsing, just "automake --version | grep 
> > HiRes" in that case, or "case `automake --version` in *HiRes*) ...;; 
> > easc" to avoid running grep if you want.
>
> I specifically want to hear what Karl thinks.
>
> I lean towards Jacob's view that automake --version | grep HiRes will
> suffice. Not having a new option seems simpler/better in terms of later
> understanding, too. --thanks, karl.

Did I misunderstand which program's --version output we are talking about?
I thought we were talking about automake's testsuite probing the behavior
of *autom4te*, but all the quoted text seems to be imagining a probe of
*automake* instead.  Yinz can change automake yourselves, you don't need
me to jump in :-)

Assuming this is really about autom4te, how does this look?

$ ./tests/autom4te --version
autom4te (GNU Autoconf) 2.72d.6-49ab3-dirty (subsecond timestamps supported)
Copyright (C) 2023 Free Software Foundation, Inc.
[etc]

If subsecond timestamps are *not* supported, you'll get instead

autom4te (GNU Autoconf) 2.72d.6-49ab3-dirty
Copyright (C) 2023 Free Software Foundation, Inc.
[etc]

I'm not using the identifier "HiRes" because the use of Time::HiRes is an
implementation detail that could change.  For instance, if there's a third
party CPAN module that lets us get at nanosecond-resolution timestamps
*without* loss of precision due to conversion to an NV (aka IEEE double)
we could, in the future, look for that first.

The implementation is just

BEGIN
{
  our $subsecond_timestamps = 0;
  eval
{
  require Time::HiRes;
  import Time::HiRes qw(stat);
  $subsecond_timestamps = 1;
}
}

Jacob, can you confirm that's an accurate test, given all the things you
said earlier about ways that grepping the source code might get it wrong?

zw

Re: rhel8 test failure confirmation?

[Zack Weinberg]

On Sat, Dec 2, 2023, at 8:58 PM, Jacob Bachmeyer wrote:
> Zack Weinberg wrote:
>> Either way is no problem from my end, but it would be more work
>> for automake (parsing --version output, instead of just checking the
>> exit status of autom4te --assert-high-resolution-timestamp-support)
>> Karl, do you have a preference here?  I can make whatever you decide
>> on happen, in the next couple of days.
>
> There would not need to be much parsing, just "automake --version | grep 
> HiRes" in that case, or "case `automake --version` in *HiRes*) ...;; 
> easc" to avoid running grep if you want.

I specifically want to hear what Karl thinks.

zw

bug#62896: [Configure] Bug with check for PERL when path has spaces (i.e. Windows)

[Zack Weinberg]

On Sat, Dec 2, 2023, at 9:08 PM, Mike Frysinger wrote:
> On 02 Dec 2023 15:53, Karl Berry wrote:
>> Exit status yes, but at least historically, grep -q has been
>> considered non-portable, in favor of grep ... >/dev/null.
>
> i get that `grep -q` is something historically we've avoided, but i
> think it's one of those bad habits we should just get out of.

Hmm, `grep -q` *is* in POSIX, but I seem to recall tripping over a
system that didn't have it (probably either a Solaris successor, or AIX)
during the run-up to Autoconf 2.71. I'm cc:ing Paul Eggert for comment
as I believe he's the person who most recently revised the grep section
of
https://www.gnu.org/software/autoconf/manual/autoconf-2.71/html_node/Limitations-of-Usual-Tools.html
.

zw

Re: rhel8 test failure confirmation?

[Zack Weinberg]

On Sat, Dec 2, 2023, at 7:33 PM, Jacob Bachmeyer wrote:
> Zack Weinberg wrote:
>> Would it help if we added a command line option to autom4te that made
>> it report whether it thought it could use high resolution timestamps?
>> Versions of autom4te that didn't recognize this option should be
>> conservatively assumed not to support them.
>
> Why not just add that information to the --version message?  Add a
> "(HiRes)" tag somewhere if Time::HiRes is available?

Either way is no problem from my end, but it would be more work
for automake (parsing --version output, instead of just checking the
exit status of autom4te --assert-high-resolution-timestamp-support)
Karl, do you have a preference here?  I can make whatever you decide
on happen, in the next couple of days.

>> (Of course there's the additional wrinkle that whether high
>> resolution timestamps *work* depends on what filesystem
>> autom4te.cache is stored in
>
> Is this actually still a problem (other than for ensuring the cache is
> used in the testsuite)

I don't *think* so but I don't understand the problem 100% so I
could be missing something.

zw

Re: rhel8 test failure confirmation?

[Zack Weinberg]

On Sat, Dec 2, 2023, at 6:37 PM, Karl Berry wrote:
> The best way to check if high-resolution 
> timestamps are available to autom4te is to have perl load 
> Autom4te::FileUtils and check if that also loaded Time::HiRes.
>
> The problem with that turned out to be that Time::HiRes got loaded from
> other system modules, resulting in the test thinking that autom4te used
> it when that wasn't actually the case. That's what happened in practice
> with your patch.

Would it help if we added a command line option to autom4te that made it report 
whether it thought it could use high resolution timestamps? Versions of 
autom4te that didn't recognize this option should be conservatively assumed not 
to support them.

(Of course there's the additional wrinkle that whether high resolution 
timestamps *work* depends on what filesystem autom4te.cache is stored in, but 
that's even harder to probe... one problem at a time?)

zw

Re: rhel8 test failure confirmation?

[Zack Weinberg]

On Sat, Dec 2, 2023, at 6:58 AM, Mike Frysinger wrote:
> On 06 Apr 2023 21:29, Jacob Bachmeyer wrote:
>> Karl Berry wrote:
>> > jb> a more thorough test would locate the autom4te script and grep it
>> > for the perllibdir that was substituted when autoconf was
>> > configured.
>> >
>> > I guess that would work.
>> 
>> Challenge accepted.  Here's a refined version: 
>
> this doesn't work on systems that wrap `autom4te`.  Gentoo for example wraps
> all autoconf & automake scripts to support parallel installs of different
> versions.  this way we can easily have access to every autoconf version.  we
> got this idea from Mandrake, so we aren't the only ones ;).
...
> what is the first autoconf release that has the fix ?
> -mike

2.71 does not have the fix. Can you please test the 2.72d beta that I just put 
out
(see https://lists.gnu.org/archive/html/autoconf/2023-11/msg0.html)?

(yes, the version numbering for autoconf betas looks strange, it's a convention
that predates semver and I don't have time to argue with anyone about it)

zw

Re: Detect --disable-dependency-tracking in Makefile.am

[Zack Weinberg]

On Sat, Sep 30, 2023, at 1:07 AM, Jan Engelhardt wrote:
> On Saturday 2023-09-30 05:27, Dave Hart wrote:
>
>>I've added code to the ntp.org Makefile.am files to ensure the static
>>utility library libntp.a is up-to-date for each program that uses it, to
>>ensure the build is correct.  When building the project, this adds a bunch
>>of extra submake invocations which slows down the build.  I'd like to omit
>>those when --disable-dependency-tracking is used, to speed one-off builds
>>that are done by end-users and packagers.
>
> submake is pretty much independent of and from dependency tracking.
>
> The general direction contemporary projects take is to just not do submake
> anymore, because it's a slowdown whether or not deptracking is used.

To elaborate on that: it's been known since *1997* that recursive make is
always a bad idea. See  for gory
details.

It's too bad that Automake still, 26 years later, can only support non-recursive
builds in a fairly clunky way (manually adding subdirectory prefixes to every
single object file).  It shouldn't be *that* hard to make "subdirs = lib bin"
mush all the rules derived from {lib,bin}/Makefile.am into the top-level
generated Makefile.in, at least when there aren't any manually written rules
in the subdirectories.

zw

[bug#65600] [PATCH] Allow parameters AM_PROG_LEX

[Zack Weinberg]

> Karl Berry , Fri Sep 01 2023 00:28:04 GMT+0200
> (Central European Summer Time)
>> Bogdan: I will adjust the patch, one way or another. Nothing more for
>> you to do here after all :). --thanks, karl.

Karl: The same problem was also reported as bug 65730, with a patch
supplied by Łukasz Stelmach.  That patch is preferable to Bogdan's
patch, for two reasons.  First, Łukasz's patch uses $@ to forward *all*
arguments to AC_PROG_LEX, regardless of what they are, making it robust
to the possibility of *more* arguments being added to AC_PROG_LEX in the
future (for example, we might decide to add an option to specify that
you need Flex specifically).  Second, Łukasz's patch checks whether
AC_PROG_LEX has already been invoked, and if so does not run it again.
That makes the updated AM_PROG_LEX continue to be compatible with the
workaround we (Autoconf) have been suggesting for this issue, namely
to write

AC_PROG_LEX([noyywrap])
AM_PROG_LEX

It looks like Bogdan's patch does include a new set of tests that are
worth preserving, but please use Łukasz's actual code change.  (I regret
I cannot volunteer to merge the two patches as I am in the middle of
reinstalling the operating system on my workstation, and that's already
taken a week and a half with no end in sight.  You know how these things
go, I'm sure.)

zw

bug#64968: automake --version isn't work

[Zack Weinberg]

On Mon, Jul 31, 2023, at 1:39 PM, Zack Weinberg wrote:
> On Mon, Jul 31, 2023, at 2:02 AM, 874882199--- via Bug reports for 
> Automake wrote:
>> In Docker(in the vmware ubunt also has the problem):
>> Environment:
>> Ubuntu 20.04 5.15.0-78-generic.
>> Automake1.4 from ftp://gcc.gnu.org/pub/java/automake-gcj-1.4.tar.gz.
>
> This is a very old version of automake.  Do you still have the problem 
> if you use https://ftp.gnu.org/gnu/automake/automake-1.16.tar.gz 
> instead?

correction: please try 
<https://ftp.gnu.org/gnu/automake/automake-1.16.5.tar.gz>,
which is really the most recent release.

zw

bug#64968: automake --version isn't work

[Zack Weinberg]

On Mon, Jul 31, 2023, at 2:02 AM, 874882199--- via Bug reports for Automake 
wrote:
> In Docker(in the vmware ubunt also has the problem):
> Environment:
> Ubuntu 20.04 5.15.0-78-generic.
> Automake1.4 from ftp://gcc.gnu.org/pub/java/automake-gcj-1.4.tar.gz.

This is a very old version of automake.  Do you still have the problem if you 
use https://ftp.gnu.org/gnu/automake/automake-1.16.tar.gz instead?

zw

Re: Issue with AM_PROG_LEX

[Zack Weinberg]

On Mon, Jul 31, 2023, at 7:37 AM, FX Coudert wrote:
> Hello,
>
> I have a configure.ac file that calls AM_PROG_LEX. This now generates 
> warnings:
...
> I am not sure I can actually fix those: AM_PROG_LEX does not seem to 
> accept an argument. Probably it should, and pass it down to AC_PROG_LEX 
> if provided?

Yes, this does ultimately need to be fixed in automake, but in the meantime you 
*may* be able to work around it by writing

AC_PROG_LEX([noyywrap])
AM_PROG_LEX

(Change "noyywrap" to "yywrap" if you actually need yywrap from libl.a.)

This will work iff AM_PROG_LEX uses AC_REQUIRE to invoke AC_PROG_LEX. I'm 
writing this on my phone so I can't easily check.

zw 

bug#63052: Bug Mite have been found

[Zack Weinberg]

On Mon, Apr 24, 2023, at 5:46 AM, Travis Goff wrote:
> /kalibrate-rtl$ bash /usr/share/automake-1.16/missing --help
> [... help output ...]

Could you please explain what went wrong here? Be as specific as possible. What 
do you think happened, and what do you think should having happened instead?

Re: rhel8 test failure confirmation?

[Zack Weinberg]

On Tue, Apr 4, 2023, at 3:51 PM, Bogdan wrote:
>   Nice. The 0 and 1 may not be portable to each OS in the Universe 
> (see EXIT_SUCCESS and EXIT_FAILURE in exit(3)), but should be 
> good/portable enough for our goals. Or maybe some other simple solution.

ISO C guarantees that exit(0) has the same effect as exit(EXIT_SUCCESS).
[It does *not*, however, guarantee that EXIT_SUCCESS == 0.]

zw

bug#33779: Color-coded output from autoconf/automake (was Re: bug#33779: Wrong lib-list in install-%DIR%LTLIBRARIES)

[Zack Weinberg]

On Mon, Jan 23, 2023, at 4:38 AM, Bert Wesarg via Bug reports for Automake 
wrote:
> On Fri, Jan 13, 2023 at 6:58 AM Mike Frysinger  wrote:
> No, I don't have one. It just crossed my eyes while working on more
> silent rules in Automake. I made Ben recently aware of these changes,
> which are availalbe here:
>
> https://github.com/bertwesarg/automake/commits/more-and-colorful-silence
>
> and also Autoconf is now more colorful:
>
> https://github.com/bertwesarg/autoconf/commits/color

Both of those URLs are 404 for me.

FYI, with my Autoconf hat on, I will not be inclined to take patches that 
hardcode "ANSI" terminal control codes.  $TERM must be honored in detail.  The 
easiest way to do that is probably to probe for the existence of the `tput` 
command and then use that to set shell variables with all the necessary control 
codes.  When working on patches for Automake, also keep in mind that $TERM 
might change between "./configure" and "make".

zw

[bug#60807] [PATCH v2] tests: reuse am_cv_filesystem_timestamp_resolution

[Zack Weinberg]

On Sat, Jan 14, 2023, at 7:18 PM, Mike Frysinger wrote:
> Rather than assume such coarse delays, re-use existing logic for
> probing the current filesystem resolution.  This speeds up the
> testsuite significantly.  On my system, it speeds -j1 up quite a
> lot -- by ~30%.  While I didn't gather many samples to produce a
> statistically significant distribution, my runs seem to be fairly
> consistent with the values below with deviations of <1 minute.

No objection to this patch in itself, but I want to make sure you're
aware that the "existing logic for probing the current filesystem
resolution" has a bug where, if you start running the script at just
the wrong time, it will erroneously detect a finer timestamp resolution
than the system actually supports.   For instance, if we can sleep for
0.1 second, the filesystem's timestamp resolution is 2s, and the sleep
loop happens to start executing at 0h00m59.9s, then it'll tick over to
0h01m00.0s and conftest.file.a and conftest.file.b will have distinct
timestamps.  This happens to me quite reliably: whenever I try to
run the Automake test suite inside AFS, I'll get a couple of spurious
test failures because of this bug.

zw

[bug#59994] [PATCH] tests: Don't try to prevent flex to include unistd.h

[Zack Weinberg]

On Fri, Jan 13, 2023, at 6:33 PM, Karl Berry wrote:
>> your patch *and* consistently test flex with "--never-interactive".
>
> Making flex non-interactive sounds desirable in any case, but
> --never-interactive is not mentioned in the 2.6.0 --help message.
> Instead there is -B (--batch), although --never-interactive is
> recognized, I can see experimentally.

--never-interactive is documented in the Texinfo manual ("Options
Affecting Scanner Behavior" node).  It looks to me like the --help
output intentionally lists only the most commonly used options.

--never-interactive and --batch are technically orthogonal.  I don't
know any reason a *real* scanner would want to use --never-interactive
*without* --batch.  However, to address the specific problem with *test*
scanners we are discussing here, --never-interactive is what we want,
because --never-interactive is the option that makes the generated code
not call isatty().

> I do not know when --never-interactive, or any of its variants, was
> implemented. We should check that before using it ...

I'm pleasantly surprised to report that the Git repo for flex at
 has history going back all the way
to 1987.

The *feature* of optionally not calling isatty() on the input file
has been present since version 2.5.1 (released 1995-03-28), in the
form of `%option {always,never}-interactive` directives inside the
scanner.  The *command line option* --never-interactive was added
in version 2.5.6 (along with long command line options in general).
That version was released somewhere between 2002-04-19 and 2002-04-23;
the exact date is not recorded anywhere I can find.

zw

Re: Builds with 'configure' not at the top of srcdir

[Zack Weinberg]

On Thu, Jan 12, 2023, at 9:24 PM, Eduardo Hernández wrote:
> I've been trying to separate the build system and source directory
> completely. Part of that would be to have the 'configure' script in the
> 'build' directory, away from the 'src' directory

This goes against a basic design assumption in both Automake and Autoconf: you 
are expected to be making tarball releases that include the configure script 
and a bunch of other generated files (aclocal.m4, config.h.in, Makefile.in if 
Automake is used, etc.)  That assumption goes back to an era when most software 
distribution did _not_ involve direct access to the core developers' version 
control system (if there even _was_ one) and when asking people to install 
Automake and Autoconf (and all the stuff _they_ depend on, notably M4 and 
Perl), before they could install any _other_ GNU software, would've been a 
non-starter.

So, all the generated code assumes that the location of the configure script 
*defines* the top level of the source tree.  The `--srcdir` option is only 
intended for use as a manual override when the automatic probe for the location 
of the configure script doesn't work.

This design arguably could stand to be reconsidered, but it'll be a lot of work 
to get to a place where you can do the thing you would like to do.  Are you up 
for that work?

zw

[bug#60535] [PATCH] depend2: switch echo|sed to automatic vars

[Zack Weinberg]

On Thu, Jan 5, 2023, at 6:47 PM, Karl Berry wrote:
> Please excuse my curmudgeonness, but it's not clear to me that avoiding
> sed is worth these hassles in working around the implementation-specific
> bugs in the automatic variables.

It seems to me that this would be worth doing *if* we could get it to the
point where gmake (and any other implementation with similar optimizations)
didn't need to invoke a shell for the command at all.  I thought the rules
for that were documented but I can't find them at the moment, and I got lost
in gmake's job.c's maze of #ifdefs; anyone know what they are?

It might also be worth doing if we could guarantee that `sed` would not be
used by *any* rules, as that would be a solid step in the direction of
removing `sed` from the set of shell utilities potentially needed at build time
for a generic package (of course, if the package's own build logic uses sed,
there's nothing we can do about that).  ISTR there are several existing automake
rules with complex embedded shell scripts that use sed, though.

zw

[bug#59993] [PATCH 2/2] tests: Fix implicit function declaration errors

[Zack Weinberg]

On Thu, 15 Dec 2022 11:48:44 -0500, Frederic Berat wrote:
> On Mon, Dec 12, 2022 at 9:19 PM Zack Weinberg  wrote:
> > > return yylex ();
> > Please change `int main ()` to `int main (void)` as long as you're in here.
> I'm hesitant on this. That isn't the purpose of the patch, and if I
> start to do that, there are about 48 files that would need to be
> touched (which omit void for empty argument list).

Yeah, I suppose that ought to be its own separate patch, in that
case.

> Kind of a noisy change with no real benefit.

It’d be a step in the direction of making the testsuite run cleanly
with -Wstrict-prototypes and/or -Wold-style-definition, which might
plausibly help with the related effort to support compilers that
reject old-style definitions.  But I agree that it’s not a high
priority and shouldn’t block this patch.

zw

[bug#59992] [PATCH 1/2] tests: Fix 'type defaults' error in link_cond due to main not being properly declared

[Zack Weinberg]

On Thu, Dec 15, 2022, at 2:12 AM, Frederic Berat wrote:
> For the record:
>
> cat > less.c <<'END'
> /* Valid C but deliberately invalid C++ */
> int main (void)
> {
>   int new = 0;
>   return new;
> }
> END
>
> $> g++ less.c
> less.c: In function ‘int main()’:
> less.c:4:7: error: expected unqualified-id before ‘new’
> 4 |   int new = 0;
>   |   ^~~
> less.c:5:13: error: expected type-specifier before ‘;’ token
> 5 |   return new;
>   | ^

Thanks, I haven't had time to look this up myself. Please go ahead and commit 
the patch.

zw

[bug#59993] [PATCH 2/2] tests: Fix implicit function declaration errors

[Zack Weinberg]

On Tue, Dec 13, 2022, at 1:30 AM, Frederic Berat wrote:
> On Mon, Dec 12, 2022 at 9:19 PM Zack Weinberg  wrote:
>> > --- a/t/ax/depcomp.sh
>> > +++ b/t/ax/depcomp.sh
>> > @@ -243,6 +243,7 @@ cat > sub/subfoo.h <<'END'
>> >   #include 
>> >   extern int subfoo (void);
>> >   END
>> > +cp sub/subfoo.h sub/subfoo.save
>> >
>> >   cat > src/baz.c <<'END'
>> >   #include "baz.h"
>> > @@ -399,8 +400,7 @@ do_test ()
>> > && rewrite "$srcdir"/sub/subfoo.h echo 'choke me' \
>> > && not $MAKE \
>> > && delete "$srcdir"/sub/subfoo.h \
>> > -  && edit "$srcdir"/sub/subfoo.c -e 1d \
>> > -  && edit "$srcdir"/foo.h -e 2d \
>> > +  && mv  "$srcdir"/sub/subfoo.save "$srcdir"/sub/subfoo.h \
>> > && make_ok \
>> > || r='not ok'
>> >   result_ "$r" "$pfx dependency tracking works"
>>
>> These changes don't seem to have anything to do with the patch as
>> described.  They should be submitted separately.
>
> Actually, that is related. The edit removes the "#include subfoo.h"
> line from foo.h, because the subfoo.h file got overwritten earlier.
> This subfoo.h is the one that had the function declaration in ("extern
> int subfoo (void);" specifically).
> Either you need the original header back, or you need to get the
> declaration back another way. I felt this way was easier than trying
> to insert a declaration without breaking the test.

OK, I clearly don't understand what this test is doing, so I'm going to
defer to someone else to review it.

zw

[bug#59991] [PATCH 0/2] Port tests to modern C

[Zack Weinberg]

On Mon, Dec 12, 2022, at 5:57 PM, Karl Berry wrote:
> Zack, would you like be co-maintainer or at least co-developer of
> Automake? There is, evidently, no one else in the world interested in
> being actively involved with Automake on the maintainer side.

I have to decline; I don't have anything like the time required and also I am 
almost completely unfamiliar with the innards of Automake (except for the 
handful of Perl modules shared with Autoconf).

zw

Re: [bug#59991] [PATCH 0/2] Port tests to modern C

[Zack Weinberg]

On Mon, Dec 12, 2022, at 5:57 PM, Karl Berry wrote:
> Zack, would you like be co-maintainer or at least co-developer of
> Automake? There is, evidently, no one else in the world interested in
> being actively involved with Automake on the maintainer side.

I have to decline; I don't have anything like the time required and also I am 
almost completely unfamiliar with the innards of Automake (except for the 
handful of Perl modules shared with Autoconf).

zw

[bug#59994] [PATCH] tests: Don't try to prevent flex to include unistd.h

[Zack Weinberg]

On 2022-12-12 2:07 AM, Frederic Berat wrote:


This patch is mainly a proposal. While the macro can simply be removed
as explained below, another possibility it to add a flex option
"--never-interactive" to prevent flex to make use of "isatty".

This is related to an effort to prepare Automake for future GCC/Clang
versions which set c99 as default standard to be used.

Future version of flex make the "NO_UNISTD" flag a no-op, and include
unistd.h by default.

In current version of flex, not having this header leads to implicit
function declarations that are not compatible with c99 standard.

On top of that, while flex dedicated test were having this macro set,
the yacc ones didn't have it despise their use of Flex. Thus, if this
macro was ever useful, it looks like nobody actually cared.


I'm guessing that the idea here was to avoid using unistd.h on platforms 
that don't supply that header at all.  I suggest that we should apply 
your patch *and* consistently test flex with "--never-interactive".


zw

[bug#59993] [PATCH 2/2] tests: Fix implicit function declaration errors

[Zack Weinberg]

On 2022-12-12 2:05 AM, Frederic Berat wrote:

This is related to an effort to prepare Automake for future GCC/Clang
versions which set c99 as default standard to be used.
Function should be properly declared prior to use in order to be
compatible with c99 standard.


OK in principle, but ...


--- a/t/ax/depcomp.sh
+++ b/t/ax/depcomp.sh
@@ -243,6 +243,7 @@ cat > sub/subfoo.h <<'END'
  #include 
  extern int subfoo (void);
  END
+cp sub/subfoo.h sub/subfoo.save
  
  cat > src/baz.c <<'END'

  #include "baz.h"
@@ -399,8 +400,7 @@ do_test ()
&& rewrite "$srcdir"/sub/subfoo.h echo 'choke me' \
&& not $MAKE \
&& delete "$srcdir"/sub/subfoo.h \
-  && edit "$srcdir"/sub/subfoo.c -e 1d \
-  && edit "$srcdir"/foo.h -e 2d \
+  && mv  "$srcdir"/sub/subfoo.save "$srcdir"/sub/subfoo.h \
&& make_ok \
|| r='not ok'
  result_ "$r" "$pfx dependency tracking works"


These changes don't seem to have anything to do with the patch as 
described.  They should be submitted separately.



--- a/t/lex-nodist.sh
+++ b/t/lex-nodist.sh
@@ -62,6 +62,8 @@ CLEANFILES = $(nodist_prog_SOURCES)
  END
  
  cat > main.c << 'END'

+extern int yylex (void);
+
  int main ()
  {
return yylex ();


Please change `int main ()` to `int main (void)` as long as you're in here.


--- a/t/ltconv.sh
+++ b/t/ltconv.sh
@@ -91,6 +91,12 @@ echo 'int sub22 () { return 22; }' > sub2/sub22/sub22.c
  
  cat >test.c <<'EOF'

  #include 
+
+extern int sub1 (void);
+extern int sub2 (void);
+extern int sub21 (void);
+extern int sub22 (void);
+
  int main ()
  {
if (1 != sub1 ())


Same here.


--- a/t/yacc-basic.sh
+++ b/t/yacc-basic.sh
@@ -49,6 +49,7 @@ a : 'a' { exit(0); };
  END
  
  cat > foo.c << 'END'

+extern int yyparse(void);
  int main () { yyparse (); return 1; }
  END


And here.


--- a/t/yacc-clean.sh
+++ b/t/yacc-clean.sh
@@ -74,6 +76,8 @@ END
  cp sub1/parse.y sub2/parse.y
  
  cat > sub1/main.c << 'END'

+extern int yyparse(void);
+
  int main ()
  {
return yyparse ();


And here.


--- a/t/yacc-nodist.sh
+++ b/t/yacc-nodist.sh
@@ -78,6 +78,8 @@ BUILT_SOURCES = parse.h
  END
  
  cat > sub1/main.c << 'END'

+extern int yyparse(void);
+
  int main ()
  {
return yyparse ();


And here as well.

[bug#59992] [PATCH 1/2] tests: Fix 'type defaults' error in link_cond due to main not being properly declared

[Zack Weinberg]

On 2022-12-12 2:05 AM, Frederic Berat wrote:

This is related to an effort to prepare Automake for future GCC/Clang
versions which set c99 as default standard to be used.
Not properly declaring main as "int main(...)" is rejected since c99.

...

  /* Valid C but deliberately invalid C++ */
-main ()
+int main (void)


Note the comment.  Is this test program still invalid C++ after your change?

zw

Re: Question WHY is gnu make does not stop on error on rule

[Zack Weinberg]

On Thu, Dec 8, 2022, at 5:18 AM, aotto wrote:> Hi,
> I use "automake" to setup a "gnu make" build-environment and I have the 
> following rule to create a special file:
> Problem: the "$(c_Meta)" failed with error but MAKE continue… why??
...
> $(csmkkernel_meta) $(csmqmsgque_meta) $(cslcconfig_meta):
>     @$(call green,$(c_Meta) -meta $@ -header $<)
>     @$(c_Meta) -meta $@ -header $< && touch $@
...
>    (file ".../NHI1/sbin/meta/c_Meta.tcl" line 1441)
> Reaping losing child 0x55901938a2b0 PID 26076
> Removing child 0x55901938a2b0 PID 26076 from chain.
> Successfully remade target file 
> '.../NHI1/theLink/csmsgque/.LibMqMsgque_cs.meta'.

Please: Get rid of those leading @ signs, run it again *without* -d, and post 
the *complete and unedited* output.

zw

Re: make check(s) pre-release problems

[Zack Weinberg]

Please don't top-post on this mailing list.

On Tue, Oct 11, 2022, at 12:15 PM, Frederic Berat wrote:
> On Fri, Oct 7, 2022 at 6:11 PM Zack Weinberg  wrote:
>> On Thu, Oct 6, 2022, at 4:25 PM, Karl Berry wrote:
>>>> No errors on RHEL7+autoconf2.71
>>>
>>> Puzzling. Can you easily try RHEL8 or one of its derivatives?
>>> It surprises me that that is the culprit, but it seems possible.
>>
>> Unfortunately, no.
>
> I don't know if that will help, or if that is completely unrelated,
> but I'm currently stumbling into a weird issue while working on a
> new package release for autoconf on Fedora: about 200 tests are now
> failing, all related to aclocal checks.
>
> My current investigation shows that it would be related to a bash
> update from 5.1.x to 5.2x which seems to have changed the behavior
> of the "SHLVL" shell variable.

This is a known issue, fixed on Autoconf development trunk, see
https://git.savannah.gnu.org/cgit/autoconf.git/commit/?id=412166e185c00d6eacbe67dfcb0326f622ec4020

I intend to make a bugfix release of Autoconf in the near future.

> I actually wonder if your sudden "parallelism" failure could be
> somehow linked to an update of bash, similar to mine ?

It's certainly possible.

zw

_AM_FILESYSTEM_TIMESTAMP_RESOLUTION incorrect result (was Re: make check(s) pre-release problems)

[Zack Weinberg]

On Thu, Oct 6, 2022, at 4:19 PM, Zack Weinberg wrote:
> On Thu, Oct 6, 2022, at 1:04 PM, Zack Weinberg wrote:
>> On 2022-10-04 6:58 PM, Karl Berry wrote:
>>> Perhaps easier to debug: there are two targets to be run before making a
>>> release, check-no-trailing-backslash-in-recipes and check-cc-no-c-o,
>>> to try to ensure no reversion wrt these features. A special shell and
>>> compiler are configured, respectively (shell scripts that check the
>>> behavior).
>>
>> I'm running these targets now and will report what I get.
>
> No errors on RHEL7+autoconf2.71 (same environment I used for the Python fixes)
> from a serial "make check-no-trailing-backslash-in-recipes".  The other one is
> running now.

One failure from a serial "make check-cc-no-c-o":

FAIL: t/aclocal-autoconf-version-check
==

Running from installcheck: no
Test Protocol: none
...
configure: error: newly created file is older than distributed files!
Check your system clock
make: *** [config.status] Error 1

This doesn't appear to have anything to do with "make check-cc-no-c-o" mode, 
the problem is that the filesystem timestamp resolution was incorrectly 
detected:

configure:1965: checking whether sleep supports fractional seconds
configure:1979: result: true
configure:1982: checking the filesystem timestamp resolution
configure:2020: result: 0.1
configure:2024: checking whether build environment is sane
configure:2079: error: newly created file is older than distributed files!
Check your system clock

The filesystem I'm working in only records timestamps at second granularity. (I 
don't know why ls is printing .1 instead of .0 but it's always 
.1.)

$ touch A && sleep 0.1 && touch B && ls --full-time -t A B
-rw-r--r-- 1 zweinber users 0 2022-10-07 11:20:05.1 -0400 A
-rw-r--r-- 1 zweinber users 0 2022-10-07 11:20:05.1 -0400 B

I *think* this is a bug in _AM_FILESYSTEM_TIMESTAMP_RESOLUTION where, if it 
starts looping at *just the wrong time*, the first 0.1-second sleep will 
straddle a second boundary and we'll break out of the loop immediately and 
think we have 0.1-second timestamp resolution.

zw

Re: make check(s) pre-release problems

[Zack Weinberg]

On Thu, Oct 6, 2022, at 4:25 PM, Karl Berry wrote:
> No errors on RHEL7+autoconf2.71
>
> Puzzling. Can you easily try RHEL8 or one of its derivatives?
> It surprises me that that is the culprit, but it seems possible.

Unfortunately, no.  CMU is mostly an Ubuntu shop these days.  It's only dumb 
luck that I happen to have access to a machine that hasn't had a major system 
upgrade since 2012 (and with my day-job hat on I'm actively trying to *get* it 
upgraded -- to Ubuntu.)

zw

Re: make check(s) pre-release problems

[Zack Weinberg]

On Thu, Oct 6, 2022, at 1:04 PM, Zack Weinberg wrote:
> On 2022-10-04 6:58 PM, Karl Berry wrote:
>> Perhaps easier to debug: there are two targets to be run before making a
>> release, check-no-trailing-backslash-in-recipes and check-cc-no-c-o,
>> to try to ensure no reversion wrt these features. A special shell and
>> compiler are configured, respectively (shell scripts that check the
>> behavior).
>
> I'm running these targets now and will report what I get.

No errors on RHEL7+autoconf2.71 (same environment I used for the Python fixes)
from a serial "make check-no-trailing-backslash-in-recipes".  The other one is
running now.

zw

Re: make check(s) pre-release problems

[Zack Weinberg]

On 2022-10-04 6:58 PM, Karl Berry wrote:

With Zack's latest Python fixes, I was hoping to move towards an
Automake release, but I find myself stymied by apparently random and
unreproducible test failures. I haven't exhausted every conceivable
avenue yet, but I thought I would write in hopes that others (Zack, past
Automake developers, anyone else ...) could give it a try, and/or have
some insights.

For me, running a parallel make check (with or without parallelizing the
"internal" makes), or make distcheck, fails some tests, e.g., nodef,
nodef2, testsuite-summary-reference-log. The exact tests that fail
changes from run to run. Running the tests on their own succeeds. Ok, so
it's something in the parallelism. But why? And how to debug?


I can't reproduce this problem myself, but my first thought is that some 
of the tests, when run concurrently, could be overwriting each other's 
files somehow.  I can think of two ways to investigate that hypothesis: 
look for tests that write files outside a directory dedicated to that 
test, and, after a failed test run, look for files that are corrupted,

then try to figure out which tests would be stomping on those files.


Perhaps easier to debug: there are two targets to be run before making a
release, check-no-trailing-backslash-in-recipes and check-cc-no-c-o,
to try to ensure no reversion wrt these features. A special shell and
compiler are configured, respectively (shell scripts that check the
behavior).


I'm running these targets now and will report what I get.

zw

Re: python debugging on trunk

[Zack Weinberg]

On Mon, Sep 26, 2022, at 12:23 PM, Karl Berry wrote:
> Is anyone up for debugging some Python-related test failures on
> RHEL-based systems?

I have access to a RHEL7 system, I know Python, and this sounds much
less unpleasant than everything else I'm supposed to be doing today.

> I have a suspicion the problem is that on RHEL systems, "python" invokes
> "python2" (because that's what the Python maintainers recommend, as I
> understand it), but the tests are assuming it invokes "python3".

Indeed:

$ lsb_release -d
Description: Red Hat Enterprise Linux Workstation release 7.9 (Maipo)
$ python --version
Python 2.7.5
$ python3 --version
Python 3.6.8

I have very strong opinions on this topic, having had to deal with a
lot of badly-written Python 2 code that, in the worst case, silently
corrupts data when run under Python 3.  IMNSHO, the bare command name
"python" and the absolute pathname "/usr/bin/python" MUST[rfc2119] be
*permanently reserved* for the Python 2 interpreter.  It's fine for
them not to exist at all, but if they exist and they invoke Python 3,
that is a *bug* in the distribution.

(Yes, there's a PEP that says something different -- IMO it is wrong,
its authors probably hadn't encountered the *really* troublesome
Python 2 code that I have.)

---
RHEL7, unsurprisingly, ships with Autoconf 2.69.  Before attempting to
build Automake from Git, I built Autoconf 2.71 from the tarball
release, installed it in a subdirectory of my homedir, and made that
directory come first on PATH.  There were no unexpected test failures
in Autoconf's testsuite. Other relevant tool versions include:

$ perl --version
This is perl 5, version 16, subversion 3 (v5.16.3) built for 
x86_64-linux-thread-multi
(with 44 registered patches, see perl -V for more detail)

$ m4 --version
m4 (GNU M4) 1.4.16

$ /bin/sh --version
GNU bash, version 4.2.46(2)-release (x86_64-redhat-linux-gnu)

$ make --version
GNU Make 3.82

$ sort --version
sort (GNU coreutils) 8.22

$ gcc --version
gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44)

$ libtool --version
libtool (GNU libtool) 2.4.2

I see these Automake testsuite failures:

Testsuite summary for GNU Automake 1.16i

# TOTAL: 2988
# PASS:  2877
# SKIP:  54
# XFAIL: 39
# FAIL:  18
# XPASS: 0
# ERROR: 0

FAIL: t/canon7
FAIL: t/fort5
FAIL: t/instmany-python
FAIL: t/ltinit
FAIL: t/nobase-python
FAIL: t/pr401b
FAIL: t/py-compile-basedir
FAIL: t/py-compile-basic
FAIL: t/py-compile-destdir
FAIL: t/py-compile-option-terminate
FAIL: t/python-pr10995
FAIL: t/python-prefix
FAIL: t/python-vars
FAIL: t/python10
FAIL: t/python12
FAIL: t/python3
FAIL: t/subobj9
FAIL: t/suffix5

That's _most_ of the failures you got, plus some more:

+FAIL: t/canon7
+FAIL: t/fort5
+FAIL: t/ltinit
+FAIL: t/pr401b
+FAIL: t/python10
+FAIL: t/subobj9
+FAIL: t/suffix5
-FAIL: t/subobj-pr13928-more-langs

There are only two failure modes:

canon7, fort5, ltinit, pr401b, subobj9, suffix5

ERROR: files left in build directory after distclean:
./.__afs

I did the build in an AFS volume; these are the AFS equivalent of
the NFS "silly rename" hack (apparently it can happen with SMB as well).
"make distcleancheck" should ignore these.

instmany-python, nobase-python, py-compile-basic, py-compile-destdir,
py-compile-option-terminate, pyghon-pr10995, python-prefix,
python-vars, python10, python12, python3

AttributeError: 'module' object has no attribute 'implementation'

This is coming from this code in lib/py-compile (there are three
copies of it):

if hasattr(sys.implementation, 'cache_tag'):
py_compile.compile(filepath, 
importlib.util.cache_from_source(filepath), path)
else:
py_compile.compile(filepath, filepath + 'c', path)

sys.implementation was added in Python 3.3 and
importlib.util.cache_from_source was added in 3.4.
Python 2.7's importlib _only_ has 'import_module'.

The appended patch should address both issues.  Note that I have only
tested it with Python 2.7 and 3.6.  It _may_ not be correct for Python
in the 3.0--3.3 (inclusive) range; I cannot conveniently test that.

zw

  * lib/am/distdir.am (distcleancheck_listfiles): Filter out "silly rename"
  files, unavoidably created by deleting files that are still open in some
  process on various network file systems.

  * lib/py-compile: Test directly for availability of
  importlib.util.cache_from_source.  Untangle logic for when to generate
  -O and -OO bytecode.  Reformat embedded Python fragments.

diff --git a/lib/am/distdir.am b/lib/am/distdir.am
index 0f591266d..b8d90e572 100644
--- a/lib/am/distdir.am
+++ b/lib/am/distdir.am
@@ -568,7 +568,9 @@ distuninstallcheck:
 ## Define distcleancheck_listfiles and distcleancheck separately
 ## from distcheck, so that they can be overridden by the user.
 .PHONY: distcleancheck
-distcleancheck_listfiles = find . -type f -print
+distcleancheck_listfiles = \
+  find . \( -type f -a \! \
+  \( -name .nfs* -o -name .smb* -o -name .__afs* \) \) -print
 distcleancheck: 

bug#58025: [PATCH] Ensure `byte-compile-dest-file-function' is used

[Zack Weinberg]

On Sat, Sep 24, 2022, at 5:45 AM, Richard Hopkins wrote:
> On 2022-09-23 16:15, Zack Weinberg wrote:
>> Thank you for the patch.  Are you able to test it with a version of
>> GNU Emacs older than 23.2?  I see that you tested it with XEmacs 21,
>> but as I recall there were quite substantial differences between
>> XEmacs 21 and GNU Emacs of similar vintage.
>
> I've now managed to test this on Emacs 21.4.1 (Slackware 12.0) and
> the byte compilation works - loading bytecomp is fine, and
> `byte-compile-dest-file' is defined as expected.  The other patch
> to respect silent rules also works on 21.4.1.
>
> If we do need to support that far back I will investigate the "-Q"
> / "--no-site-file" handling to improve compatibility across the
> board.

I'm not an official maintainer for Automake, but I think we probably
don't have to worry about Emacs any older than v21.  Would anyone
else like to express an opinion?

> "-Q" will error before GNU Emacs 22, and is ignored on XEmacs.
>
> "-no-site-file" (single hypen) should be used instead of
> "--no-site-file" as it works on all of them.
>
> "-Q" also shouldn't be specified on later GNU Emacs as it affects
> the result of `am_cv_lispdir' calculation due to excluding
> site lisp directories from `load-path' which it's trying to find.
> This is because "-Q" also adds "--no-site-lisp" in later GNU Emacs.
>
> So, the plan will be to not use "-Q" and to use "-q -no-site-file"
> instead.

This sounds like it would be a worthwhile change regardless of where
we decide to draw the line on supporting old versions of Emacs.

zw

bug#58025: [PATCH] Ensure `byte-compile-dest-file-function' is used

[Zack Weinberg]

On Fri, Sep 23, 2022, at 8:24 AM, em...@unbit.co.uk wrote:
> The attached `git format-patch` is based on automake v1.16.5 and fixes
> the following warning
>
>  Warning (bytecomp): byte-compile-dest-file is obsolete (as of 23.2);
>  Set byte-compile-dest-file-function instead.
>
> The solution is to ensure bytecomp is loaded which defines
> byte-compile-dest-file-function so it can be used when available,
> and fallback to the original byte-compile-dest-file for earlier
> GNU Emacs and XEmacs.
>
> So far I've tested the result on
>
> * CentOS 7.9 (distro emacs 24.3)
> * OpenBSD 7.1 (custom emacs 28.2)
> * OpenSUSE Leap 15.4 (distro emacs 27.2, xemacs 21.5)

Thank you for the patch.  Are you able to test it with a version of GNU Emacs 
older than 23.2?  I see that you tested it with XEmacs 21, but as I recall 
there were quite substantial differences between XEmacs 21 and GNU Emacs of 
similar vintage.

In addition, if you are able to do the archaeology to report *when bytecomp.el 
was added to Emacs*, i.e. how far back you have to go before `emacs -l 
bytecomp` will fail, that would be helpful.  I'm betting it's well before the 
oldest version we care about, given that bytecomp.el (in 27.1) lists its oldest 
copyright year as 1985 and  as the original author, but I'd 
still like to know for sure.

zw

bug#32868: Check if make supports nested variables is done twice when using silent rules

[Zack Weinberg]

> i'm inclined to bring this back as the way to opt-in to silent-rules
> by default.

And I’m still absolutely opposed to making it even *possible* to have silent 
rules be on by default. In fact I’d like to see —enable-silent-rules removed, 
with make V=0 the *only* way to switch to silent mode.

Re: bug#19539: [1.15] AC_CONFIG_AUX_DIR should be called early

[Zack Weinberg]

On Wed, Feb 9, 2022, at 1:02 AM, Mike Frysinger wrote:
> On 08 Feb 2022 22:11, Glenn Morris wrote:
>>
>> I see autoconf uses savannah for bug reports.
>> So long as simply sending a mail to bug-autoconf doesn't open a savannah 
>> issue,
>> I can set the autoconf maintainer to bug-autoconf, so that mails go to
>> the right place. (It tends not to work well if two bug trackers talk to
>> each other.)
>
> Zach: do you know if there's an e-mail<->savannah gateway ?

I don't know either way, but I do know that bug-autoconf is just a regular 
mailing list, not connected to savannah issue tracking.  So Glenn's suggestion 
is probably the best for the short term.

This situation is exactly why I was thinking about switching autoconf over to 
debbugs: it's _useful_ for automake to be able to reassign bugs to autoconf and 
vice versa.  In fact, just last week I had to tell someone to re-file a bug 
against automake because we don't have the ability to reassign bugs in that 
direction.  (https://savannah.gnu.org/support/?110599)  However, I do not have 
time right now to wrangle the switch-over, and I also don't know how to get the 
existing set of open autoconf bugs migrated into debbugs.  (The closed bugs can 
stay where they are.)  Anyone reading this have time to tackle the work?

zw

Re: [PATCH] python: prioritize python 3.x over 2.x

[Zack Weinberg]

On Thu, Jan 27, 2022, at 9:20 PM, Jacob Bachmeyer wrote:
>> Now, nobody listens to me, and I'm painfully aware that "python"
>> pointing to Python 3 is common, but that doesn't mean Autotools should
>> make the situation even worse.  Autotools should, on its own motion,
>> discourage the use of "python".  Thus, my suggested changes.
>
> Are Python 2 and Python 3 different enough that perhaps Automake should 
> have separate AM_PATH_PYTHON2 and AM_PATH_PYTHON3 macros, with 
> AM_PATH_PYTHON deprecated, since few programs can usefully run with 
> either Python 2 or 3?

Long term, yes, probably.

zw

Re: [PATCH] python: prioritize python 3.x over 2.x

[Zack Weinberg]

On Wed, Jan 26, 2022, at 10:10 PM, Mike Frysinger wrote:
> On 26 Jan 2022 10:07, Zack Weinberg wrote:
>> Please also move the plain "python" command down to the python2
>> block (right after "python2" would be the best place I think).  Any
>> system on which "python" runs Python 3 is grossly misconfigured.
>
> i don't think that's accurate.  Python PEP-0394 explicitly permits
> `python` to be any version, and many distros do just that

Yes, I know.  PEP 394 is wrong.  Its authors grossly underestimated
the volume of Python 2 code that still exists today and, if
accidentally executed with a v3 interpreter, will *silently*
malfunction -- not just crashing, but corrupting data with no warning.
I had to deal with this kind of code all the time in my previous day
job and I'm still quite peeved that the Python devs blew me off about it.

What PEP 394 *should* have said is that the command name "python" and
the #! path /usr/bin/python MUST[rfc2119] be *permanently reserved*
for the Python 2.7 interpreter, and "python3" is the only acceptable
not-specifically-versioned command name for the 3.x interpreter, even
in virtualenvs.

> distros that remove `python` entirely as part of Python 2 removal
> are doing it wrong.

On the contrary, those distros are the only ones doing it right.

Now, nobody listens to me, and I'm painfully aware that "python"
pointing to Python 3 is common, but that doesn't mean Autotools should
make the situation even worse.  Autotools should, on its own motion,
discourage the use of "python".  Thus, my suggested changes.

> if your argument is that language incompatibilities make supporting
> Python 2 and Python 3 with a single interpreter name either
> difficult or impossible (due to incompatible syntax changes), i'll
> remind you that Python 3 is not actually that much better here.

No, my argument is that there is a huge (millions of lines of code at
least) installed base of programs written for Python 2 string
semantics, that may *never* be updated to Python 3, and that can
*silently corrupt data* if executed by accident with a v3 interpreter.
They run without complaint but they do the Wrong Thing.

(Whether you've ever seen such a program strongly depends on the
circles you work in.  It's possible to code defensively enough that
your Python 2 programs crash reliably under the v3 interpreter, and I
expect most *packaged* software is written by people who do that
without even thinking about it, so you might not realize it's possible
to get the silent data corruption, but in the land of one-off
unpackaged scripts (that nonetheless need to work reliably for
decades) this phenomenon is everywhere.)

> the Python 3.0 - 3.8 series might have been *mostly* stable to try
> and encourage people to migrate off of Python 2.7, there were
> versions that changed syntax and broke things, and it's only going
> to get worse now that Python 2.7 is "gone".  off the top of my head,
> Python 3.5 added type hints, and Python 3.7 added async/await
> keywords.  both of those completely break in older versions of
> Python 3.

As far as I know these things can only cause SyntaxError crashes if
you run a program with an interpreter that's too old, not silent data
corruption, so it's not at all the same thing in my book.

zw

Re: [PATCH] python: prioritize python 3.x over 2.x

[Zack Weinberg]

On Wed, Jan 26, 2022, at 10:07 AM, Zack Weinberg wrote:
> On Wed, Jan 26, 2022, at 6:37 AM, Mike Frysinger wrote:
>> Since Python 2.x went EOL years ago, stop searching for it before
>> any of the Python 3 versions.
>
> +1

It occurred to me right after pushing "send" that this is a potentially 
breaking change, for anyone who hasn't bothered to update their code to Python 
3 yet and is relying on AM_PATH_PYTHON's search order to prefer Python 2.  I'm 
still in favor of the change, but I think it should get a NEWS entry.

zw

bug#53530: [PATCH] python: add 3.10 - 3.15 to the version search list

[Zack Weinberg]

On Wed, Jan 26, 2022, at 6:37 AM, Mike Frysinger wrote:
> Fixes automake bug https://bugs.gnu.org/53530.
>
> Based on the cadence of Automake releases, add the current Python
> release (3.10), the current Python development (3.11), and then 4
> more versions on top of that.  It doesn't hurt to check for a few
> extra versions here since this is the fallback logic when the main
> `python` and `python3` programs aren't found.

It's unfortunate that we keep having to update this list.  I wonder if there's 
a clean way to detect all the available /^python\d\.\d+/ commands and then sort 
them.

zw

Re: [PATCH] python: prioritize python 3.x over 2.x

[Zack Weinberg]

On Wed, Jan 26, 2022, at 6:37 AM, Mike Frysinger wrote:
> Since Python 2.x went EOL years ago, stop searching for it before
> any of the Python 3 versions.

+1

>m4_define_default([_AM_PYTHON_INTERPRETER_LIST],
> -[python python2 python3 dnl
> +[python python3 dnl
>   python3.15 python3.14 python3.13 python3.12 python3.11 python3.10 dnl
>   python3.9 python3.8 python3.7 python3.6 python3.5 python3.4 python3.3 dnl
>   python3.2 python3.1 python3.0 dnl
> + python2 dnl
>   python2.7 python2.6 python2.5 python2.4 python2.3 python2.2 python2.1 dnl
>   python2.0])

Please also move the plain "python" command down to the python2 block (right 
after "python2" would be the best place I think).  Any system on which "python" 
runs Python 3 is grossly misconfigured.

zw

bug#31728: [PATCH] dmalloc: mark macro as obsolete

[Zack Weinberg]

On Sat, Jan 22, 2022, at 4:44 PM, Karl Berry wrote:
> Sorry, I still disagree with "deprecating" AM_WITH_DMALLOC (or anything
> else). My wish would be to add some strong wording in the manual about
> how it doesn't do anything especially useful, new code shouldn't use it,
> etc., and let it go at that.

I understand where you are coming from but I want to make a counterargument.
Autotools in general have a problem where there are lots of macros that
solve problems that are no longer important, but people keep including them
in configure.ac anyway, because they don't realize that they are no longer
important.  Often the code doesn't actually work in the situation the macro
is testing for, because they haven't been diligent enough with the #ifdefs
or whatever, and nobody notices.

Putting strong wording in the manual is not enough to get people to stop
using these macros in new code, because people aren't reading the manual,
they are copying configure.ac fragments from other programs (which may or
may not have been written back in the day when the portability problems
addressed by the macros _were_ still important).

> I don't argue that AM_WITH_DMALLOC is useful.  But since it has been
> there forever, I argue that deprecating it is harmful, in that it causes
> gratuitous work for maintainers whose configure.ac's happen to have it.

Deprecation warnings are a blunt instrument, and I appreciate that they
make extra work for maintainers, but I will argue that that extra work
is _worthwhile_ work because it means their configure.ac's become
_better examples_ for the sorts of people who want to copy and tinker
existing code rather than reading documentation.

zw

Re: [PATCH 1/2] m4: warn when AM_SILENT_RULES default is used

[Zack Weinberg]

On Sun, Dec 12, 2021, at 1:09 AM, Mike Frysinger wrote:
> To help ease people into enabling silent rules by default, warn if
> a package doesn't make an explicit selection.

-1 for the same reason that changing the default is a bad idea.

If we are to make *any* change in this area I would advocate instead for a step 
in the opposite direction:  remove the DEFAULT argument from AM_SILENT_RULES.  
The default is hardcoded to "no".  In other words, individual packages are no 
longer allowed to make the user-hostile decision of switching the default to 
"yes".

In addition to the ergonomics benefit, taking away that knob would smooth the 
path to having the AM_SILENT_RULES *feature* be enabled by default: that is, 
all automake-using packages get the --enable-silent-rules switch and the 
makefile machinery to back it up, without having to mention AM_SILENT_RULES in 
configure.ac.

zw

Re: [RFC/PATCH] m4: enable silent build rules by default

[Zack Weinberg]

On Sun, Dec 12, 2021, at 1:10 AM, Mike Frysinger wrote:
> On 07 Dec 2021 21:58, Zack Weinberg wrote:
>> On Tue, Dec 7, 2021, at 9:49 PM, Mike Frysinger wrote:
>> > This has been available since automake 1.11 released over a decade
>> > ago.  Let's flip the default to enable silent builds by default.
>> 
>> Please don't *ever* make this change.  It is absolutely essential that the 
>> default build output be verbose, else anyone who's stuck trying to debug a 
>> build on a CI server that they have no direct access to is in a world of 
>> hurt.
>
> people can trivially add --disable-silent-rules to their configure 
> invocations.
> i really don't think this is as onerous as you make it out to be.

I can tell you've never had this experience, or you wouldn't say that.  And 
probably you wouldn't be proposing this patch in the first place :-P

There's some piece of software.  You don't know how its build system works, and 
the process of configuring and building it is hidden behind at least one layer 
of packaging automation.  It's got a bug in its build, but only on some 
architecture you do not have shell access to, only some CI system that you do 
not administer and whose only interface is "upload software, get back logs 
minutes to hours later."  It builds Just Fine on your computer.

The logs you got back from your first attempt to build it are useless because 
the equivalent of --enable-silent-rules is on by default.

You spend anywhere between five minutes and an hour figuring out which build 
system the software uses, what its equivalent of --disable-silent-rules is, and 
where to poke that into the packaging automation.  You throw another build at 
the CI.  Half an hour later you get back logs that are *still* useless because 
you misunderstood where the poke goes.

You tinker with it on your computer and get the poke into the right place and 
get back logs that are not useless.  You throw another build at the CI.  Half 
an hour later you get back logs that are *still* useless because the CI uses a 
slightly different build process (think pbuilder vs sbuild) and that means the 
poke is still not in the right place.

This cycle can go on for DAYS if you're sufficiently unlucky.

Yes, an awful lot of this is the fault of the packaging automation and the CI 
in question.  But Automake can't do shit about that.  But what Automake *can* 
do is avoid making the situation even worse, by ... sticking to what it already 
does: verbose builds by default.

-

I'd like to generalize the above point a little.  I saw someone (possibly you) 
advocating for --enable-silent-rules by default because it makes it less likely 
for actual warnings and errors to get lost amid a flood of chatter.  This is 
absolutely true.  I use --enable-silent-rules all the time on the software that 
I work on frequently, because in *that* situation the chatter *is* irrelevant 
-- I already have the build environment set up correctly, I'm not going to need 
to figure out why the C compiler can't find  or anything like 
that.  And even with compilation-mode doing the actual work of scanning for 
warnings, the terse log *is* easier on my poor tired brain.

--enable-silent-rules is good for day-to-day maintenance.  
--disable-silent-rules, on the other hand, is what people want when they have 
no prior familiarity with some software, and they're just trying to fix one 
bug, because in that situation they may *not* have things set up right and the 
build system hiding details of what it's actually doing is more likely to be 
aggravating than not.

I'm arguing that the out-of-the-box, default behavior of Automake-driven builds 
should optimize for the experience of people who are just trying to fix one 
bug, *not* for the experience of day-to-day maintainers.  That might sound 
backward but think about it.  Day-to-day maintainers have the time and the 
incentive to set up customizations.  They have plenty of other reasons to learn 
the ins and outs of the configuration process.  It's *easy* to get in the habit 
of typing `./configure --enable-silent-rules` instead of just `./configure` 
once you know that that does a thing you like.  The out-of-the-box experience 
does not *need* to be tuned for regular maintainers' convenience.  Rather, it 
should be tuned for the convenience of people who have never tried to build the 
software before and do *not* have the time or patience to mess with the 
defaults.

zw

Re: [PATCH] gitignore: update

[Zack Weinberg]

On Sun, Dec 12, 2021, at 1:07 AM, Mike Frysinger wrote:
> Ignore all *~ files as editors like to create them, as do some
> autotool steps.  This also matches what autoconf is doing.
...

> +*~
> +.#*

Suggest also adding \#*# (emacs autosave files) and .*.sw[op] (vim autosave 
files).

zw

Re: [RFC/PATCH] m4: enable silent build rules by default

[Zack Weinberg]

On Tue, Dec 7, 2021, at 9:49 PM, Mike Frysinger wrote:
> This has been available since automake 1.11 released over a decade
> ago.  Let's flip the default to enable silent builds by default.

Please don't *ever* make this change.  It is absolutely essential that the 
default build output be verbose, else anyone who's stuck trying to debug a 
build on a CI server that they have no direct access to is in a world of hurt.

zw

Re: `make dist` fails with current git

[Zack Weinberg]

On Wed, Oct 13, 2021, at 2:32 PM, Nick Bowler wrote:
> On 2021-10-13, Zack Weinberg  wrote:
>> On Wed, Oct 13, 2021, at 2:11 PM, Nick Bowler wrote:
>>> I think this happened because your CI system has done a shallow clone.
>>> So the changelog generation failed because the git log is incomplete.
>>
>> I did a --single-branch clone, but not a shallow one.  Shouldn't the trunk
>> be self-contained?
>
> I think --single-branch should be fine: it seems to work for me...
>
> But with e.g., a "--depth 1" clone I get the exact same error you
> reported (because the commit IDs mentioned in .git-log-fix are not
> available).

After a little more investigation, a --single-branch clone indeed works fine.  
I was confused by two unrelated bugs, which happen regardless of what you clone.

1. `make dist` immediately after `configure` fails due to a missing dependency 
somewhere:

$ make dist
make  dist-xz dist-gzip am__post_remove_distdir='@:'
make[1]: Entering directory '/home/zack/projects/gnu/B-automake'
make  distdir-am
make[2]: Entering directory '/home/zack/projects/gnu/B-automake'
make[3]: Entering directory '/home/zack/projects/gnu/B-automake'
make[3]: Leaving directory '/home/zack/projects/gnu/B-automake'
Updating ../automake/doc/version.texi
  GEN  ../automake/doc/amhello-1.0.tar.gz
../automake/doc/amhello-1.0.tar.gz: recipe failed.
See file '/home/zack/projects/gnu/automake/doc/amhello/amhello-output.tmp' for 
details
make[2]: *** [Makefile:3759: ../automake/doc/amhello-1.0.tar.gz] Error 1
make[2]: Leaving directory '/home/zack/projects/gnu/B-automake'
make[1]: *** [Makefile:3245: distdir] Error 2
make[1]: Leaving directory '/home/zack/projects/gnu/B-automake'
make: *** [Makefile:: dist] Error 2

Running `make` first is a sufficient workaround.

2. `make dist` from a build directory that is not a subdirectory of the source 
directory fails during ChangeLog generation, with a different error than the 
one I reported earlier:

make[1]: Entering directory '/home/zack/projects/gnu/B-automake'
make  distdir-am
make[2]: Entering directory '/home/zack/projects/gnu/B-automake'
  GEN  ChangeLog
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
gitlog-to-changelog: error closing pipe from git log --log-size 
'--pretty=format:%H:%ct  %an  <%ae>%n%n%s%n%n%b' '--since=2011-12-28 00:00:00'
make[2]: *** [../automake/maintainer/maint.mk:48: ChangeLog] Error 1
make[2]: Leaving directory '/home/zack/projects/gnu/B-automake'
make[1]: *** [Makefile:3245: distdir] Error 2
make[1]: Leaving directory '/home/zack/projects/gnu/B-automake'
make: *** [Makefile:: dist] Error 2

I will see if I can find time to patch these.

zw

Re: `make dist` fails with current git

[Zack Weinberg]

On Wed, Oct 13, 2021, at 2:11 PM, Nick Bowler wrote:
> I think this happened because your CI system has done a shallow clone.
> So the changelog generation failed because the git log is incomplete.

I did a --single-branch clone, but not a shallow one.  Shouldn't the trunk be 
self-contained?

zw

Re: `make dist` fails with current git

[Zack Weinberg]

On Wed, Oct 13, 2021, at 11:54 AM, Bob Friesenhahn wrote:
> On Wed, 13 Oct 2021, Zack Weinberg wrote:
>>
>> Looks like some kind of problem with automatic ChangeLog generation?
>
> To me this appears to be the result of skipping an important step in 
> what should be the process.  It seems that there should be a 
> requirement that 'make distcheck' succeed prior to a commit.  Then 
> this issue would not have happened.

Well, yes, but how do I _fix_ it? :-)

<.< I _may_ be investigating the possibility of setting up CI for automake so 
that problems like this are at least noticed in a timely fashion.  But if the 
tree is broken to begin with, it's troublesome...

zw

`make dist` fails with current git

[Zack Weinberg]

$ make V=1 dist
make  dist-xz dist-gzip am__post_remove_distdir='@:'
make[1]: Entering directory '/home/zack/projects/gnu/autoconf/ci/b-automake'
make  distdir-am
make[2]: Entering directory '/home/zack/projects/gnu/autoconf/ci/b-automake'
set -e; set -u; \
if test -d ../s-automake/.git; then \
  rm -f ChangeLog-t \
&& /usr/bin/perl ../s-automake/lib/gitlog-to-changelog \
   --amend=../s-automake/.git-log-fix --since='2011-12-28 00:00:00' 
--no-cluster --format '%s%n%n%b' >ChangeLog-t \
&& chmod a-w ChangeLog-t \
&& mv -f ChangeLog-t ChangeLog \
|| exit 1; \
elif test ! -f ../s-automake/ChangeLog; then \
  echo "Source tree is not a git checkout, and no pre-existent" \
   "ChangeLog file has been found there" >&2; \
  exit 1; \
fi
gitlog-to-changelog:../s-automake/.git-log-fix: unused entry: 
3b369e6bbe0fb6d7359398935706c87dd9375cb6
gitlog-to-changelog:../s-automake/.git-log-fix: unused entry: 
22729165f6bb902daeb8a4d8e7cb06982390f327
make[2]: *** [../s-automake/maintainer/maint.mk:48: ChangeLog] Error 1
make[2]: Leaving directory '/home/zack/projects/gnu/autoconf/ci/b-automake'
make[1]: *** [Makefile:3245: distdir] Error 2
make[1]: Leaving directory '/home/zack/projects/gnu/autoconf/ci/b-automake'
make: *** [Makefile:: dist] Error 2

Looks like some kind of problem with automatic ChangeLog generation?

The commits it's complaining about are

commit 22729165f6bb902daeb8a4d8e7cb06982390f327
Author: Peter Rosin 
Date:   Fri Feb 17 10:13:15 2012 +0100

fixup: .git-log-fix should not be executable

* .fix-git-log: Set mode 644.

commit 3b369e6bbe0fb6d7359398935706c87dd9375cb6
Author: Stefano Lattarini 
Date:   Thu Feb 16 22:29:32 2012 +0100

maint: use AC_PACKAGE_BUGREPORT to avoid duplication

* configure.ac: In the message reporting whether the user is about
to build an alpha or beta version, use the autoconf-provided
AC_PACKAGE_BUGREPORT macro instead of duplicating the bur reporting
address.

bug#50469: [bison-3.8] bug or side effect to flex & automake

[Zack Weinberg]

On Wed, Sep 8, 2021, at 1:31 AM, Akim Demaille wrote:
> One big problem with the Autotools as of today is that they promote
> the use of macros/build rules for Yacc, not for Bison.

The contract of AC_PROG_YACC is to find something that will generate
parsers from POSIX-compliant input; all three of bison, byacc, and
classic yacc are considered.

We would be happy to take contributions of AC_PROG_BISON (and
AC_PROG_FLEX), but as you know there isn’t anyone doing active
development of Autoconf right now, so it’s not going to happen without
someone stepping up.

zw

Re: [PATCH] dist: add new "pure-dist" automake option

[Zack Weinberg]

On Fri, Jul 2, 2021 at 11:09 AM Allison Karlitskaya
 wrote:
>
> Since v1.15.1-204-gac47c22e3, "make dist" has been depending on
> $(BUILT_SOURCES) to avoid problems when building some GNU packages which
> need to compile themselves as part of building their tarballs (for
> example, to generate manpage content from --help output).

I have no comment on your patch, but I want to mention that the
"compile self in order to generate manpages that go into the tarball"
process is troublesome for several other reasons; for the recent
release of autoconf itself, I wound up writing a script that
insinuates itself into where help2man was looking for the executables
and extracts the --help text directly from the *source*.

zw

Re: config.sub/config.guess using nonportable $(...) substitutions

[Zack Weinberg]

On Tue, Mar 9, 2021 at 5:00 PM Tim Rice  wrote:
> On Tue, 9 Mar 2021, Warren Young wrote:
> > On Mar 9, 2021, at 1:26 PM, Paul Eggert  wrote:
> > Solaris 10 dates from early 2005.  We gave it 16 years of direct support, 
> > and now it’s on a sort of “extended” support if you point Autoconf 
> > configure scripts at a reasonable shell.
>
> The thing is, it is not even necessary to point Autoconf configure
> scripts at a reasonable shell. The configure script will find a capable
> shell and run itself with that shell.
>
> UnixWare has a /bin/sh as crufty as Solaris 10. I just pulled the
> latest config.guess and config.sub into my OpenSSH tree and configure
> was just as happy as before. Like Solaris /bin/sh, UniWare's does
> not handle $(...) either.

N.B. I changed Autoconf's "find us a better shell" logic in 2.70 to
include checking for $(...) _specifically because of_ the change being
debated here.  Configure scripts themselves still use `...`
nigh-exclusively.

I don't want to either tear down or defend Ben's change, but it's my
personal opinion that use of $(...) is a lot more defensible in a
configure script than in config.{sub,guess}, because it _has_ that
"find a better shell" step at the beginning.  Also, configure would
get more out of being able to use $(...) than config.{sub,guess} do,
just because its code is typically more complicated, and configure
scripts tend to pull in third-party code from all over the place that
may or may not ever have been tested on anything old.  For these
reasons I probably wouldn't revert the Autoconf change even if the
config.{sub,guess} change were reverted.

zw

Re: RFC: Bump minimum Perl to 5.18.0 for next major release of both Autoconf and Automake

[Zack Weinberg]

On Fri, Jan 29, 2021 at 5:54 PM Karl Berry  wrote:

I don't know why, but I only received this message today.

> But, I think it would be wise to give users a way to override the
> requirement, of course with the caveat "don't blame us if it doesn't
> work", unless there are true requirements such that nothing at all would
> work without 5.18.0 -- which seems unlikely (and undesirable, IMHO).
> 2013 is not that long ago, in autotime.

This is a reasonable suggestion but Perl makes it difficult.  You
specify the interpreter version requirement for a Perl script with a
statement like

use 5.006;   # what we have now

at the top of the script.  The number has to be a constant, and it has
to appear at the top of every .pm file as well as the main script.  I
don't think it makes sense to use a config.status substitution
variable for this -- then we'd have to generate _all_ of the .pm files
through config.status!

What we could do is something like this instead:

   use 5.008;  # absolute minimum requirement
   use if $] >= 5.016, feature => ':5.16';  # enable a number of
desirable features from newer perls

+ documentation that we're only _testing_ with the newer perls.

I did some more research on perl's version history (notes at end) and
I think the right thresholds are 5.10 for absolute minimum and 5.16
for 'we aren't going to test with anything older than this'.  5.10 is
the oldest perl that shipped Digest::SHA, which I have a specific need
for in autom4te; it is also the oldest perl to support `state`
variables and the `//` operator, both of which could be quite useful.
The new features in 5.12, 5.14, and 5.16 mostly have to do with
Unicode, which we do not strictly _need_ but which, if we turn them
on, give us a better chance of Just Doing The Right Thing with
user-supplied Unicode text.  The top-of-each-file boilerplate would
look something like this:

use 5.010;
use strict;
use warnings FATAL => ’all';
use utf8;
use if $] >= 5.016, feature => ’:5.16';
no  if $] >= 5.022, warnings => 'experimental::re_strict';
use if $] >= 5.022, re   => 'strict';

_Possibly_ we would also want `use open qw(:std :utf8)`, I'm not sure.
(That means "treat all files as encoded in UTF-8 unless overridden
below.")

Perl 5.10 shipped in 2007, so that's another six years of compatibility window.

zw


# perl 5.8

 - released 2002-07
 - support for `open FH, "-|", LIST`
 - Definitely useful modules: `Storable`, `Time::HiRes`, `if`, `open`,
   `threads` (already in use when available)
 - Possibly useful modules: `Digest::MD5`, `MIME::Base64`

# perl 5.10

 - released 2007-12
 - first version supporting `use feature`, `state`, the `//`
   operator, named capture groups, and a bunch of other regex
   enhancements
 - Possibly useful modules: `Digest::SHA`, `Compress::Zlib`

# perl 5.12

 - released 2010-04
 - first version for which `use 5.xx` implies `use strict`

# perl 5.14

 - released 2011-05
 - support for `s///r` (returns result of substitution, input variable
   is not modified)
 - first complete implementation of `use feature 'unicode_strings'`
 - Possibly useful module: `HTTP::Tiny`

# perl 5.16

 - released 2012-05
 - a whole bunch of unicode-related bug fixes
 - bugs relevant to autotools were fixed in `FindBin` and `IPC::Open3`

# perl 5.18

 - released 2013-05
 - first version with `experimental` warnings category
 - oldest version typically available on cloud platforms due to
   important security fixes (which are not directly relevant for
   use in autotools)
 - oldest version in which HTTP::Tiny implements HTTPS correctly

Re: RFC: Bump minimum Perl to 5.18.0 for next major release of both Autoconf and Automake

[Zack Weinberg]

On Fri, Jan 29, 2021 at 5:54 PM Karl Berry  wrote:

I don't know why, but I only received this message today.

> raise the minimum version requirement for Perl to 5.18.0
>
> Sounds sensible to me, for the reasons you outlined.
>
> But, I think it would be wise to give users a way to override the
> requirement, of course with the caveat "don't blame us if it doesn't
> work", unless there are true requirements such that nothing at all would
> work without 5.18.0 -- which seems unlikely (and undesirable, IMHO).
> 2013 is not that long ago, in autotime.
>
> don't know what the Automake maintainers' current release schedule
>
> We (I, at least, and I'm pretty sure it's true for Jim too) have no schedule.
>
> looks like but I imagine this would be targeted for 1.17.0,
>
> I guess we would declare the version making this change as 1.17.0.
> I don't have any other plans that would involve a new minor version.
>
> that happens (ideally at about the same time as the Autoconf release
> making the same change).
>
> Sounds feasible, since it doesn't sound like it will be especially quick
> (I'm glad). --thanks, karl.
>

Re: Automake's file locking

[Zack Weinberg]

On Thu, Jan 28, 2021 at 6:51 PM Bruno Haible  wrote:
> Zack Weinberg wrote:
> > There is a potential way forward here.  The *only* place in all of
> > Autoconf and Automake where XFile::lock is used, is by autom4te, to
> > take an exclusive lock on the entire contents of autom4te.cache.
> > For this, open-file locks are overkill; we could instead use the
> > battle-tested technique used by Emacs: symlink sentinels.  (See
> > https://git.savannah.gnu.org/cgit/emacs.git/tree/src/filelock.c .)
>
> I can confirm that, while flock() is the most basic/elementary locking
> facility [1], its emulation in gnulib [2] does not really work on
> Solaris. The unit test regularly fails on Solaris.
>
> Therefore I like the idea of merely relying on the atomicity of
> file creation / file rename operations.
>
> These files should reside inside the autom4te.cache directory. I would
> not like to change all my scripts and Makefiles that do
>   rm -rf autom4te.cache

Agreed.  The approach I'm currently considering is: with the
implementation of the new locking protocol, autom4te will create a
subdirectory of autom4te.cache named after its own version number, and
work only in that directory (thus preventing different versions of
autom4te from tripping over each other).  Each request will be somehow
reduced to a strong hash and given a directory named after the hash
value.  The existence of this directory signals that an autom4te
process is working on a request, and the presence of 'request',
'output', and 'traces' files in that directory signals that the cache
for that request is valid.  If the directory for a request exists but
the output files don't, autom4te will busy-wait for up to some
definite timeout before stealing the lock and starting to work on that
request itself.

This would be substantially easier to implement with access to the
Storable, Digest::SHA, and Time::HiRes modules, and that's the
principal reason I suggested bumping our minimum Perl requirement to
5.18 in <https://lists.gnu.org/archive/html/autoconf/2021-01/msg00133.html>.

zw

Re: automake variable prefix 'check_'

[Zack Weinberg]

On Tue, Feb 2, 2021 at 1:24 PM DUDZIAK Krzysztof
 wrote:
> As one can't find string "distcheck" in GCS

By GCS do you mean the GNU Coding Standards?

> it looks like it wasn't GCS
> which constitutes support and usage of `distcheck' target.
> Maybe it is POSIX, or UNIX.

As far as I know, the distcheck target was invented by automake. It
probably should be added to the GNU Coding Standards, if it's not
there already, but I don't know if anyone is working on that document
anymore.

> I wonder if recipe to build `distcheck' target
> as compiled by automake gets same form as John Calcote
> describes it in chapter 3 his Practitioner's Guide (2nd ed.).

I have not read this book. Since it was published ten years ago,
anything it describes may well be out of date.  I don't think
distcheck has changed much in that time, but I could be wrong.

> For one Makefile generated by autotools in one of my last projects
> I see building `check' target is included in `distcheck' recipe.
> If yes, all products prefixed with `check_'  would be built also
> when `make distcheck' command is run.

Yes, this is intentional. I don't think I understand what your question is.

zw

Re: PLV, PSV in manual

[Zack Weinberg]

On Tue, Feb 2, 2021 at 1:19 PM DUDZIAK Krzysztof
 wrote:
> Isn't it that strange if for manual* following searches for a string result 
> in no matches?
> search for "PLV"
> search for "PSV"
> search for "Product List Variable"
> search for "Product Source Variable"

I've never heard these terms before and don't know what they mean. Can
you please explain what these terms mean and what you expected the
automake manual to have to say about them?

zw

RFC: Bump minimum Perl to 5.18.0 for next major release of both Autoconf and Automake

[Zack Weinberg]

I propose that, as of the next major feature release of both Autoconf
and Automake, we raise the minimum version requirement for Perl to
5.18.0.  (Currently it is 5.6.0.)  On the Autoconf side, the version
number and timeframe for the first release with this change are still
TBD but it would *not* come from the post-2.70 release branch.  I
don't know what the Automake maintainers' current release schedule
looks like but I imagine this would be targeted for 1.17.0, whenever
that happens (ideally at about the same time as the Autoconf release
making the same change).

Increasing the minimum version requirement would have these benefits:

 - Access to useful library modules, notably Digest::SHA, List::Util,
Storable, and Time::HiRes.
 - Comprehensive interpreter support for Unicode (specifically, 'use
feature "unicode_strings"').  Neither autoconf nor automake makes much
use of non-ASCII characters themselves, but having Unicode supported
in the interpreter would make it easier for us to be sure that we
don't mess up non-ASCII text in our inputs.
 - Any number of bug fixes; I have personally tripped over problems
with File::Spec and File::Temp in 5.6.0.
 - Any number of performance enhancements.

Perl 5.18.0 was released in May 2013.  (Perl 5.6.0 was released in
March 2000.)  I selected it as the minimum because the Perl
maintainers recommend all users of older interpreters upgrade to at
least this version, due to security fixes (see
https://perldoc.perl.org/perlsec#Algorithmic-Complexity-Attacks --
inputs to Autoconf and Automake are trusted, but this rationale means
it's hard to find a CI platform that offers anything older than 5.18,
and we ought to be testing our minimum version requirement.)  I'm open
to arguments for setting it either lower or higher.

zw

Re: Automake's file locking (was Re: Autoconf/Automake is not using version from AC_INIT)

[Zack Weinberg]

On Thu, Jan 28, 2021 at 2:16 PM Bob Friesenhahn
 wrote:
> On Thu, 28 Jan 2021, Zack Weinberg wrote:
> >
> > The main reason I can think of, not to do this, is that it would make
> > the locking strategy incompatible with that used by older autom4te;
> > this could come up, for instance, if you’ve got your source directory
> > on NFS and you’re building on two different clients in two different
> > build directories.  On the other hand, this kind of version skew is
> > going to cause problems anyway when they fight over who gets to write
> > generated scripts to the source directory, so maybe it would be ok to
> > declare “don’t do that” and move on.  What do others think?
>
> This is exactly what I do.  I keep the source files on a file server
> so that I can build on several different types of clients.  This used
> to even include Microsoft Windows clients using CIFS.

Do you use different versions of autoconf and/or automake on the
different clients?

> The lock appears to be taken speculatively since it is taken before
> Autotools checks that there is something to do.
...
> The most common case is that there is nothing for Autotools to do
> since the user is most often doing a 'make' for some other purpose.

It looks to me like the lock is taken at exactly the point where
autom4te decides that it *does* have something to do. It might be
possible to change it to take a *read* lock first and only upgrade to
a write lock if new files are to be added to the cache, but Make
shouldn't be running the autotools at all if they have nothing to do
(which I suppose takes us over to the *other* thread about your
problems with automake and configure's dependencies :-)

zw

Re: Automake's file locking (was Re: Autoconf/Automake is not using version from AC_INIT)

[Zack Weinberg]

On Mon, Jan 25, 2021 at 11:18 AM Bob Friesenhahn
 wrote:
> On Mon, 25 Jan 2021, Zack Weinberg wrote:
> > Automake "just" calls Perl's 'flock' built-in (see 'sub lock' in
> > Automake/XFile.pm) (this code is copied into Autoconf under the
> > Autom4te:: namespace).  It would be relatively straightforward to
> > teach it to try 'fcntl(F_SETLKW, ...)' if that fails.

I was wrong about this.  It is not practical to make a Perl program
directly call ‘fcntl(F_SETLKW)’ without use of CPAN modules.  This is
because, at the C level, ‘fcntl(F_SETLKW)’ takes a ‘struct flock’
argument, and we have no way of knowing what the layout of that
structure is.  (See https://metacpan.org/pod/File%3a%3aFcntlLock for
gory details.  We can’t adopt the implementation of that module,
because it relies on running a C compiler, and the locking is needed
at a point when we do not yet know whether a C compiler is available.)

> It may be that moving forward to 'fcntl(F_SETLKW, ...)' by default and
> then falling back to legacy 'flock' would be best.  Or perhaps
> discarding use of legacy 'flock' entirely.
>
> Most likely the decision as to what to do was based on what was the
> oldest primitive supported at the time.

I need to reemphasize that the decision here was made by the Perl
developers, not the Automake or Autoconf developers, and it was made a
very long time ago—judging by e.g. references to h2ph in ‘perldoc
Fcntl’, probably circa perl 5.000!  I am also going to guess that the
motivation for preferring flock was related to the motivation for this
grumpy aside in OpenBSD’s ‘fcntl(2)’ manpage:

| This interface follows the completely stupid semantics of System V
| and IEEE Std 1003.1-1988 (“POSIX.1”) that require that *all* locks
| associated with a file for a given process are removed when *any* file
| descriptor for that file is closed by that process
| ...
| The flock(2) interface has much more rational last close semantics

(Emphasis in original.)  As I recall, at the time, *neither* flock nor
fcntl locks were honored *at all* over NFS, so that wouldn’t have been
a consideration.



There is a potential way forward here.  The *only* place in all of
Autoconf and Automake where XFile::lock is used, is by autom4te, to
take an exclusive lock on the entire contents of autom4te.cache.
For this, open-file locks are overkill; we could instead use the
battle-tested technique used by Emacs: symlink sentinels.  (See
https://git.savannah.gnu.org/cgit/emacs.git/tree/src/filelock.c .)

The main reason I can think of, not to do this, is that it would make
the locking strategy incompatible with that used by older autom4te;
this could come up, for instance, if you’ve got your source directory
on NFS and you’re building on two different clients in two different
build directories.  On the other hand, this kind of version skew is
going to cause problems anyway when they fight over who gets to write
generated scripts to the source directory, so maybe it would be ok to
declare “don’t do that” and move on.  What do others think?

zw

Automake's file locking (was Re: Autoconf/Automake is not using version from AC_INIT)

[Zack Weinberg]

On Mon, Jan 25, 2021 at 9:52 AM Bob Friesenhahn
 wrote:
> At the moment it is a big deal for me because the locking prototol
> that Autoconf/Automake is using does not work with NFS mounts for
> Illumos-derived systems when the client is also an Illumos-derived
> system, because Illumos failed to support the legacy locking protocol
> used when the system locking daemon was re-implemented from scratch.
...
> It is likely that a small patch to Automake Perl-based locking code
> could solve this issue by using the same fall-back to using POSIX
> locking rather than legacy locking the same way that GNU/Linux does.
> It may also be that using POSIX locking in the first place is the
> solution.

Automake "just" calls Perl's 'flock' built-in (see 'sub lock' in
Automake/XFile.pm) (this code is copied into Autoconf under the
Autom4te:: namespace).  It would be relatively straightforward to
teach it to try 'fcntl(F_SETLKW, ...)' if that fails.  Do you know
whether that would be sufficient?  If not, we may be up a creek, since
depending on CPAN modules is a non-starter.

zw

Re: Future plans for Autotools

[Zack Weinberg]

On Wed, Jan 20, 2021 at 5:15 PM Zack Weinberg  wrote:
> Now we've all had a while to recover from the long-awaited Autoconf
> 2.70 release, I'd like to start a conversation about where the
> Autotools in general might be going in the future.

> Now we've all had a while to recover from the long-awaited Autoconf
> 2.70 release, I'd like to start a conversation about where the
> Autotools in general might be going in the future.

Thanks for all the great thoughtful responses that came in overnight!
I see many people expressing similar sentiments, so rather than
respond to each of you individually I'm going to summarize and reply
to each strand in this one message.

> continuous integration for Autotools

This is one of my highest priorities for what volunteer time I have in
the next few months.  The biggest difficulty with CI, as I see it, is
that Autotools needs to be tested on a very wide variety of operating
systems, and ideally *versions* of operating systems, and unusual
compilers, and so on, in order to confirm that it's still fulfilling
its contract.  Testing just on the "big three" operating systems
(Linux, Windows, MacOS) typically supported by commercial CI providers
... would be better than nothing, but would not have helped all that
much with the 2.70 release, if I'm honest.  GNU's Hydra instance
(https://www.gnu.org/software/devel.en.html#Hydra -- I didn't
previously know about this, thank you Dan for pointing it out) is a
strong contender just because it has FreeBSD and Solaris as well.

> implementation languages - can't we do better than shell+perl+m4?

I deeply sympathize with frustration with M4 and portable shell as
implementation languages.  Perl isn't so bad IMO, but being restricted
to perl 5.6.0 with no CPAN modules is also a headache.

Having said that, switching to *anything else* would be a gigantic
task -- multiple full-time person-years of effort just for the core --
and would mean either porting or losing all of the third-party macro
libraries.  I don't think it makes sense to explore this possibility
any further unless both a lot of funding and a lot of developers
materialize.

A lesser possibility we could think about, would be dusting off GNU M4
2.0 and thinking about what we could change about the M4 language to
make it friendlier for Autoconf's purposes.  Again I would want
funding and developers lined up, but we'd not need quite so much of
either.

And we probably could bump up the minimum version requirement for Perl
quite some ways, since it's only needed at autoreconf time.  I'm not
sure how much Autoconf would gain from this, though, since it's only
incrementally changed since 5.6 and Autoconf doesn't do *that* much
with it. Automake uses it much more heavily, though, perhaps people
from that project would like to comment?

[Nate Bargmann]
> One strength of the Autotools that I think stands above the rest is
> the fact that a user of a distributed package does not need to
> install any of the Autotools packages.

I definitely agree with this, and I'm putting it right next to the
"implementation languages" note because this is a compelling reason to
stick to shell as the language for generated configure scripts.  The
only alternative that would come anywhere near as close in terms of
"just unpack and run" portability is Perl.  Which is a tempting
possibility -- it would wipe out dozens of internal headaches due to
shell's limitations and portability quirks -- but again, multiple
full-time person-years of effort, losing the third-party macro
libraries, and we *would* be making life harder for people starting
from scratch on OSes that don't bundle Perl.

> Autotools are too difficult to work on

Yes indeed.  On the assumption that we are stuck with shell, M4, and
Perl, however, what can we do about it?  I have a few ideas, maybe you
have more:

- A linter for configure.ac and third-party macro libraries.
  This wouldn't be any fun to write, because it would have to parse
  both M4 and shell, *accurately*, but it would be valuable to every
  user of autoconf.

- The parser for the linter could also be used as the basis for a
  reimplementation of autoupdate that *doesn't* rely on running
  configure.ac through M4 with most of the macros disabled.  That
  implementation choice is responsible for most of the bugs in
  autoupdate, and for most of its limitations (e.g. not being able to
  fix quotation errors).

- Audit existing configure.ac's looking for pain points: where are
  people having trouble doing the checks they need, where are people
  not realizing that they don't need to work so hard, where are people
  not realizing that they don't need to check for that at all...

- Audit the core for workarounds that are no longer necessary now we
  can rely on shell functions (I'm looking at you, AC_LANG_*).

- Better documentation and better debugging tools.  Speaking of which,
  here's an interesting bit fro

Future plans for Autotools

[Zack Weinberg]

Now we've all had a while to recover from the long-awaited Autoconf
2.70 release, I'd like to start a conversation about where the
Autotools in general might be going in the future.  Clearly any future
development depends on finding people who will do the work, but before
we worry about that I think we might want to figure out what work we
_want_ to do.

As a starting point, I wrote up a "strengths, weaknesses,
opportunities, and threats" analysis for Autotools -- this is a
standard project management technique, if you're not familiar with it,
there's a nice writeup in the draft of the book my friend and
colleague Sumana Harihareswara is writing [
https://changeset.nyc/resources/getting-unstuck-sampler-offer.html ].
I'm going to paste the full text of this analysis below, please reply
inline.  You can also read it on my blog at
https://www.owlfolio.org/development/autoconf-swot/ .

zw


I’ve been a contributor to GNU projects for many years, notably both
GCC and GNU libc, and recently I led the effort to make the first
release of Autoconf since 2012 (release announcement for Autoconf
2.70). For background and context, see the LWN article my colleague
Sumana Harihareswara of Changeset Consulting wrote.

Autoconf not having made a release in eight years is a symptom of a
deeper problem. Many GNU projects, including all of the other
components of the Autotools (Automake, Libtool, Gnulib, etc.) and the
software they depend upon (GNU M4, GNU Make, etc.) have seen a steady
decline in both contributor enthusiasm and user base over the past
decade. I include myself in the group of declining enthusiasts; I
would not have done the work leading up to the Autoconf 2.70 release
if I had not been paid to do it. (I would like to say thank you to the
project funders: Bloomberg, Keith Bostic, and the GNU Toolchain Fund
of the FSF.)

The Autotools are in particularly bad shape due to the decline in
contributor enthusiasm. Preparation for the Autoconf 2.70 release took
almost twice as long as anticipated; I made five beta releases between
July and December 2020, and merged 157 patches, most of them bugfixes.
On more than one occasion I was asked why I was going to the
trouble—isn’t Autoconf (and the rest of the tools by implication)
thoroughly obsolete? Why doesn’t everyone switch to something newer,
like CMake or Meson? (See the comments on Sumana’s LWN article for
examples.)

I personally don’t think that the Autotools are obsolete, or even all
that much more difficult to work with than some of the alternatives,
but it is a fair question. Should development of the Autotools
continue? If they are to continue, we need to find people who have the
time and the inclination (and perhaps also the funding) to maintain
them steadily, rather than in six-month release sprints every eight
years. We also need a proper roadmap for where further development
should take these projects. As a starting point for the conversation
about whether the projects should continue, and what the roadmap
should be, I was inspired by Sumana’s book in progress on open source
project management (sample chapters are available from her website) to
write up a strengths, weaknesses, opportunities, and threats analysis
of Autotools.

This inventory can help us figure out how to build on new
opportunities, using the Autotools’ substantial strengths, and where
to invest to guard against threats and shore up current weaknesses.

Followup discussion should go to the Autoconf mailing list.

Strengths

In summary: as the category leader for decades, the Autotools benefit
from their architectural approach, interoperability, edge case
coverage, standards adherence, user trust, and existing install base.

Autoconf’s feature-based approach to compiled-code portability scales
better than lists of system quirks.
The Autotools carry 30+ years’ worth of embedded knowledge about
portability traps for C programs and shell-based build scripting on
Unix (and to a lesser extent Windows and others), including variants
of Unix that no other comparable configuration tool supports.
Autoconf and Automake support cross-compilation better than competing
build systems.
Autoconf and Automake support software written in multiple languages
better than some competing build systems (but see below).
Autoconf is very extensible, and there are lots of third-party macros available.
Tarball releases produced by Autotools have fewer build dependencies
than tarball releases produced by competing tools.
Tarball releases produced by Autotools have a predictable,
standardized (literally; it’s a key aspect of the GNU Coding
Standards) interface for setting build-time options, building them,
testing them, and installing them.
Automake tries very hard to generate Makefiles that will work with any
Make implementation, not just GNU make, and not even just (GNU or BSD)
make.
The Autotools have excellent reference-level documentation (better
than CMake and Meson’s).
As they are GNU projects, users can 

Re: Getting srcdir in script executed using m4_esyscmd in AC_INIT

[Zack Weinberg]

On Tue, Jan 5, 2021 at 8:43 AM Bob Friesenhahn
 wrote:
> On Mon, 4 Jan 2021, Zack Weinberg wrote:
> >>
> >> Something which surprises me is that the distribution tarballs became
> >> noticeably larger.
> >
> > This is expected. The bulk of the increase is probably due to the test
> > programs used to probe for C and/or C++ 2011 support in the compilers.
> > There were also some fixes for compatibility problems that involved
> > emitting more code throughout the script. I can explain the
> > rationale for any piece of the diff between the old and new configure
> > scripts that you're curious about.
>
> To be clear, the size difference was intended to be due to the list of
> parameters passed to AC_INIT and AM_INIT_AUTOMAKE and re-autotooling.
> In fact this is still using Autoconf 2.69.

Oh, I misunderstood you.

> Something I found which surprised me is that Automake has added a GNU
> COPYING file to my non-GNU non-GPL package.  Once the file appeared in
> the source directory, it seems that it continued to be used and
> included in spite of the Automake options provided.
>
> While I see more detail sprinkled around the generated files, it seems
> that this COPYING file may be the biggest issue.

Hmm.  If you remove COPYING, make sure that you have `foreign` in
either `AUTOMAKE_OPTIONS` or the options list passed to
`AM_INIT_AUTOMAKE`, and then re-run automake --add-missing, does
COPYING come back?  My understanding is that automake --add-missing is
supposed to install a copy of the GPL *only if* it's in `gnu` mode and
can't find any other statement of licensing.

zw

Re: Build and test failures with Autoconf 2.70

[Zack Weinberg]

Karl Berry wrote:
> Zack Weinberg wrote:
>> They all appear to be cases of autoconf and/or aclocal
>> getting run when the test suite does not expect them to be run.  I am
>>  stumped as to why
>
> In short: because the 2.70 autom4te decided not to update configure.
>
> I don't know why not, but I have a guess: maybe the new autom4te
> ignores everything after AC_OUTPUT. Is that correct? Because that is
> where some of these tests are adding a new AC_ macro.
[...]
> in the successful run with autoconf 2.69, it was the
> $AUTOCONF run that updated configure (as one would expect).
>
> In the failing run with autoconf 2.70, the $AUTOCONF run ends with
> the line: autom4te: 'configure' is unchanged
>
> The failure follows.

Thanks, that was the clue I needed.  The newer autoconf wasn't
ignoring code after AC_OUTPUT, but what it *was* doing was leaving the
timestamp on configure untouched when a change in configure.ac did not
change the generated code.  (As if it were using move-if-change to
update configure.)  AC_LIBSOURCES doesn't change the generated code,
because it does all its work via traces, so those tests all failed.

I've fixed this in Autoconf trunk now
(revision 07130c3e48d12ec155ac5d7630dc7477b6989940)
and will shortly merge it to the 2.70 release branch.
With this patch applied, I see no failures in the Automake
testsuite on a GNU/Linux system.

zw

p.s. Sorry for the delay, I forgot I wasn't subscribed to any of the
Automake mailing lists.  I am now.

Re: [PATCH] Add --jobserver/jobserver option for GCC -flto=jobserver

[Zack Weinberg]

On Wed, Oct 28, 2020 at 4:00 PM H.J. Lu  wrote:
> On Wed, Oct 28, 2020 at 11:40 AM Nick Bowler  wrote:
> > On 2020-10-28, Zack Weinberg  wrote:
> > > On Wed, Oct 28, 2020 at 2:16 PM Nick Bowler  wrote:
> > >> On 2020-10-28, H.J. Lu  wrote:
> > >> > GCC introduced some time ago option -flto=jobserver in order to use the
> > >> > GNU Make jobserver when parallelising LTO builds.  It is actually a
> > >> > similar "recursive make".  When doing a recursive make, you need to
> > >> > place a '+' character at the beginning of the recipe line in order to
> > >> > let GNU Make pass the jobserver file descriptors to the child
> > >> > processes.
> > >> >
> > >> > Add the --jobserver option to add a '+' character to the recipe line in
> > >> > program.am and ltlibrary.am.
> > > ...
> > >> Surely this needs to be a configure-time option, perhaps combined
> > >> with some sort of configure test, since otherwise users won't get
> > >> this choice, right?  As an automake option the choice made by whomever
> > >> prepares the distribution will get baked into distributed Makefile.in
> > >> files...
> > >
> > > I was going to say something very similar: there shouldn't be an
> > > option at all.  The decision of whether or not to put + at the
> > > beginning of the recipe line should be made _when make is run_, based
> > > on whether -flto=jobserver actually appears in $(LDFLAGS) or wherever.
> >
> > Since this check cannot be done inside a rule, testing LDFLAGS at make
> > time is probably impractical to do portably.

LTO_PLUS = $(if $(findstring -flto=jobserver,$(LDFLAGS)),+)

I think it's okay if this only works with GNU Make.

> "make -n" will execute the recipe with the "+" prefix.   But --jobserver
> is off by default.  People who use --jobserver prefer a working GCC
> -flto=jobserver over a broken "make -n".

What Nick is saying - and I agree with him - is that it's not okay for
the package maintainer to make that decision.

zw

Re: [PATCH] Add --jobserver/jobserver option for GCC -flto=jobserver

[Zack Weinberg]

On Wed, Oct 28, 2020 at 2:16 PM Nick Bowler  wrote:
> On 2020-10-28, H.J. Lu  wrote:
> > GCC introduced some time ago option -flto=jobserver in order to use the
> > GNU Make jobserver when parallelising LTO builds.  It is actually a
> > similar "recursive make".  When doing a recursive make, you need to
> > place a '+' character at the beginning of the recipe line in order to
> > let GNU Make pass the jobserver file descriptors to the child processes.
> >
> > Add the --jobserver option to add a '+' character to the recipe line in
> > program.am and ltlibrary.am.
...
> Surely this needs to be a configure-time option, perhaps combined
> with some sort of configure test, since otherwise users won't get
> this choice, right?  As an automake option the choice made by whomever
> prepares the distribution will get baked into distributed Makefile.in
> files...

I was going to say something very similar: there shouldn't be an
option at all.  The decision of whether or not to put + at the
beginning of the recipe line should be made _when make is run_, based
on whether -flto=jobserver actually appears in $(LDFLAGS) or wherever.

> This will break "make -n", won't it?  Some sort of test of $(MAKEFLAGS)
> is probably required in this command.

I concur with this as well.

zw

Re: AC_DIAGNOSE not obsolete, please

[Zack Weinberg]

Both sets of patches have been pushed.  The changes to the Autoconf
manual are slightly different in the version I committed, I found some
small errors on proofreading the patch.

According to 
https://codesearch.debian.net/search?q=-pkg%3Aautoconf+%5CbAC_DIAGNOSE%5Cb=0
this would have been a problem for macros distributed by gettext and
libtool as well as automake, and maybe more (I didn't read through all
853 pages :-) so I'm glad you caught it.

zw

Re: AC_DIAGNOSE not obsolete, please

[Zack Weinberg]

On Mon, Oct 5, 2020 at 5:40 PM Karl Berry  wrote:
>
> Zack and all - Thien-Thi (cc'd) just reported two failures in automake
> make check with autoconf-2.69c (bugs.gnu.org/43819). The critical
> message appears to be:
>
> ./lib/autoconf/general.m4:2328: AC_DIAGNOSE is expanded from...
> /home/ttn/local/.b/automake-1.16.2/m4/init.m4:29: AM_INIT_AUTOMAKE is 
> expanded from...
> configure.ac:2: the top level
> configure.ac:2: warning: The macro `AC_DIAGNOSE' is obsolete.
>
> Since AM_INIT_AUTOMAKE calls AC_DIAGNOSE, can you please not make
> it obsolete? Otherwise upgrading will be a nightmare for everyone.

Seems reasonable.  What do you think of the patches below?  The first
set, applicable to autoconf, makes it so autoupdate will still replace
AC_DIAGNOSE with m4_warn, but autoconf *won't* complain about
AC_DIAGNOSE being obsolete.  This is handled with a new argument to
AU_DEFUN and AU_ALIAS, which can be either empty, for the historical
behavior of warning and updating, or the word "silent" for just update
with no warnings from autoconf.  Autoconf 2.69 and previous will
ignore the new argument, so this is safe to use in third-party macros
if desired.

The second set, applicable to automake, is additional tweaks I needed
to make to the automake testsuite to get it to pass 100%.  It seems
that both autoconf and automake have become pickier about incomplete
configure scripts.

(m4_warn was already available in Autoconf 2.50, by the way.)

zw

--- autoconf patches ---
diff --git a/doc/autoconf.texi b/doc/autoconf.texi
index dea85e4a..07ab9763 100644
--- a/doc/autoconf.texi
+++ b/doc/autoconf.texi
@@ -14888,29 +14888,52 @@ Obsoleting Macros
 Autoconf provides a simple means to obsolete a macro.

 @anchor{AU_DEFUN}
-@defmac AU_DEFUN (@var{old-macro}, @var{implementation}, @ovar{message})
+@defmac AU_DEFUN (@var{old-macro}, @var{implementation},
@ovar{message}, @ovar{silent})
 @auindex{DEFUN}
-Define @var{old-macro} as @var{implementation}.  The only difference
-with @code{AC_DEFUN} is that the user is warned that
-@var{old-macro} is now obsolete.
+Define @var{old-macro} as @var{implementation}, just like
+@code{AC_DEFUN}, but also declare @var{old-macro} to be obsolete.
+When @command{autoupdate} is run, occurrences of @var{old-macro} will
+be replaced by the text of @var{implementation} in the updated
+@file{configure.ac} file.

-If she then uses @command{autoupdate}, the call to @var{old-macro} is
-replaced by the modern @var{implementation}.  @var{message} should
-include information on what to do after running @command{autoupdate};
-@command{autoupdate} prints it as a warning, and includes it
-in the updated @file{configure.ac} file.
+If a simple textual replacement is not enough to finish the job of
+updating a @file{configure.ac} to modern style, provide instructions for
+whatever additional manual work is required as @var{message}.  These
+instructions will be printed by @command{autoupdate}, and embedded in the
+updated @file{configure.ac} file, next to the text of @var{implementation}.

-The details of this macro are hairy: if @command{autoconf} encounters an
-@code{AU_DEFUN}ed macro, all macros inside its second argument are expanded
-as usual.  However, when @command{autoupdate} is run, only M4 and M4sugar
-macros are expanded here, while all other macros are disabled and
-appear literally in the updated @file{configure.ac}.
+Normally, @command{autoconf} will also issue a warning (in the
+``obsolete'' category) when it expands @code{old-macro}.  This warning
+does not include @var{message}; it only advises the maintainer to run
+@command{autoupdate}.  If it is inappropriate to issue this warning, set
+the @var{silent} argument to the word @code{silent}.  One might want to
+use a silent @code{AU_DEFUN} when @code{old-macro} is used in a
+widely-distributed third-party macro.  If that macro's maintainers are
+aware of the need to update their code, it's unnecessary to nag all
+of the transitive users of @code{old-macro} as well.  This capability
+was added to @code{AU_DEFUN} in Autoconf 2.70; older versions of
+autoconf will ignore the @var{silent} argument and issue the warning
+anyway.
+
+@strong{Caution:} If @var{implementation} contains M4 or M4sugar macros,
+they will be evaluated when @command{autoupdate} is run, not emitted
+verbatim like the rest of @var{implementation}.  If this is wrong, you
+will need to work around it---see for instance the definition
+of @code{AC_FOREACH} in @file{general.m4}.
 @end defmac

-@defmac AU_ALIAS (@var{old-name}, @var{new-name})
+@defmac AU_ALIAS (@var{old-name}, @var{new-name}, @ovar{silent})
 @auindex{ALIAS}
-Used if the @var{old-name} is to be replaced by a call to @var{new-macro}
-with the same parameters.  This happens for example if the macro was renamed.
+A shorthand version of @code{AU_DEFUN}, to be used when a macro has
+simply been renamed.  @command{autoupdate} will replace @var{old-name}
+with a call to @var{new-macro}, with the 

[RFC PATCH 1/3] New utility function Automake::ChannelDefs::merge_WARNINGS.

[Zack Weinberg]

This function merges a list of warnings categories into the environment
variable WARNINGS, returning a new value to set it to.  The intended use
is in code of the form

{
  local $ENV{WARNINGS} = merge_WARNINGS ("this", "that");

  # run a command here with WARNINGS=this,that,etc
}

This is not actually used in automake, but will be in autoconf.

* lib/Automake/ChannelDefs.pm (merge_WARNINGS): New function.
---
 lib/Automake/ChannelDefs.pm | 67 +++--
 1 file changed, 65 insertions(+), 2 deletions(-)

diff --git a/lib/Automake/ChannelDefs.pm b/lib/Automake/ChannelDefs.pm
index b98191b4c..e9aaf2f80 100644
--- a/lib/Automake/ChannelDefs.pm
+++ b/lib/Automake/ChannelDefs.pm
@@ -63,7 +63,8 @@ BEGIN
 
 our @ISA = qw (Exporter);
 our @EXPORT = qw (_error   
- _warning _WARNINGS _warnings);
+ _warning _WARNINGS _warnings
+ _WARNINGS);
 
 =head2 CHANNELS
 
@@ -107,7 +108,7 @@ Warnings related to GNU Coding Standards.
 
 =item C
 
-Warnings about obsolete features (silent by default).
+Warnings about obsolete features.
 
 =item C
 
@@ -409,6 +410,68 @@ sub parse_warnings (@)
   switch_warning ($_werror ? 'error' : 'no-error');
 }
 
+=item C
+
+Merge the warnings categories in the environment variable C
+with the warnings categories in C<@CATEGORIES>, and return a new
+value for C.  Values in C<@CATEGORIES> take precedence.
+Use like this:
+
+local $ENV{WARNINGS} = merge_WARNINGS @additional_warnings;
+
+=cut
+
+sub merge_WARNINGS (@)
+{
+  my $werror = '';
+  my $all_or_none = '';
+  my %warnings;
+
+  my @categories = split /,/, $ENV{WARNINGS} || '';
+  push @categories, @_;
+
+  foreach (@categories)
+{
+  if (/^(?:no-)?error$/)
+{
+  $werror = $_;
+}
+  elsif (/^(?:all|none)$/)
+{
+  $all_or_none = $_;
+}
+  else
+{
+  # The character class in the second match group is ASCII \S minus
+  # comma.  We are generous with this because category values may come
+  # from WARNINGS and we don't want to assume what other programs'
+  # syntaxes for warnings categories are.
+  /^(no-|)([\w\[\]\/\\!"#$%&'()*+-.:;<=>?@^`{|}~]+)$/
+or die "Invalid warnings category: $_";
+  $warnings{$2} = $1;
+}
+}
+
+  my @final_warnings;
+  if ($all_or_none)
+{
+  push @final_warnings, $all_or_none;
+}
+  else
+{
+  foreach (sort keys %warnings)
+{
+  push @final_warnings, $warnings{$_} . $_;
+}
+}
+  if ($werror)
+{
+  push @final_warnings, $werror;
+}
+
+  return join (',', @final_warnings);
+}
+
 =item C
 
 Configure channels for strictness C<$STRICTNESS_NAME>.
-- 
2.28.0

[RFC PATCH 3/3] Update documentation of warnings options and strictness levels.

[Zack Weinberg]

The warning categories ‘cross’ and ‘portability-recursive’ were not mentioned
in the manual.

Also clarify the relationship between warnings categories and strictness
levels, and streamline the description of strictness levels by merging the
“Gnits” section into the “Strictness” section.

* doc/automake.texi (Gnits, Strictness): Combine these sections.
  Minor revisions to explanation of strictness levels.
  (automake Invocation): Add documentation of all the warnings
  categories that have been added since the last time this section
  was updated.  Minor clarifications.
---
 doc/automake.texi | 250 --
 1 file changed, 129 insertions(+), 121 deletions(-)

diff --git a/doc/automake.texi b/doc/automake.texi
index 3a6c1815d..738eb84cb 100644
--- a/doc/automake.texi
+++ b/doc/automake.texi
@@ -118,7 +118,6 @@ Top
 * Include:: Including extra files in an Automake template
 * Conditionals::Conditionals
 * Silencing Make::  Obtain less verbose output from @command{make}
-* Gnits::   The effect of @option{--gnu} and 
@option{--gnits}
 * Not Enough::  When Automake is not Enough
 * Distributing::Distributing the Makefile.in
 * API Versioning::  About compatibility between Automake versions
@@ -1913,7 +1912,9 @@ General Operation
 
 @node Strictness
 @section Strictness
-
+@c "Gnits" used to be a separate section.
+@c This @anchor allows old links to still work.
+@anchor{Gnits}
 @cindex Non-GNU packages
 
 While Automake is intended to be used by maintainers of GNU packages, it
@@ -1921,44 +1922,107 @@ Strictness
 not want to use all the GNU conventions.
 
 @cindex Strictness, defined
-@cindex Strictness, @option{foreign}
-@cindex @option{foreign} strictness
-@cindex Strictness, @option{gnu}
-@cindex @option{gnu} strictness
-@cindex Strictness, @option{gnits}
-@cindex @option{gnits} strictness
+To this end, Automake supports three levels of @dfn{strictness}---how
+stringently Automake should enforce conformance with GNU conventions.
+Each strictness level can be selected using an option of the same name;
+see @ref{Options}.
 
-To this end, Automake supports three levels of @dfn{strictness}---the
-strictness indicating how stringently Automake should check standards
-conformance.
-
-The valid strictness levels are:
+The strictness levels are:
 
 @table @option
+@item gnu
+@cindex Strictness, @option{gnu}
+@cindex @option{gnu} strictness
+@opindex --gnu
+This is the default level of strictness.  Automake will check for
+basic compliance with the GNU standards for software packaging.
+@xref{Top,,, standards, The GNU Coding Standards} for full details
+of these standards.  Currently the following checks are made:
+
+@itemize @bullet
+@item
+The files @file{INSTALL}, @file{NEWS}, @file{README}, @file{AUTHORS},
+and @file{ChangeLog}, plus one of @file{COPYING.LIB}, @file{COPYING.LESSER}
+or @file{COPYING}, are required at the topmost directory of the package.
+
+If the @option{--add-missing} option is given, @command{automake} will
+add a generic version of the @file{INSTALL} file as well as the
+@file{COPYING} file containing the text of the current version of the
+GNU General Public License existing at the time of this Automake release
+(version 3 as this is written,
+@uref{https://www.gnu.org/@/copyleft/@/gpl.html}).
+However, an existing @file{COPYING} file will never be overwritten by
+@command{automake}.
+
+@item
+The options @option{no-installman} and @option{no-installinfo} are
+prohibited.
+@end itemize
+
+Future versions of Automake will add more checks at this level of
+strictness; it is advisable to be familiar with the precise requirements
+of the GNU standards.
+
+Future versions of Automake may, at this level of strictness, require
+certain non-standard GNU tools to be available to maintainer-only
+Makefile rules.  For instance, in the future @command{pathchk}
+(@pxref{pathchk invocation,,, coreutils, GNU Coreutils})
+may be required to run @samp{make dist}.
+
 @item foreign
+@cindex Strictness, @option{foreign}
+@cindex @option{foreign} strictness
+@opindex --foreign
 Automake will check for only those things that are absolutely
 required for proper operation.  For instance, whereas GNU standards
 dictate the existence of a @file{NEWS} file, it will not be required in
 this mode.  This strictness will also turn off some warnings by default
 (among them, portability warnings).
-The name comes from the fact that Automake is intended to be
-used for GNU programs; these relaxed rules are not the standard mode of
-operation.
-
-@item gnu
-Automake will check---as much as possible---for compliance to the GNU
-standards for packages.  This is the default.
 
 @item gnits
+@cindex Strictness, @option{gnits}
+@cindex @option{gnits} strictness
+@opindex --gnits
 Automake will check for compliance to the as-yet-unwritten @dfn{Gnits
 standards}.  These are based on 

[RFC PATCH 0/3] Work around autoconf/automake warnings skew (automake side)

[Zack Weinberg]

This patch set completes the work necessary on automake’s side to bring
ChannelDefs.pm back into sync between automake and autoconf.  It also
updates the documentation of warnings options and arranges to suppress
warnings from autom4te --trace in a cleaner fashion.

See <https://lists.gnu.org/archive/html/autoconf-patches/2020-09/msg8.html>
for the rationale for merge_WARNINGS, which is not used in automake, but will
be in autoconf.

Zack Weinberg (3):
  New utility function Automake::ChannelDefs::merge_WARNINGS.
  Use WARNINGS=none to suppress warnings from autom4te runs.
  Update documentation of warnings options and strictness levels.

 bin/aclocal.in  |  19 ++-
 bin/automake.in |   7 +-
 doc/automake.texi   | 250 +++-
 lib/Automake/ChannelDefs.pm |  67 +-
 4 files changed, 208 insertions(+), 135 deletions(-)

-- 
2.28.0

[RFC PATCH 2/3] Use WARNINGS=none to suppress warnings from autom4te runs.

[Zack Weinberg]

aclocal uses autom4te in trace mode to scan configure.ac for macros whose
definition is not yet available.  It has a kludge to prevent this from
producing spurious warnings, but a cleaner, fully backward compatible, way to
get the same effect is to set WARNINGS=none in the environment and not pass
down any -W options.  (This is better than passing -Wnone on the command line
because it automatically inherits to any subprocesses started by autom4te.)

Perl’s ‘local’ feature can be used to make the enviironment variable setting
temporary, reverting to the previous value when we exit the function.

automake also runs autom4te (technically autoconf) in trace mode; warnings
from this invocation will not be *spurious*, but in the common case where
the person running automake is going to run autoconf next, they will be
duplicated.  Therefore, make the same change to automake.

* bin/aclocal.in (trace_used_macros)
* bin/automake.in (scan_autoconf_traces):
  Use “local $ENV{WARNINGS}='none'” to suppress warnings from autom4te.
---
 bin/aclocal.in  | 19 ---
 bin/automake.in |  7 ++-
 2 files changed, 14 insertions(+), 12 deletions(-)

diff --git a/bin/aclocal.in b/bin/aclocal.in
index 42e8d839a..ca2f96324 100644
--- a/bin/aclocal.in
+++ b/bin/aclocal.in
@@ -762,22 +762,19 @@ sub trace_used_macros ()
   my %files = map { $map{$_} => 1 } keys %macro_seen;
   %files = strip_redundant_includes %files;
 
-  # When AC_CONFIG_MACRO_DIRS is used, avoid possible spurious warnings
-  # from autom4te about macros being "m4_require'd but not m4_defun'd";
-  # for more background, see:
-  # https://lists.gnu.org/archive/html/autoconf-patches/2012-11/msg4.html
-  # as well as autoconf commit 'v2.69-44-g1ed0548', "warn: allow aclocal
-  # to silence m4_require warnings".
-  my $early_m4_code .= "m4_define([m4_require_silent_probe], [-])";
+  # Suppress all warnings from this invocation of autom4te.
+  # In particular we want to avoid spurious warnings about
+  # macros being "m4_require'd but not m4_defun'd" because
+  # aclocal.m4 is not yet available.
+  local $ENV{WARNINGS} = 'none';
 
   my $traces = ($ENV{AUTOM4TE} || '@am_AUTOM4TE@');
   $traces .= " --language Autoconf-without-aclocal-m4 ";
-  $traces = "echo '$early_m4_code' | $traces - ";
 
   # Support AC_CONFIG_MACRO_DIRS also with older autoconf.
   # Note that we can't use '$ac_config_macro_dirs_fallback' here, because
-  # a bug in option parsing code of autom4te 2.68 and earlier will cause
-  # it to read standard input last, even if the "-" argument is specified
+  # a bug in option parsing code of autom4te 2.68 and earlier would cause
+  # it to read standard input last, even if the "-" argument was specified
   # early.
   # FIXME: To be removed in Automake 2.0, once we can assume autoconf
   #2.70 or later.
@@ -807,7 +804,7 @@ sub trace_used_macros ()
   # characters (like newlines).
   (map { "--trace='$_:\$f::\$n'" } (keys %macro_seen)));
 
-  verb "running $traces $configure_ac";
+  verb "running WARNINGS=$ENV{WARNINGS} $traces $configure_ac";
 
   my $tracefh = new Automake::XFile ("$traces $configure_ac |");
 
diff --git a/bin/automake.in b/bin/automake.in
index 67b729045..a88b835a5 100644
--- a/bin/automake.in
+++ b/bin/automake.in
@@ -5249,6 +5249,11 @@ sub scan_autoconf_traces
sinclude => 1,
  );
 
+  # Suppress all warnings from this invocation of autoconf.
+  # The user is presumably about to run autoconf themselves
+  # and will see its warnings then.
+  local $ENV{WARNINGS} = 'none';
+
   my $traces = ($ENV{AUTOCONF} || '@am_AUTOCONF@') . " ";
 
   # Use a separator unlikely to be used, not ':', the default, which
@@ -5257,8 +5262,8 @@ sub scan_autoconf_traces
   map { "--trace=$_" . ':\$f:\$l::\$d::\$n::\${::}%' }
   (keys %traced));
 
+  verb "running WARNINGS=$ENV{WARNINGS} $traces";
   my $tracefh = new Automake::XFile ("$traces $filename |");
-  verb "reading $traces";
 
   @cond_stack = ();
   my $where;
-- 
2.28.0