Re: rhel8 test failure confirmation?
Zack Weinberg wrote: On Mon, Dec 4, 2023, at 7:14 PM, Jacob Bachmeyer wrote: Zack Weinberg wrote: [snip everything addressed in the other thread] Yes, there was a bit of confusion here; not only is the FileUtils module synchronized between autom4te and automake Thanks for reminding me that I need to make sure all those files are actually in sync before I cut the final 2.72 release. require Time::HiRes; import Time::HiRes qw(stat); I believe that the import is not actually necessary The previous line is a "require", not a "use", so I believe it _is_ necessary. Have I misunderstood? ... should do no harm as long as any use of stat in the code is prepared to handle floating-point timestamps. There's only one use, in 'sub mtime', and that's the place where we actively want the floating-point timestamps. Yes, before seeing your actual patch, I had the mistaken impression that this code was in autom4te itself, not the FileUtils module. The import is needed in the FileUtils module, so the patch is correct. -- Jacob
Re: rhel8 test failure confirmation?
On Mon, Dec 4, 2023, at 7:14 PM, Jacob Bachmeyer wrote: > Zack Weinberg wrote: [snip everything addressed in the other thread] > Yes, there was a bit of confusion here; not only is the FileUtils > module synchronized between autom4te and automake Thanks for reminding me that I need to make sure all those files are actually in sync before I cut the final 2.72 release. >> require Time::HiRes; >> import Time::HiRes qw(stat); > > I believe that the import is not actually necessary The previous line is a "require", not a "use", so I believe it _is_ necessary. Have I misunderstood? > ... should do no harm as long as any use of stat in the code > is prepared to handle floating-point timestamps. There's only one use, in 'sub mtime', and that's the place where we actively want the floating-point timestamps. zw
Re: rhel8 test failure confirmation?
Zack Weinberg wrote:
On Sun, Dec 3, 2023, at 4:49 PM, Karl Berry wrote:
There would not need to be much parsing, just "automake --version | grep
> HiRes" in that case, or "case `automake --version` in *HiRes*) ...;;
> easc" to avoid running grep if you want.
I specifically want to hear what Karl thinks.
I lean towards Jacob's view that automake --version | grep HiRes will
suffice. Not having a new option seems simpler/better in terms of later
understanding, too. --thanks, karl.
Did I misunderstand which program's --version output we are talking about?
I thought we were talking about automake's testsuite probing the behavior
of *autom4te*, but all the quoted text seems to be imagining a probe of
*automake* instead.
Yes, there was a bit of confusion here; not only is the FileUtils module
synchronized between autom4te and automake, but those two are near
"sound-alikes" as I read them. Oops.
The issue here seems to be determining if a fix that (I think)
originated in automake has been applied to the active autom4te.
[...]
I'm not using the identifier "HiRes" because the use of Time::HiRes is an
implementation detail that could change. For instance, if there's a third
party CPAN module that lets us get at nanosecond-resolution timestamps
*without* loss of precision due to conversion to an NV (aka IEEE double)
we could, in the future, look for that first.
That is fine, but "[HiRes]" or "[HiResTime]" is much shorter and we
could use it as the name of the feature regardless of the underlying
implementation. Characters in the first line of `autom4te --version`
are a finite resource if we want it to fit on a standard 80-column
terminal without wrapping. If we need to distinguish, "[HiRes]
[HiRes-ns]" could be used to indicate your hypothetical integer
nanosecond-resolution timestamp support, which would indicate also
having sub-second timestamp support.
I also suggest changing the tag, since the GNU Coding Standards call for
the version number to be indicated by the last space, but parenthesized
text between the name and version is supposed to be the package, so this
would lead to:
$ ./tests/autom4te --version
autom4te [HiResTime] (GNU Autoconf) 2.72d.6-49ab3-dirty
Copyright (C) 2023 Free Software Foundation, Inc.
[...]
Is this workable all the way around, everyone? Or should the feature be
indicated with another line after the license notice? ("This autom4te
has subsecond timestamp resolution.") My apologies for neglecting to
check this before suggesting a tag in the --version output.
The implementation is just
BEGIN
{
our $subsecond_timestamps = 0;
eval
{
require Time::HiRes;
import Time::HiRes qw(stat);
$subsecond_timestamps = 1;
}
}
Jacob, can you confirm that's an accurate test, given all the things you
said earlier about ways that grepping the source code might get it wrong?
That will determine if (a) Time::HiRes could be loaded and (b)
Time::HiRes::stat could be imported. This is the same test that
Autom{ak,4t}e::FileUtils effectively uses to use Time::HiRes::stat. I
believe that the import is not actually necessary (i.e. Time::HiRes
always exported Time::HiRes::stat) but it should do no harm as long as
any use of stat in the code is prepared to handle floating-point
timestamps. As long as the autom4te script and its library are
consistent (which is the distribution's problem if they screw that up),
this test should be accurate.
-- Jacob
Re: rhel8 test failure confirmation?
I thought we were talking about automake's testsuite probing the behavior of *autom4te* Yes, exactly. Sorry for the confusion. $ ./tests/autom4te --version autom4te (GNU Autoconf) 2.72d.6-49ab3-dirty (subsecond timestamps supported) Looks good to me. Thanks. a third party CPAN module Good point. The implementation is just Your patch sent to the autoconf list looked fine to me, FWIW. Perhaps worth mentioning somewhere that Automake is the consumer of this information that caused the change. You could point to https://bugs.gnu.org/64756 if you want something more convenient than the long multi-month thread on the automake discussion list (which I pointed to from that bug). Jacob, can you confirm that's an accurate test, +1 for Jacob's confirmation being welcome :).
Re: rhel8 test failure confirmation?
On Sun, Dec 3, 2023, at 4:49 PM, Karl Berry wrote:
>> There would not need to be much parsing, just "automake --version | grep
> > HiRes" in that case, or "case `automake --version` in *HiRes*) ...;;
> > easc" to avoid running grep if you want.
>
> I specifically want to hear what Karl thinks.
>
> I lean towards Jacob's view that automake --version | grep HiRes will
> suffice. Not having a new option seems simpler/better in terms of later
> understanding, too. --thanks, karl.
Did I misunderstand which program's --version output we are talking about?
I thought we were talking about automake's testsuite probing the behavior
of *autom4te*, but all the quoted text seems to be imagining a probe of
*automake* instead. Yinz can change automake yourselves, you don't need
me to jump in :-)
Assuming this is really about autom4te, how does this look?
$ ./tests/autom4te --version
autom4te (GNU Autoconf) 2.72d.6-49ab3-dirty (subsecond timestamps supported)
Copyright (C) 2023 Free Software Foundation, Inc.
[etc]
If subsecond timestamps are *not* supported, you'll get instead
autom4te (GNU Autoconf) 2.72d.6-49ab3-dirty
Copyright (C) 2023 Free Software Foundation, Inc.
[etc]
I'm not using the identifier "HiRes" because the use of Time::HiRes is an
implementation detail that could change. For instance, if there's a third
party CPAN module that lets us get at nanosecond-resolution timestamps
*without* loss of precision due to conversion to an NV (aka IEEE double)
we could, in the future, look for that first.
The implementation is just
BEGIN
{
our $subsecond_timestamps = 0;
eval
{
require Time::HiRes;
import Time::HiRes qw(stat);
$subsecond_timestamps = 1;
}
}
Jacob, can you confirm that's an accurate test, given all the things you
said earlier about ways that grepping the source code might get it wrong?
zw
Re: rhel8 test failure confirmation?
Karl Berry wrote: > There would not need to be much parsing, just "automake --version | grep > HiRes" in that case, or "case `automake --version` in *HiRes*) ...;; > easc" to avoid running grep if you want. I specifically want to hear what Karl thinks. I lean towards Jacob's view that automake --version | grep HiRes will suffice. Not having a new option seems simpler/better in terms of later understanding, too. --thanks, karl. P.S. As for case vs. grep, personally I find a simple if...grep easier to comprehend/test/debug than a case statement. (Especially the macro-ized AS_CASE, which just makes me have to look up its syntax every time I see it.) Also fewer lines of source. Granted calling the external grep is less efficient, but that seems insignificant to me. I understand Paul and others may disagree ... I agree that if...grep is more direct. I suggested the case alternative because it stands out in my memory after I needed it once, but I do not recall exactly why that contortion was needed. In configure, the efficiency difference is trivial because configure already runs many, many, many subprocesses. One more grep will not make a difference on any reasonable platform. -- Jacob
Re: rhel8 test failure confirmation?
upthread somewhere Karl (iirc) threw out a bikeshed idea like --has=. Pretty sure it wasn't me :).
Re: rhel8 test failure confirmation?
> There would not need to be much parsing, just "automake --version | grep > HiRes" in that case, or "case `automake --version` in *HiRes*) ...;; > easc" to avoid running grep if you want. I specifically want to hear what Karl thinks. I lean towards Jacob's view that automake --version | grep HiRes will suffice. Not having a new option seems simpler/better in terms of later understanding, too. --thanks, karl. P.S. As for case vs. grep, personally I find a simple if...grep easier to comprehend/test/debug than a case statement. (Especially the macro-ized AS_CASE, which just makes me have to look up its syntax every time I see it.) Also fewer lines of source. Granted calling the external grep is less efficient, but that seems insignificant to me. I understand Paul and others may disagree ...
Re: rhel8 test failure confirmation?
Mike Frysinger wrote:
On 02 Dec 2023 17:07, Jacob Bachmeyer wrote:
Mike Frysinger wrote:
On 06 Apr 2023 21:29, Jacob Bachmeyer wrote:
Karl Berry wrote:
jb> a more thorough test would locate the autom4te script and grep it
for the perllibdir that was substituted when autoconf was
configured.
I guess that would work.
Challenge accepted. Here's a refined version: (lines \-folded for email)
if $PERL -I${autom4te_perllibdir:-$(sed -n \
'/autom4te_perllibdir/{s/^.*|| //;s/;$//;s/^.//;s/.$//;p;q}' \
<$(command -v autom4te))} -MAutom4te::FileUtils \
-e 'exit defined $INC{q[Time/HiRes.pm]} ? 0 : 1'; then
# autom4te uses Time::HiRes
else
# autom4te does not use Time::HiRes
fi
this doesn't work on systems that wrap `autom4te`. [...]
[...]
so i don't know why we would need to set/export autom4te_perllibdir in our
wrapper. we've been doing this for over 20 years without ever setting that
var (or any other internal autoconf/automake var), so i'm pretty confident
our approach is OK.
No, not in the wrapper---in the automake ebuild script that runs
configure to match the autom4te that the wrapper will run. That test I
proposed checks for autom4te_perllibdir in the environment before
extracting it from autom4te precisely so distributions like Gentoo would
have a knob to turn if their packaging breaks that test.
That said, it turns out that this whole line of discussion is now a red
herring; see below.
[...]
i'm not aware of anything loading the Autom4te perl modules outside of the
autoconf projects. does that actually happen ? i don't think we want to
have automake start loading autoconf perl modules directly. going through
the CLI interface seems better at this point.
Autoconf and Automake are very closely associated; there is even some
shared code that is synchronized between them. Autom4te::FileUtils is
also Automake::FileUtils, for example.
The test we are discussing here was intended for Automake's configure
script to use to check if the installed Autoconf has high-resolution
timestamp support. It turned out that the test I wrote can give a false
positive, as some versions of other dependencies of Autom4te::FileUtils
/also/ use Time::HiRes, causing the test to correctly report that
Time::HiRes was loaded, but Autom4te::FileUtils nonetheless does not
actually use it. The test could probably be improved to fix the false
positives, but that would be getting into deep magic in Perl that might
not be fully reliable across Perl versions. (It would be necessary to
determine if (a) Time::HiRes::stat exists and (b)
Autom4te::FileUtils::stat is an alias to it. Having configure build a
special XSUB just to check this is well into "ridiculous" territory.)
As such, the Automake maintainers replaced this particular test with a
simpler test that just locates Autom4te/FileUtils.pm and greps it for
"Time::HiRes", thus the error message you received, which initially had
me confused because the test I proposed cannot produce that message as
it does not use grep.
An important bit of context to keep in mind is that I am not certain
that timestamp resolution is still a problem outside of the Automake and
Autoconf testsuites, since Autoconf and Automake now require cache files
to actually be newer than their sources and consider the cache files
stale if the timestamps are equal. This is a problem for the testsuite
because the testsuite is supposed to actually exercise the caching
mechanism, and higher-resolution timestamps can significantly reduce the
amount of time required to run the tests by reducing the delays needed
to ensure the caches will be valid.
-- Jacob
Re: rhel8 test failure confirmation?
On Sat, Dec 2, 2023, at 8:58 PM, Jacob Bachmeyer wrote: > Zack Weinberg wrote: >> Either way is no problem from my end, but it would be more work >> for automake (parsing --version output, instead of just checking the >> exit status of autom4te --assert-high-resolution-timestamp-support) >> Karl, do you have a preference here? I can make whatever you decide >> on happen, in the next couple of days. > > There would not need to be much parsing, just "automake --version | grep > HiRes" in that case, or "case `automake --version` in *HiRes*) ...;; > easc" to avoid running grep if you want. I specifically want to hear what Karl thinks. zw
Re: rhel8 test failure confirmation?
Zack Weinberg wrote: On Sat, Dec 2, 2023, at 7:33 PM, Jacob Bachmeyer wrote: Zack Weinberg wrote: Would it help if we added a command line option to autom4te that made it report whether it thought it could use high resolution timestamps? Versions of autom4te that didn't recognize this option should be conservatively assumed not to support them. Why not just add that information to the --version message? Add a "(HiRes)" tag somewhere if Time::HiRes is available? Either way is no problem from my end, but it would be more work for automake (parsing --version output, instead of just checking the exit status of autom4te --assert-high-resolution-timestamp-support) Karl, do you have a preference here? I can make whatever you decide on happen, in the next couple of days. There would not need to be much parsing, just "automake --version | grep HiRes" in that case, or "case `automake --version` in *HiRes*) ...;; easc" to avoid running grep if you want. -- Jacob
Re: rhel8 test failure confirmation?
On 02 Dec 2023 17:07, Jacob Bachmeyer wrote:
> Mike Frysinger wrote:
> > On 06 Apr 2023 21:29, Jacob Bachmeyer wrote:
> >> Karl Berry wrote:
> >>> jb> a more thorough test would locate the autom4te script and grep it
> >>> for the perllibdir that was substituted when autoconf was
> >>> configured.
> >>>
> >>> I guess that would work.
> >>>
> >> Challenge accepted. Here's a refined version: (lines \-folded for email)
> >>
> >> if $PERL -I${autom4te_perllibdir:-$(sed -n \
> >> '/autom4te_perllibdir/{s/^.*|| //;s/;$//;s/^.//;s/.$//;p;q}'
> >> \
> >> <$(command -v autom4te))} -MAutom4te::FileUtils \
> >> -e 'exit defined $INC{q[Time/HiRes.pm]} ? 0 : 1'; then
> >># autom4te uses Time::HiRes
> >> else
> >># autom4te does not use Time::HiRes
> >> fi
> >>
> >
> > this doesn't work on systems that wrap `autom4te`. Gentoo for example wraps
> > all autoconf & automake scripts to support parallel installs of different
> > versions. this way we can easily have access to every autoconf version. we
> > got this idea from Mandrake, so we aren't the only ones ;).
>
> If you install a wrapper script, (instead of, for example, making
> autom4te, etc. easily-repointable symlinks), then you must also set
symlinks are vastly inferior. they require privilege to modify, are not atomic,
and bleed into the rest of the system and other users. the wrappers we have in
Gentoo on the otherhand respect env vars (so you can e.g. WANT_AUTOCONF=2.53),
and by default scan the current set of generated files to see what was used, and
then see what's installed to automatically match. so if you're running automake
in a project built with automake 1.11 and you have that installed, we will use
it automatically for you. or you can force WANT_AUTOMAKE=latest to always use
the latest version.
> autom4te_perllibdir in the environment to the appropriate directory when
> building autoconf/automake. This (with the Gentoo-specific knowledge of
> where the active autom4te is actually located) should be easy to add to
> the ebuild.
>
> If autom4te_perllibdir is set in the environment, its value will be used
> instead of extracting that information from the autom4te script.
when e.g. autoconf 2.69 is built to use versioned paths at configure, compile,
and install time, there is no need to muck around with internal vars like the
autom4te_perllibdir. `autoconf-2.69` & `autom4te-2.69` always invokes exactly
the 2.69 version regardless of what `autoconf` does.
so i don't know why we would need to set/export autom4te_perllibdir in our
wrapper. we've been doing this for over 20 years without ever setting that
var (or any other internal autoconf/automake var), so i'm pretty confident
our approach is OK.
> > seems like the only reliable option is to invoke autom4te.
> > am_autom4te_ver=`$AUTOM4TE --version | sed -n '1{s:.*) ::;p}'
> > AS_CASE([$am_autom4te_ver],
> > ... do the matching ...
> >
> > what is the first autoconf release that has the fix ?
>
> The problem with testing autoconf versions for this is that Time::HiRes
> is an *optional* module in Perl. It was available from CPAN before it
> was bundled with Perl, and distributions technically can *unbundle* it
> from later Perl releases if they want. The only reliable way to know if
> Time::HiRes is available (without effectively reimplementing Perl's
> module search) is to try to load it. Autom4te now (correctly) uses
> Time::HiRes if it is available and falls back to Perl builtins if not,
> for any version of Perl. The best way to check if high-resolution
> timestamps are available to autom4te is to have perl load
> Autom4te::FileUtils and check if that also loaded Time::HiRes.
i'm not aware of anything loading the Autom4te perl modules outside of the
autoconf projects. does that actually happen ? i don't think we want to
have automake start loading autoconf perl modules directly. going through
the CLI interface seems better at this point.
-mike
signature.asc
Description: PGP signature
Re: rhel8 test failure confirmation?
On 02 Dec 2023 18:33, Jacob Bachmeyer wrote: > Zack Weinberg wrote: > > On Sat, Dec 2, 2023, at 6:37 PM, Karl Berry wrote: > >> The best way to check if high-resolution > >> timestamps are available to autom4te is to have perl load > >> Autom4te::FileUtils and check if that also loaded Time::HiRes. > >> > >> The problem with that turned out to be that Time::HiRes got loaded from > >> other system modules, resulting in the test thinking that autom4te used > >> it when that wasn't actually the case. That's what happened in practice > >> with your patch. > > > > Would it help if we added a command line option to autom4te that made it > > report whether it thought it could use high resolution timestamps? Versions > > of autom4te that didn't recognize this option should be conservatively > > assumed not to support them. > > Why not just add that information to the --version message? Add a > "(HiRes)" tag somewhere if Time::HiRes is available? All versions that > know to check if Time::HiRes is loaded will also know how to use it, > unlike the earlier test. parsing out the exact version from --version is already a pita. trying it freeform text that we also want tools to do substring searching on sounds extremely fragile and the opposite of future-proof. libtool at least has a --config option that dumps shell-link syntax. -mike signature.asc Description: PGP signature
Re: rhel8 test failure confirmation?
On 02 Dec 2023 19:17, Zack Weinberg wrote: > On Sat, Dec 2, 2023, at 6:37 PM, Karl Berry wrote: > > The best way to check if high-resolution > > timestamps are available to autom4te is to have perl load > > Autom4te::FileUtils and check if that also loaded Time::HiRes. > > > > The problem with that turned out to be that Time::HiRes got loaded from > > other system modules, resulting in the test thinking that autom4te used > > it when that wasn't actually the case. That's what happened in practice > > with your patch. > > Would it help if we added a command line option to autom4te that made it > report whether it thought it could use high resolution timestamps? Versions > of autom4te that didn't recognize this option should be conservatively > assumed not to support them. upthread somewhere Karl (iirc) threw out a bikeshed idea like --has=. i don't know if any of the autotools or GNU tools have such things that we could duplicate to avoid bikeshedding too much. that would make it work in this case easily, and setup framework for future checks. -mike signature.asc Description: PGP signature
Re: rhel8 test failure confirmation?
On Sat, Dec 2, 2023, at 7:33 PM, Jacob Bachmeyer wrote: > Zack Weinberg wrote: >> Would it help if we added a command line option to autom4te that made >> it report whether it thought it could use high resolution timestamps? >> Versions of autom4te that didn't recognize this option should be >> conservatively assumed not to support them. > > Why not just add that information to the --version message? Add a > "(HiRes)" tag somewhere if Time::HiRes is available? Either way is no problem from my end, but it would be more work for automake (parsing --version output, instead of just checking the exit status of autom4te --assert-high-resolution-timestamp-support) Karl, do you have a preference here? I can make whatever you decide on happen, in the next couple of days. >> (Of course there's the additional wrinkle that whether high >> resolution timestamps *work* depends on what filesystem >> autom4te.cache is stored in > > Is this actually still a problem (other than for ensuring the cache is > used in the testsuite) I don't *think* so but I don't understand the problem 100% so I could be missing something. zw
Re: rhel8 test failure confirmation?
Zack Weinberg wrote: On Sat, Dec 2, 2023, at 6:37 PM, Karl Berry wrote: The best way to check if high-resolution timestamps are available to autom4te is to have perl load Autom4te::FileUtils and check if that also loaded Time::HiRes. The problem with that turned out to be that Time::HiRes got loaded from other system modules, resulting in the test thinking that autom4te used it when that wasn't actually the case. That's what happened in practice with your patch. Would it help if we added a command line option to autom4te that made it report whether it thought it could use high resolution timestamps? Versions of autom4te that didn't recognize this option should be conservatively assumed not to support them. Why not just add that information to the --version message? Add a "(HiRes)" tag somewhere if Time::HiRes is available? All versions that know to check if Time::HiRes is loaded will also know how to use it, unlike the earlier test. (Of course there's the additional wrinkle that whether high resolution timestamps *work* depends on what filesystem autom4te.cache is stored in, but that's even harder to probe... one problem at a time?) Yes; even standard-resolution timestamps might not be "all there" with FAT and its infamous 2-second timestamp resolution. Is this actually still a problem (other than for ensuring the cache is used in the testsuite) after Bogdan's patches to require that cache files be strictly newer than their source files? -- Jacob
Re: rhel8 test failure confirmation?
On Sat, Dec 2, 2023, at 6:37 PM, Karl Berry wrote: > The best way to check if high-resolution > timestamps are available to autom4te is to have perl load > Autom4te::FileUtils and check if that also loaded Time::HiRes. > > The problem with that turned out to be that Time::HiRes got loaded from > other system modules, resulting in the test thinking that autom4te used > it when that wasn't actually the case. That's what happened in practice > with your patch. Would it help if we added a command line option to autom4te that made it report whether it thought it could use high resolution timestamps? Versions of autom4te that didn't recognize this option should be conservatively assumed not to support them. (Of course there's the additional wrinkle that whether high resolution timestamps *work* depends on what filesystem autom4te.cache is stored in, but that's even harder to probe... one problem at a time?) zw
Re: rhel8 test failure confirmation?
The best way to check if high-resolution timestamps are available to autom4te is to have perl load Autom4te::FileUtils and check if that also loaded Time::HiRes. The problem with that turned out to be that Time::HiRes got loaded from other system modules, resulting in the test thinking that autom4te used it when that wasn't actually the case. That's what happened in practice with your patch. Hence the current test that merely greps the autom4te source for HiRes, which is not sufficient, granted. But we have to do something. Distributions unbundling Time::HiRes ... at some point I don't mind saying that people have to set am_cv_sleep_fractional_seconds=false by hand. karl
Re: rhel8 test failure confirmation?
Mike Frysinger wrote:
On 06 Apr 2023 21:29, Jacob Bachmeyer wrote:
Karl Berry wrote:
jb> a more thorough test would locate the autom4te script and grep it
for the perllibdir that was substituted when autoconf was
configured.
I guess that would work.
Challenge accepted. Here's a refined version: (lines \-folded for email)
if $PERL -I${autom4te_perllibdir:-$(sed -n \
'/autom4te_perllibdir/{s/^.*|| //;s/;$//;s/^.//;s/.$//;p;q}' \
<$(command -v autom4te))} -MAutom4te::FileUtils \
-e 'exit defined $INC{q[Time/HiRes.pm]} ? 0 : 1'; then
# autom4te uses Time::HiRes
else
# autom4te does not use Time::HiRes
fi
this doesn't work on systems that wrap `autom4te`. Gentoo for example wraps
all autoconf & automake scripts to support parallel installs of different
versions. this way we can easily have access to every autoconf version. we
got this idea from Mandrake, so we aren't the only ones ;).
If you install a wrapper script, (instead of, for example, making
autom4te, etc. easily-repointable symlinks), then you must also set
autom4te_perllibdir in the environment to the appropriate directory when
building autoconf/automake. This (with the Gentoo-specific knowledge of
where the active autom4te is actually located) should be easy to add to
the ebuild.
If autom4te_perllibdir is set in the environment, its value will be used
instead of extracting that information from the autom4te script.
[...]
seems like the only reliable option is to invoke autom4te.
am_autom4te_ver=`$AUTOM4TE --version | sed -n '1{s:.*) ::;p}'
AS_CASE([$am_autom4te_ver],
... do the matching ...
what is the first autoconf release that has the fix ?
The problem with testing autoconf versions for this is that Time::HiRes
is an *optional* module in Perl. It was available from CPAN before it
was bundled with Perl, and distributions technically can *unbundle* it
from later Perl releases if they want. The only reliable way to know if
Time::HiRes is available (without effectively reimplementing Perl's
module search) is to try to load it. Autom4te now (correctly) uses
Time::HiRes if it is available and falls back to Perl builtins if not,
for any version of Perl. The best way to check if high-resolution
timestamps are available to autom4te is to have perl load
Autom4te::FileUtils and check if that also loaded Time::HiRes.
-- Jacob
Re: rhel8 test failure confirmation?
On Sat, Dec 2, 2023, at 6:58 AM, Mike Frysinger wrote: > On 06 Apr 2023 21:29, Jacob Bachmeyer wrote: >> Karl Berry wrote: >> > jb> a more thorough test would locate the autom4te script and grep it >> > for the perllibdir that was substituted when autoconf was >> > configured. >> > >> > I guess that would work. >> >> Challenge accepted. Here's a refined version: > > this doesn't work on systems that wrap `autom4te`. Gentoo for example wraps > all autoconf & automake scripts to support parallel installs of different > versions. this way we can easily have access to every autoconf version. we > got this idea from Mandrake, so we aren't the only ones ;). ... > what is the first autoconf release that has the fix ? > -mike 2.71 does not have the fix. Can you please test the 2.72d beta that I just put out (see https://lists.gnu.org/archive/html/autoconf/2023-11/msg0.html)? (yes, the version numbering for autoconf betas looks strange, it's a convention that predates semver and I don't have time to argue with anyone about it) zw
Re: rhel8 test failure confirmation?
On 06 Apr 2023 21:29, Jacob Bachmeyer wrote:
> Karl Berry wrote:
> > jb> a more thorough test would locate the autom4te script and grep it
> > for the perllibdir that was substituted when autoconf was
> > configured.
> >
> > I guess that would work.
>
> Challenge accepted. Here's a refined version: (lines \-folded for email)
>
> if $PERL -I${autom4te_perllibdir:-$(sed -n \
> '/autom4te_perllibdir/{s/^.*|| //;s/;$//;s/^.//;s/.$//;p;q}' \
> <$(command -v autom4te))} -MAutom4te::FileUtils \
> -e 'exit defined $INC{q[Time/HiRes.pm]} ? 0 : 1'; then
># autom4te uses Time::HiRes
> else
># autom4te does not use Time::HiRes
> fi
this doesn't work on systems that wrap `autom4te`. Gentoo for example wraps
all autoconf & automake scripts to support parallel installs of different
versions. this way we can easily have access to every autoconf version. we
got this idea from Mandrake, so we aren't the only ones ;).
as such, running git configure produces errors:
checking for autom4te... /usr/bin/autom4te
grep: /Autom4te/FileUtils.pm: No such file or directory
checking the filesystem timestamp resolution... 2
and then breaks some tests:
$ grep ^FAIL: test-suite.log
FAIL: t/depcomp2
FAIL: t/distcheck-no-prefix-or-srcdir-override
t/depcomp2.log showing:
+ cat stderr
grep: /Autom4te/FileUtils.pm: No such file or directory
+ sed /rm:.*conftest\.dSYM/d stderr
+ test -s stderr2
+ cat stderr2
grep: /Autom4te/FileUtils.pm: No such file or directory
+ _am_exit 1
seems like the only reliable option is to invoke autom4te.
am_autom4te_ver=`$AUTOM4TE --version | sed -n '1{s:.*) ::;p}'
AS_CASE([$am_autom4te_ver],
... do the matching ...
what is the first autoconf release that has the fix ?
-mike
signature.asc
Description: PGP signature
Re: rhel8 test failure confirmation?
Are you sure about that? Yes. Well, as sure as I can be. I don't see any $(...) constructs in Automake's *.m4 now, and this is certainly deliberate. We discussed shell portability many times over the decades of Automake development. I can't quickly find where it's documented, if anywhere, but as far as I know it is the status quo and I certainly see no reason to break new ground for this small patch. I got a fair bit of pushback on removing $(...) from config.guess Yes, I know. I was one of the people who reported problems because of that. on the grounds that Autoconf locates a POSIX shell and uses it for the bulk of configure My understanding is that Zack implemented that precisely to defend against the config.guess snafu. So for all but the latest Autoconf versions, it will not happen. Let's try: Thanks much! --all the best, karl.
Re: rhel8 test failure confirmation?
Karl Berry wrote:
Hi Jacob,
The guess was the two most probable locations: /usr/share/autoconf and
/usr/local/share/autoconf.
Wouldn't have worked on my own system :).
Challenge accepted.
Thanks!
if $PERL -I${autom4te_perllibdir:-$(sed -n \
'/autom4te_perllibdir/{s/^.*|| //;s/;$//;s/^.//;s/.$//;p;q}' \
<$(command -v autom4te))} -MAutom4te::FileUtils \
-e 'exit defined $INC{q[Time/HiRes.pm]} ? 0 : 1'; then
# autom4te uses Time::HiRes
unfortunately we are highly restricted in what we can use in basic
automake/conf shell code (as opposed to in the tests). Neither the
"command" command nor $(...) syntax can be used.
Are you sure about that? I got a fair bit of pushback on removing
$(...) from config.guess (where it actually is a problem because
config.guess is supposed to identify a variety of pre-POSIX systems and
can be run independently of configure) on the grounds that Autoconf
locates a POSIX shell and uses it for the bulk of configure (and the
auxiliary scripts like config.guess). Of course, Autoconf's "find a
POSIX shell" logic does not help DejaGnu, which installs a copy of
config.guess and runs it with /bin/sh according to its #! line...
For the former, I think there's an autoconf/make macro to look up a
program name along PATH?
From a quick glance at the manual, that would be
AC_PATH_PROG([AUTOM4TE], [autom4te]).
[...]
Would you be up for tweaking the check to use such
least-common-denominator shell stuff?
Let's try:
AC_PATH_PROG([AUTOM4TE], [autom4te])
if test x$autom4te_perllibdir = x; then
autom4te_perllibdir=`sed -n \
'/autom4te_perllibdir/{s/^.*|| //;s/;$//;s/^.//;s/.$//;p;q}' <$AUTOM4TE`
fi
if $PERL -I$autom4te_perllibdir -MAutom4te::FileUtils \
-e 'exit defined $INC{q[Time/HiRes.pm]} ? 0 : 1'; then
...
The backslash-newline in the sed command was added as a precaution
against line-wrap in email; the line could be combined.
Ordinarily Perl could not be used either, but since Automake is written
in Perl, I don't see a problem with doing so here. (If the system
doesn't have Perl, Automake won't get far.)
If the system lacks Perl, autom4te will not work either. The proposed
test uses Perl to determine a characteristic of a program that is
written in Perl. :-)
Not sure if $PERL is already
defined by the time at which this would be run, but it should be
possible to arrange with an ac prerequisite if needed.
That should be easy enough to rearrange, since this check must come
/after/ the autoconf version check---the pattern is only valid since
autoconf-2.52f, but Automake requires autoconf-2.65 or later.
-- Jacob
Re: rhel8 test failure confirmation?
Hi Jacob,
The guess was the two most probable locations: /usr/share/autoconf and
/usr/local/share/autoconf.
Wouldn't have worked on my own system :).
Challenge accepted.
Thanks!
if $PERL -I${autom4te_perllibdir:-$(sed -n \
'/autom4te_perllibdir/{s/^.*|| //;s/;$//;s/^.//;s/.$//;p;q}' \
<$(command -v autom4te))} -MAutom4te::FileUtils \
-e 'exit defined $INC{q[Time/HiRes.pm]} ? 0 : 1'; then
# autom4te uses Time::HiRes
unfortunately we are highly restricted in what we can use in basic
automake/conf shell code (as opposed to in the tests). Neither the
"command" command nor $(...) syntax can be used.
For the former, I think there's an autoconf/make macro to look up a
program name along PATH? For the latter, good old `...` suffices. (Not
sure there can be newlines in `...` though, even backslashed.)
Would you be up for tweaking the check to use such
least-common-denominator shell stuff?
Ordinarily Perl could not be used either, but since Automake is written
in Perl, I don't see a problem with doing so here. (If the system
doesn't have Perl, Automake won't get far.) Not sure if $PERL is already
defined by the time at which this would be run, but it should be
possible to arrange with an ac prerequisite if needed.
Thanks,
Karl
Re: rhel8 test failure confirmation?
Karl Berry wrote:
jb> The test also guesses the location of autoconf's Perl libraries;
I'm skeptical that any "guessing" of library locations would be reliable
enough.
The guess was the two most probable locations: /usr/share/autoconf and
/usr/local/share/autoconf.
jb> a more thorough test would locate the autom4te script and grep it
for the perllibdir that was substituted when autoconf was
configured.
I guess that would work.
Challenge accepted. Here's a refined version: (lines \-folded for email)
if $PERL -I${autom4te_perllibdir:-$(sed -n \
'/autom4te_perllibdir/{s/^.*|| //;s/;$//;s/^.//;s/.$//;p;q}' \
<$(command -v autom4te))} -MAutom4te::FileUtils \
-e 'exit defined $INC{q[Time/HiRes.pm]} ? 0 : 1'; then
# autom4te uses Time::HiRes
else
# autom4te does not use Time::HiRes
fi
This version matches a patten that was introduced in commit
c737451f8c17afdb477ad0fe72f534ea837e001e on 2001-09-13 preceding
autoconf-2.52f, and Automake currently requires autoconf-2.65 or later,
so this should work.
Getting the single quotes away from the value without directly
mentioning them is the purpose of the "s/^.//;s/.$//;" part of the sed
command. Wrapping it as "$(eval echo $(sed ...))" would have been
another option to have the shell strip the single quotes.
Automake and autoconf are not two independent tools. Automake completely
relies on autoconf.
It's not for me to hand down any final pronouncements, but personally I
feel strongly that the tests should not paper over this problem by
changing the way tests work in general. With rm -rf of the cache, or
autoconf -f, etc. That is not what users do, so that's not what the
tests should do, either. Such global changes could have all kinds of
other unknown/undesirable effects on the tests.
In contrast to setting the sleep value "as appropriate", which is what
is/should be already done, so changing the conditions under which it is
set is unlikely to cause any unforeseen additional problems.
While potentially compromising the real-world validity of the testsuite
is a legitimate concern, the fact that Automake depends on Autoconf does
not preclude the Automake testsuite from working around Autoconf
limitations in order to accurately test /Automake/.
-- Jacob
Re: rhel8 test failure confirmation?
jb> The test also guesses the location of autoconf's Perl libraries;
I'm skeptical that any "guessing" of library locations would be reliable
enough.
jb> a more thorough test would locate the autom4te script and grep it
for the perllibdir that was substituted when autoconf was
configured.
I guess that would work. A simpler option would be to run
autom4te --version
and if it's "ge" 2.72, the patch is applied.
Unless there is another pretest with the patch, granted, in which case
there would be need to be some finagling. But dealing with pretest
versions will become irrelevant as soon as the release is made.
Although ordinarily we would not want to test version numbers instead of
features, in this particular case I don't see a problem with it. If, for
instance, RH patches an autoconf with a lesser version and so the check
gives the "wrong" answer, the only result is that the sleep will be
1 sec, as long as it's been in every released automake ever. Hence, not
our problem. RH can patch the test too if they want to.
bd> [version number comparison] Best, not easy :).
Comparing version numbers in complete generality is hard. In the
particular case of autom4te, it doesn't seem problematic to me.
Compare '2.71' to '2.72-beta1-snapshot-12345678' or
vice versa with the suffixes :).
I think ge gives the right answer for our particular question ("is
autom4te patched") until autom4te reaches version 10. I don't think we
need to protect against that, but if you feel the need to do so, not too
hard.
it could be possible to make a loop similar to the one
I was not imagining running the same sort of loop to somehow try to
infer if autom4te is patched. That doesn't sound right. Determining if
autom4te is patched should be as definitive as possible, not just a
heuristic. And I don't think it's any harder to accomplish that.
I agree, so it would be best to have a workaround or a plan "B" for
those unpatched, for some reasonable time, at least. What would the
plan be is the question.
The workaround I think is (by far) the best is: set the sleep to 1 if
autom4te is not patched. That's it. Thus:
if () then
;
else
sleep = 1 second; # as it always was before
fi
Only a matter of where to put it...
I was imagining at the beginning of _AM_FILESYSTEM_TIMESTAMP_RESOLUTION
in m4/sanity.m4. I.e., when computing the sleep to use.
an issue with SOME OTHER TOOL.
Automake and autoconf are not two independent tools. Automake completely
relies on autoconf.
It's not for me to hand down any final pronouncements, but personally I
feel strongly that the tests should not paper over this problem by
changing the way tests work in general. With rm -rf of the cache, or
autoconf -f, etc. That is not what users do, so that's not what the
tests should do, either. Such global changes could have all kinds of
other unknown/undesirable effects on the tests.
In contrast to setting the sleep value "as appropriate", which is what
is/should be already done, so changing the conditions under which it is
set is unlikely to cause any unforeseen additional problems.
My $0.02 FWIW. Unfortunately I still have other projects I must attend
to, so can't dig into the actual code any time soon. But you guys are
doing that perfectly well anyway :). --thanks, karl.
Re: rhel8 test failure confirmation?
Bogdan wrote:
Jacob Bachmeyer , Mon Apr 03 2023 06:16:53
GMT+0200 (Central European Summer Time)
Karl Berry wrote:
[...]
What can we do about this?
As for automake: can we (you :) somehow modify the computation of the
sleep value to determine if autom4te can handle the HiRes testing or
not
(i.e., has the patch installed)? And then use the longer sleep in
automake testing if needed.
If you can locate Autom4te::FileUtils, grepping it for "Time::HiRes"
will tell you if autom4te supports sub-second timestamps, but then
you need more checks to validate that the filesystem actually has
sub-second timestamps.
A simple check:
if $PERL -I${autom4te_perllibdir:-/usr/share/autoconf}
-I/usr/local/share/autoconf \
-MAutom4te::FileUtils -e 'exit defined $INC{q[Time/HiRes.pm]}
? 0 : 1'; then
# autom4te uses Time::HiRes
else
# autom4te does not use Time::HiRes
fi
This method also has the advantage of implicitly also checking that
$PERL has Time::HiRes installed by determining if loading
Autom4te::FileUtils causes Time::HiRes to be loaded. (In other
words, this will give the correct answer on Perl 5.6 if Time::HiRes
was installed from CPAN or on later Perls if a distribution packager
has unbundled Time::HiRes and the user has not installed its package.)
Nice. The 0 and 1 may not be portable to each OS in the Universe (see
EXIT_SUCCESS and EXIT_FAILURE in exit(3)), but should be good/portable
enough for our goals. Or maybe some other simple solution.
Generally, "exit 0" reports success to the shell and any other exit
value is taken as false. I am unsure if POSIX actually requires that,
however.
As I understand, this could even be used to actually call the sub
which checks the timestamps, so we'd have a read-to-use test. Only a
matter of where to put it... Is there some code that runs *before* all
tests that could set some environment variable passed to the tests,
create a file, or whatever?
The intended implication was that that test would go in configure.
Verifying that the filesystem actually /has/ subsecond timestamps is a
separate issue; that test only detects whether autom4te will use
subsecond timestamps /if/ they are available.
The test also guesses the location of autoconf's Perl libraries; a more
thorough test would locate the autom4te script and grep it for the
perllibdir that was substituted when autoconf was configured.
-- Jacob
Re: rhel8 test failure confirmation?
On Tue, Apr 4, 2023, at 3:51 PM, Bogdan wrote: > Nice. The 0 and 1 may not be portable to each OS in the Universe > (see EXIT_SUCCESS and EXIT_FAILURE in exit(3)), but should be > good/portable enough for our goals. Or maybe some other simple solution. ISO C guarantees that exit(0) has the same effect as exit(EXIT_SUCCESS). [It does *not*, however, guarantee that EXIT_SUCCESS == 0.] zw
Re: rhel8 test failure confirmation?
Jacob Bachmeyer , Mon Apr 03 2023 06:16:53
GMT+0200 (Central European Summer Time)
Karl Berry wrote:
[...]
What can we do about this?
As for automake: can we (you :) somehow modify the computation of the
sleep value to determine if autom4te can handle the HiRes testing or
not
(i.e., has the patch installed)? And then use the longer sleep in
automake testing if needed.
If you can locate Autom4te::FileUtils, grepping it for "Time::HiRes"
will tell you if autom4te supports sub-second timestamps, but then you
need more checks to validate that the filesystem actually has
sub-second timestamps.
A simple check:
if $PERL -I${autom4te_perllibdir:-/usr/share/autoconf}
-I/usr/local/share/autoconf \
-MAutom4te::FileUtils -e 'exit defined $INC{q[Time/HiRes.pm]}
? 0 : 1'; then
# autom4te uses Time::HiRes
else
# autom4te does not use Time::HiRes
fi
This method also has the advantage of implicitly also checking that
$PERL has Time::HiRes installed by determining if loading
Autom4te::FileUtils causes Time::HiRes to be loaded. (In other words,
this will give the correct answer on Perl 5.6 if Time::HiRes was
installed from CPAN or on later Perls if a distribution packager has
unbundled Time::HiRes and the user has not installed its package.)
Nice. The 0 and 1 may not be portable to each OS in the Universe
(see EXIT_SUCCESS and EXIT_FAILURE in exit(3)), but should be
good/portable enough for our goals. Or maybe some other simple solution.
As I understand, this could even be used to actually call the sub
which checks the timestamps, so we'd have a read-to-use test. Only a
matter of where to put it... Is there some code that runs *before* all
tests that could set some environment variable passed to the tests,
create a file, or whatever?
[...]
It seems to me that using autoconf -f or similar is papering over the
problem, so that the tests are no longer testing the normal behavior.
Which does not sound desirable.
The Automake testsuite is supposed to test Automake, not Autoconf, so
working around Autoconf issues is appropriate.
I agree with this one, too. Don't do workarounds in tests to fix
Automake issues, just fix the issue in Automake :). But workarounds,
conditional code, skipping tests, etc. are allowed when using other
tools in Automake tests.
In this case, if
always using "autoconf -f" allows us to eliminate the sleeps entirely
(and does not expand the running time of Autoconf too much), we should
do that, at least in my view.
Yes, that could be the simplest. I (or someone) just need to find
out how to time individual tests (or maybe I'll just run 'time make
check TESTS=...') and do the actual measurements :).
I was hoping to have more time for this, but unfortunately, I got
busy with other things and this may have to wait a bit. Maybe this
week, maybe the next...
--
Regards - Bogdan ('bogdro') D. (GNU/Linux & FreeDOS)
X86 assembly (DOS, GNU/Linux):http://bogdro.evai.pl/index-en.php
Soft(EN): http://bogdro.evai.pl/soft http://bogdro.evai.pl/soft4asm
www.Xiph.org www.TorProject.org www.LibreOffice.org www.GnuPG.org
Re: rhel8 test failure confirmation?
Karl Berry , Mon Apr 03 2023 02:08:23 GMT+0200
(Central European Summer Time)
I modified my autom4te using the attached patch,
Thank you very very much for finding this, Bogdan.
:) Hope it saved some headaches :)
I'm glad that Paul has already installed it in Autoconf.
I can't easily confirm it for myself right now, but I'm hopeful.
(Maybe Frederic or others can try with the development autoconf.)
What can we do about this?
As for automake: can we (you :) somehow modify the computation of the
sleep value to determine if autom4te can handle the HiRes testing or not
(i.e., has the patch installed)? And then use the longer sleep in
automake testing if needed.
Perhaps. If not possible to call the autom4te's routine/module
directly, it could be possible to make a loop similar to the one
already in place (the filesystem timestamp resolution). But this loop
would require calling autom4te the number of times needed to establish
the result, which could make just this testing longer than the gain...
In fact, I wonder about autom4te providing an explicit way to test
whether it's been patched, because I suspect that a functionality test
(run autom4te a bunch of times on a bunch of newly-created files) would
be highly susceptible to wrong results. Checking the autom4te
--version number might be the easiest and best thing to do.
Until autoconf/autom4te has a '--has=some-feature-name' option,
checking the version number would probably be the best, as you say.
Best, not easy :). Compare '2.71' to '2.72-beta1-snapshot-12345678' or
vice versa with the suffixes :). Some nice Perl one-liner should do
the trick.
I think it's pretty crucial that the automake (or autoconf) tests not
spuriously fail due to filesystem timing, even for people that have the
"old" (unpatched) autom4te. People will certainly keep using current and
older releases of autoconf for years to come.
I agree, so it would be best to have a workaround or a plan "B" for
those unpatched, for some reasonable time, at least. What would the
plan be is the question. Force '-f'? Run an initial pre-test for
autom4te and generate the $AUTOCONF variable? Run an initial pre-test
for autom4te and force the timeout in the tests (like set an
environment variable, check for that in the sanity test and assume 1
second sleep if present)?
But, fortunately, these are still "just tests".
It seems to me that using autoconf -f or similar is papering over the
problem, so that the tests are no longer testing the normal behavior.
Which does not sound desirable.
Wdyt? --thanks again, karl.
On one hand - yes, you're right. On the other - we're working around
an issue with SOME OTHER TOOL. Just like #ifdef/#endif with C
compilers, Perl's 'eval' at runtime, or whatever.
I think we're allowed to do whatever workaround we find suitable to
make Automake work or make its tests pass. Obviously, it would be best
if autoconf/autom4te we patched (preferably from the beginning), but
if we can't have everything else perfect, we do our best, including
the old 'sleep 1' in the sanity test (and probably other things).
Bogdan
P.S. Sorry for not replying earlier. I wanted to come back with some
results, but apparently, it will take a bit more time.
--
Regards - Bogdan ('bogdro') D. (GNU/Linux & FreeDOS)
X86 assembly (DOS, GNU/Linux):http://bogdro.evai.pl/index-en.php
Soft(EN): http://bogdro.evai.pl/soft http://bogdro.evai.pl/soft4asm
www.Xiph.org www.TorProject.org www.LibreOffice.org www.GnuPG.org
Re: rhel8 test failure confirmation?
Karl Berry wrote:
[...]
What can we do about this?
As for automake: can we (you :) somehow modify the computation of the
sleep value to determine if autom4te can handle the HiRes testing or not
(i.e., has the patch installed)? And then use the longer sleep in
automake testing if needed.
If you can locate Autom4te::FileUtils, grepping it for "Time::HiRes"
will tell you if autom4te supports sub-second timestamps, but then you
need more checks to validate that the filesystem actually has sub-second
timestamps.
A simple check:
if $PERL -I${autom4te_perllibdir:-/usr/share/autoconf}
-I/usr/local/share/autoconf \
-MAutom4te::FileUtils -e 'exit defined $INC{q[Time/HiRes.pm]} ?
0 : 1'; then
# autom4te uses Time::HiRes
else
# autom4te does not use Time::HiRes
fi
This method also has the advantage of implicitly also checking that
$PERL has Time::HiRes installed by determining if loading
Autom4te::FileUtils causes Time::HiRes to be loaded. (In other words,
this will give the correct answer on Perl 5.6 if Time::HiRes was
installed from CPAN or on later Perls if a distribution packager has
unbundled Time::HiRes and the user has not installed its package.)
[...]
It seems to me that using autoconf -f or similar is papering over the
problem, so that the tests are no longer testing the normal behavior.
Which does not sound desirable.
The Automake testsuite is supposed to test Automake, not Autoconf, so
working around Autoconf issues is appropriate. In this case, if always
using "autoconf -f" allows us to eliminate the sleeps entirely (and does
not expand the running time of Autoconf too much), we should do that, at
least in my view.
-- Jacob
Re: rhel8 test failure confirmation?
I modified my autom4te using the attached patch, Thank you very very much for finding this, Bogdan. I'm glad that Paul has already installed it in Autoconf. I can't easily confirm it for myself right now, but I'm hopeful. (Maybe Frederic or others can try with the development autoconf.) What can we do about this? As for automake: can we (you :) somehow modify the computation of the sleep value to determine if autom4te can handle the HiRes testing or not (i.e., has the patch installed)? And then use the longer sleep in automake testing if needed. In fact, I wonder about autom4te providing an explicit way to test whether it's been patched, because I suspect that a functionality test (run autom4te a bunch of times on a bunch of newly-created files) would be highly susceptible to wrong results. Checking the autom4te --version number might be the easiest and best thing to do. I think it's pretty crucial that the automake (or autoconf) tests not spuriously fail due to filesystem timing, even for people that have the "old" (unpatched) autom4te. People will certainly keep using current and older releases of autoconf for years to come. It seems to me that using autoconf -f or similar is papering over the problem, so that the tests are no longer testing the normal behavior. Which does not sound desirable. Wdyt? --thanks again, karl.
Re: rhel8 test failure confirmation? [PATCH for problem affecting Automake testsuite]
Thanks for reporting that. I installed the attached slightly different
patch into Autoconf on Savannah. This patch also changes one other
instance of file timestamp comparison from < to <=. Something like this
should appear in the next Autoconf release.From 713d9822bbfb2923115065efaefed34a0113f8a1 Mon Sep 17 00:00:00 2001
From: Paul Eggert
Date: Sat, 1 Apr 2023 16:44:03 -0700
Subject: [PATCH] Fix timing bug on high-speed builds
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Problem reported by Bogdan via Jacob Bachmeyer in:
https://lists.gnu.org/r/autoconf/2023-04/msg2.html
* bin/autom4te.in: If a file timestamp equals a dependency’s
timestamp, consider the file to be out of date. Although this may
result in extra work, it fixes some rare timing bugs.
---
bin/autom4te.in | 12 +---
1 file changed, 5 insertions(+), 7 deletions(-)
diff --git a/bin/autom4te.in b/bin/autom4te.in
index 4b61f0a8..71d7e6a6 100644
--- a/bin/autom4te.in
+++ b/bin/autom4te.in
@@ -910,10 +910,8 @@ sub up_to_date ($)
return 0
if ! -f $tfile || ! -f $ofile;
- # The youngest of the cache files must be older than the oldest of
+ # The younger of the cache files must be older than the oldest of
# the dependencies.
- # FIXME: These timestamps have only 1-second resolution.
- # Time::HiRes fixes this, but assumes Perl 5.8 or later.
my $tmtime = mtime ($tfile);
my $omtime = mtime ($ofile);
my ($file, $mtime) = ($tmtime < $omtime
@@ -926,7 +924,7 @@ sub up_to_date ($)
# We depend at least upon the arguments.
foreach my $dep (@ARGV)
{
- if ($mtime < mtime ($dep))
+ if ($mtime <= mtime ($dep))
{
verb "up_to_date ($file): outdated: $dep";
return 0;
@@ -949,7 +947,7 @@ sub up_to_date ($)
# timestamp of that missing file was newer).
return 0
if ! $dep;
- if ($mtime < mtime ($dep))
+ if ($mtime <= mtime ($dep))
{
verb "up_to_date ($file): outdated: $dep";
return 0;
@@ -1038,7 +1036,7 @@ $icache_file = new Autom4te::XFile $icache, O_RDWR|O_CREAT;
$icache_file->lock (LOCK_EX)
if ($flock_implemented eq "yes");
-# Read the cache index if available and older than autom4te itself.
+# Read the cache index if available and younger than autom4te itself.
# If autom4te is younger, then some structures such as C4che might
# have changed, which would corrupt its processing.
Autom4te::C4che->load ($icache_file)
@@ -1105,7 +1103,7 @@ else
# Actual M4 expansion, if the user wants it, or if $output is old
# (STDOUT is pretty old).
handle_output ($req, $output)
- if $force || mtime ($output) < mtime ($ocache . $req->id);
+ if $force || mtime ($output) <= mtime ($ocache . $req->id);
}
# If we ran up to here, the cache is valid.
--
2.37.2
Re: rhel8 test failure confirmation? [PATCH for problem affecting Automake testsuite]
Jacob Bachmeyer , Sat Apr 01 2023 04:54:22
GMT+0200 (Central European Summer Time)
A quick introduction to the situation for the Autoconf list:
The Automake maintainers have encountered a bizarre issue with
sporadic random test failures, seemingly due to "disk writes not
taking effect" (as Karl Berry mentioned when starting the thread).
Bogdan appears to have traced the issue to autom4te caching and
offered a patch. I have attached a copy of Bogdan's patch.
Bogdan's patch is a subtle change: the cache is now considered stale
unless it is /newer/ than the source files, rather than being
considered stale only if the source files are newer. In short, this
patch causes the cache to be considered stale if its timestamp
/matches/ the source file, while it is currently considered valid if
the timestamps match. I am forwarding the patch to the Autoconf list
now because I concur with the change, noting that Time:HiRes is also
limited by the underlying filesystem and therefore is not a "magic
bullet" solution. Assuming the cache files are stale unless proven
otherwise is therefore correct.
Thank you :)
Note again that this is _Bogdan's_ patch I am forwarding unchanged. I
did not write it (but I agree with it).
[further comments inline below]
Bogdan wrote:
Bogdan , Sun Mar 05 2023 22:31:55 GMT+0100
(Central European Standard Time)
Karl Berry , Sat Mar 04 2023 00:00:56
GMT+0100 (Central European Standard Time)
Note that 'config.h' is older (4 seconds) than './configure',
which
shouldn't be the case as it should get updated with new values.
Indeed. That is the same sort of thing as I was observing with nodef.
But what (at any level) could be causing that to happen?
Files just aren't getting updated as they should be.
I haven't yet tried older releases of automake to see if their tests
succeed on the systems that are failing now. That's next on my list.
[...]
Another tip, maybe: cache again. When I compare which files are
newer than the only trace file I get in the failing 'backcompat2'
test ('autom4te.cache/traces.0'), I see that 'configure.ac' is
older than this file in the succeeding run, but it's newer in the
failing run. This could explain why 'configure' doesn't get updated
to put new values in config.h (in my case) - 'autom4te' thinks it's
up-to-date.
The root cause may be in 'autom4te', sub 'up_to_date':
# The youngest of the cache files must be older than the oldest of
# the dependencies.
# FIXME: These timestamps have only 1-second resolution.
# Time::HiRes fixes this, but assumes Perl 5.8 or later.
(lines 913-916 in my version).
This comment Bogdan cites is not correct: Time::HiRes could be
installed from CPAN on Perls older than 5.8, and might be missing from
a 5.8 or later installation if the distribution packager separated it
into another package. Nor is Time::HiRes guaranteed to fix the issue;
the infamous example is the FAT filesystem, where timestamps only have
2-second resolution. Either way, Time::HiRes is now used if
available, so this "FIXME" is fixed now. :-)
Good to hear :).
I didn't comment on the comment itself ;). Time::HiRes could have
been installed on Perl < 5.8, but since then it was in the core
modules, right? So, it *should* work for users by default then, and
Autoconf wouldn't require additional installations. That was the core
message of the comment, I think.
Perhaps 'configure.ac' in the case that fails is created "not
late enough" (still within 1 second) when compared to the cache,
and the cached values are taken, generating the old version of
'configure' which, in turn, generates old versions of the output
files.
Still a guess, but maybe a bit more probable now.
Does it work when you add '-f' to '$AUTOCONF'? It does for me -
again, about 20 sequential runs of the same set of tests and about
5 parallel with 4 threads. Zero failures.
I'd probably get the same result if I did a 'rm -fr
autom4te.cache' before each '$AUTOCONF' invocation.
[...]
More input (or noise):
1) The t/backcompat2.sh test (the only test which fails for me) is a
test which modifies configure.ac and calls $AUTOCONF several times.
2) Autom4te (part of Autoconf) has a 1-second resolution in checking
if the input files are newer than the cache.
Maybe. That comment could be wrong; the actual "sub mtime" is in
Autom4te::FileUtils. Does your version of that module use
Time::HiRes? Git indicates that use of Time::HiRes was added to
Autoconf at commit 3a9802d60156809c139e9b4620bf04917e143ee2 which is
between the 2.72a and 2.72c snapshot tags.
I'm using Autoconf provided by my system and it's version 2.71
(official package, I assume). Autom4te::FileUtils is using the
built-in stat() function.
3) Thus, a sequence: 'autoconf' + quickly modify configure.ac +
quickly run 'autoconf' may cause autom4te to use the old values from
the cache instead of processing the new configure.ac.
Re: rhel8 test failure confirmation? [PATCH for problem affecting Automake testsuite]
A quick introduction to the situation for the Autoconf list:
The Automake maintainers have encountered a bizarre issue with sporadic
random test failures, seemingly due to "disk writes not taking effect"
(as Karl Berry mentioned when starting the thread). Bogdan appears to
have traced the issue to autom4te caching and offered a patch. I have
attached a copy of Bogdan's patch.
Bogdan's patch is a subtle change: the cache is now considered stale
unless it is /newer/ than the source files, rather than being considered
stale only if the source files are newer. In short, this patch causes
the cache to be considered stale if its timestamp /matches/ the source
file, while it is currently considered valid if the timestamps match. I
am forwarding the patch to the Autoconf list now because I concur with
the change, noting that Time:HiRes is also limited by the underlying
filesystem and therefore is not a "magic bullet" solution. Assuming the
cache files are stale unless proven otherwise is therefore correct.
Note again that this is _Bogdan's_ patch I am forwarding unchanged. I
did not write it (but I agree with it).
[further comments inline below]
Bogdan wrote:
Bogdan , Sun Mar 05 2023 22:31:55 GMT+0100 (Central
European Standard Time)
Karl Berry , Sat Mar 04 2023 00:00:56 GMT+0100
(Central European Standard Time)
Note that 'config.h' is older (4 seconds) than './configure',
which
shouldn't be the case as it should get updated with new values.
Indeed. That is the same sort of thing as I was observing with nodef.
But what (at any level) could be causing that to happen?
Files just aren't getting updated as they should be.
I haven't yet tried older releases of automake to see if their tests
succeed on the systems that are failing now. That's next on my list.
[...]
Another tip, maybe: cache again. When I compare which files are
newer than the only trace file I get in the failing 'backcompat2'
test ('autom4te.cache/traces.0'), I see that 'configure.ac' is older
than this file in the succeeding run, but it's newer in the failing
run. This could explain why 'configure' doesn't get updated to put
new values in config.h (in my case) - 'autom4te' thinks it's up-to-date.
The root cause may be in 'autom4te', sub 'up_to_date':
# The youngest of the cache files must be older than the oldest of
# the dependencies.
# FIXME: These timestamps have only 1-second resolution.
# Time::HiRes fixes this, but assumes Perl 5.8 or later.
(lines 913-916 in my version).
This comment Bogdan cites is not correct: Time::HiRes could be
installed from CPAN on Perls older than 5.8, and might be missing from a
5.8 or later installation if the distribution packager separated it into
another package. Nor is Time::HiRes guaranteed to fix the issue; the
infamous example is the FAT filesystem, where timestamps only have
2-second resolution. Either way, Time::HiRes is now used if available,
so this "FIXME" is fixed now. :-)
Perhaps 'configure.ac' in the case that fails is created "not late
enough" (still within 1 second) when compared to the cache, and the
cached values are taken, generating the old version of 'configure'
which, in turn, generates old versions of the output files.
Still a guess, but maybe a bit more probable now.
Does it work when you add '-f' to '$AUTOCONF'? It does for me -
again, about 20 sequential runs of the same set of tests and about 5
parallel with 4 threads. Zero failures.
I'd probably get the same result if I did a 'rm -fr autom4te.cache'
before each '$AUTOCONF' invocation.
[...]
More input (or noise):
1) The t/backcompat2.sh test (the only test which fails for me) is a
test which modifies configure.ac and calls $AUTOCONF several times.
2) Autom4te (part of Autoconf) has a 1-second resolution in checking
if the input files are newer than the cache.
Maybe. That comment could be wrong; the actual "sub mtime" is in
Autom4te::FileUtils. Does your version of that module use Time::HiRes?
Git indicates that use of Time::HiRes was added to Autoconf at commit
3a9802d60156809c139e9b4620bf04917e143ee2 which is between the 2.72a and
2.72c snapshot tags.
3) Thus, a sequence: 'autoconf' + quickly modify configure.ac +
quickly run 'autoconf' may cause autom4te to use the old values from
the cache instead of processing the new configure.ac. "Quickly" means
within the same second.
It might be broader than that if your version is already using
Time::HiRes. If so, what filesystems are involved? I could see a
possible bug where multiple writes get the same mtime if they get
flushed to disk together. Time::HiRes will not help if this happens;
your patch will work around such a bug.
4) I ran the provided list of tests (t/backcompat2.sh,
t/backcompat3.sh, t/get-sysconf.sh, t/lex-depend.sh, t/nodef.sh,
t/remake-aclocal-version-mismatch.sh, t/subdir-add2-pr46.sh,
t/testsuite-summary-reference-log.sh) in batches of
Re: rhel8 test failure confirmation?
Bogdan , Sun Mar 05 2023 22:31:55 GMT+0100 (Central
European Standard Time)
Karl Berry , Sat Mar 04 2023 00:00:56 GMT+0100
(Central European Standard Time)
Note that 'config.h' is older (4 seconds) than './configure',
which
shouldn't be the case as it should get updated with new values.
Indeed. That is the same sort of thing as I was observing with nodef.
But what (at any level) could be causing that to happen?
Files just aren't getting updated as they should be.
I haven't yet tried older releases of automake to see if their tests
succeed on the systems that are failing now. That's next on my list.
[...]
Another tip, maybe: cache again. When I compare which files are
newer than the only trace file I get in the failing 'backcompat2' test
('autom4te.cache/traces.0'), I see that 'configure.ac' is older than
this file in the succeeding run, but it's newer in the failing run.
This could explain why 'configure' doesn't get updated to put new
values in config.h (in my case) - 'autom4te' thinks it's up-to-date.
The root cause may be in 'autom4te', sub 'up_to_date':
# The youngest of the cache files must be older than the oldest of
# the dependencies.
# FIXME: These timestamps have only 1-second resolution.
# Time::HiRes fixes this, but assumes Perl 5.8 or later.
(lines 913-916 in my version).
Perhaps 'configure.ac' in the case that fails is created "not late
enough" (still within 1 second) when compared to the cache, and the
cached values are taken, generating the old version of 'configure'
which, in turn, generates old versions of the output files.
Still a guess, but maybe a bit more probable now.
Does it work when you add '-f' to '$AUTOCONF'? It does for me -
again, about 20 sequential runs of the same set of tests and about 5
parallel with 4 threads. Zero failures.
I'd probably get the same result if I did a 'rm -fr autom4te.cache'
before each '$AUTOCONF' invocation.
If it does work for you, then maybe it would be better to add '-f'
to the 'AUTOCONF' variable where it's defined? Tests with a single
'$AUTOCONF' invocation won't notice (they will create their cache
anyway), while tests with multiple calls can maybe get fixed all at once.
[...]
More input (or noise):
1) The t/backcompat2.sh test (the only test which fails for me) is a
test which modifies configure.ac and calls $AUTOCONF several times.
2) Autom4te (part of Autoconf) has a 1-second resolution in checking
if the input files are newer than the cache.
3) Thus, a sequence: 'autoconf' + quickly modify configure.ac +
quickly run 'autoconf' may cause autom4te to use the old values from
the cache instead of processing the new configure.ac. "Quickly" means
within the same second.
4) I ran the provided list of tests (t/backcompat2.sh,
t/backcompat3.sh, t/get-sysconf.sh, t/lex-depend.sh, t/nodef.sh,
t/remake-aclocal-version-mismatch.sh, t/subdir-add2-pr46.sh,
t/testsuite-summary-reference-log.sh) in batches of 20 or more runs.
5) With the tools as they are on my system, I got a failure in the
t/backcompat2.sh test in the first batch (18th round, IIRC).
6) I modified my autom4te using the attached patch, which essentially
makes the mentioned sub 'up_to_date' work as if the cache is out of
date if its modtime (up to 1-second precision) is not only earlier,
but also equal to the modtime of any dependencies (including
configure.ac).
7) After modifying autom4te, I ran 120 rounds of the same set of tests
in single-threaded mode, and additional 120 rounds in parallel mode
(-j4). Total of 240 runs of all those 8 mentioned tests each. ZERO
FAILURES.
8) I brought autom4te to its original state and started running the
tests again. I got the first failure quite early (32nd run, IIRC).
I believe that shortening the sleep in Automake in the suspected
commit (720a1153134b833de9298927a432b4ea266216fb "m4: speed up
filesystem modification checks") is not an error in Automake. It
simply causes this issue in autom4te (not enough precision) to show up.
I'm not 100% sure about my results and conclusions, because
t/backcompat2.sh doesn't seem to be that quick in modifying
configure.ac to cause a false cache hit in autom4te. But these results
seem to confirm my previous results where using -f to bypass the cache
helped.
Another thing is that we have a bit of a special case here - quick
modifications and reconfigurations. So, even though autom4te has a
"FIXME" there, probably over 99% of users are not impacted. People
usually modify configure.ac manually, then run autoconf manually
(after 1 second) and then just ./configure.
What can we do about this?
- have autom4te patched and wait for the fix to reach a release (and
get installed on every possible end-user system?), and revert the
sleep until this is done,
- revert the commit and increase the test time again,
- fix the randomly-failing tests by adding '-f' to second and next
invocations of
Re: rhel8 test failure confirmation?
Hello,
I spent some more time on testing, and I could reproduce the issue on
Fedora rawhide, which gives more freedom for reproduction.
Regarding what I said earlier, since that wasn't clear enough, I couldn't
reproduce `make check` failures with the 1.16.5 official taball in a mock
environment, only with the trunk.
After few more hours, I could isolate 2 cases:
1) Random failures from multiple tests as stated earlier
2) Spurious failures of subobj.sh
Regarding 1), bisection led to 720a1153134b833de9298927a432b4ea266216fb
"m4: speed up filesystem modification checks". Whether it creates the
problem or simply reveals it is yet unclear. With this patch I could
reproduce multiple failures relatively easily (rarely more than one run).
Once reverted, the only test that fails from time to time on my side is
"subobj.sh" (twice out of ten trials). The amount of trials is probably not
enough, but based on previous observations, this patch looks to be a good
candidate for more investigations.
Regarding 2), that's less clear as this one fails less often, more time
would be needed to bisect it.
Fred.
On Sun, Mar 5, 2023 at 10:32 PM Bogdan wrote:
> Karl Berry , Sat Mar 04 2023 00:00:56 GMT+0100
> (Central European Standard Time)
> > Note that 'config.h' is older (4 seconds) than './configure', which
> > shouldn't be the case as it should get updated with new values.
> >
> > Indeed. That is the same sort of thing as I was observing with nodef.
> > But what (at any level) could be causing that to happen?
> > Files just aren't getting updated as they should be.
> >
> > I haven't yet tried older releases of automake to see if their tests
> > succeed on the systems that are failing now. That's next on my list.
>
> [...]
>
>
> Another tip, maybe: cache again. When I compare which files are
> newer than the only trace file I get in the failing 'backcompat2' test
> ('autom4te.cache/traces.0'), I see that 'configure.ac' is older than
> this file in the succeeding run, but it's newer in the failing run.
> This could explain why 'configure' doesn't get updated to put new
> values in config.h (in my case) - 'autom4te' thinks it's up-to-date.
> The root cause may be in 'autom4te', sub 'up_to_date':
>
># The youngest of the cache files must be older than the oldest of
># the dependencies.
># FIXME: These timestamps have only 1-second resolution.
># Time::HiRes fixes this, but assumes Perl 5.8 or later.
>
> (lines 913-916 in my version).
>
> Perhaps 'configure.ac' in the case that fails is created "not late
> enough" (still within 1 second) when compared to the cache, and the
> cached values are taken, generating the old version of 'configure'
> which, in turn, generates old versions of the output files.
>
> Still a guess, but maybe a bit more probable now.
>
> Does it work when you add '-f' to '$AUTOCONF'? It does for me -
> again, about 20 sequential runs of the same set of tests and about 5
> parallel with 4 threads. Zero failures.
> I'd probably get the same result if I did a 'rm -fr autom4te.cache'
> before each '$AUTOCONF' invocation.
>
> If it does work for you, then maybe it would be better to add '-f'
> to the 'AUTOCONF' variable where it's defined? Tests with a single
> '$AUTOCONF' invocation won't notice (they will create their cache
> anyway), while tests with multiple calls can maybe get fixed all at once.
>
> And I take back that the problem doesn't arise when I call the
> single test. I just wasn't "lucky enough". Got a failure now (without
> the fix), in the 7th attempt, while 40 or 60 (I did batches of 20)
> earlier succeeded (with the fix, of course).
>
> So, maybe it's not a problem of a NEW system by itself, just a
> FASTER system + 1-second resolution in 'autom4te'.
>
> --
> Regards - Bogdan ('bogdro') D. (GNU/Linux & FreeDOS)
> X86 assembly (DOS, GNU/Linux):http://bogdro.evai.pl/index-en.php
> Soft(EN): http://bogdro.evai.pl/soft http://bogdro.evai.pl/soft4asm
> www.Xiph.org www.TorProject.org www.LibreOffice.org www.GnuPG.org
>
>
>
Re: rhel8 test failure confirmation?
Oh, and I forgot, regarding autoconf: the RHEL 8 image I tested with had
autoconf 2.69. Failures could be detected with automake trunk, not with
official tarball (at least so far).
On Mon, Mar 6, 2023 at 11:19 PM Frederic Berat wrote:
> Hello,
>
> I spent some more time on testing, and I could reproduce the issue on
> Fedora rawhide, which gives more freedom for reproduction.
>
> Regarding what I said earlier, since that wasn't clear enough, I couldn't
> reproduce `make check` failures with the 1.16.5 official taball in a mock
> environment, only with the trunk.
> After few more hours, I could isolate 2 cases:
> 1) Random failures from multiple tests as stated earlier
> 2) Spurious failures of subobj.sh
>
> Regarding 1), bisection led to 720a1153134b833de9298927a432b4ea266216fb
> "m4: speed up filesystem modification checks". Whether it creates the
> problem or simply reveals it is yet unclear. With this patch I could
> reproduce multiple failures relatively easily (rarely more than one run).
> Once reverted, the only test that fails from time to time on my side is
> "subobj.sh" (twice out of ten trials). The amount of trials is probably not
> enough, but based on previous observations, this patch looks to be a good
> candidate for more investigations.
>
> Regarding 2), that's less clear as this one fails less often, more time
> would be needed to bisect it.
>
> Fred.
>
> On Sun, Mar 5, 2023 at 10:32 PM Bogdan wrote:
>
>> Karl Berry , Sat Mar 04 2023 00:00:56 GMT+0100
>> (Central European Standard Time)
>> > Note that 'config.h' is older (4 seconds) than './configure', which
>> > shouldn't be the case as it should get updated with new values.
>> >
>> > Indeed. That is the same sort of thing as I was observing with nodef.
>> > But what (at any level) could be causing that to happen?
>> > Files just aren't getting updated as they should be.
>> >
>> > I haven't yet tried older releases of automake to see if their tests
>> > succeed on the systems that are failing now. That's next on my list.
>>
>> [...]
>>
>>
>> Another tip, maybe: cache again. When I compare which files are
>> newer than the only trace file I get in the failing 'backcompat2' test
>> ('autom4te.cache/traces.0'), I see that 'configure.ac' is older than
>> this file in the succeeding run, but it's newer in the failing run.
>> This could explain why 'configure' doesn't get updated to put new
>> values in config.h (in my case) - 'autom4te' thinks it's up-to-date.
>> The root cause may be in 'autom4te', sub 'up_to_date':
>>
>># The youngest of the cache files must be older than the oldest of
>># the dependencies.
>># FIXME: These timestamps have only 1-second resolution.
>># Time::HiRes fixes this, but assumes Perl 5.8 or later.
>>
>> (lines 913-916 in my version).
>>
>> Perhaps 'configure.ac' in the case that fails is created "not late
>> enough" (still within 1 second) when compared to the cache, and the
>> cached values are taken, generating the old version of 'configure'
>> which, in turn, generates old versions of the output files.
>>
>> Still a guess, but maybe a bit more probable now.
>>
>> Does it work when you add '-f' to '$AUTOCONF'? It does for me -
>> again, about 20 sequential runs of the same set of tests and about 5
>> parallel with 4 threads. Zero failures.
>> I'd probably get the same result if I did a 'rm -fr autom4te.cache'
>> before each '$AUTOCONF' invocation.
>>
>> If it does work for you, then maybe it would be better to add '-f'
>> to the 'AUTOCONF' variable where it's defined? Tests with a single
>> '$AUTOCONF' invocation won't notice (they will create their cache
>> anyway), while tests with multiple calls can maybe get fixed all at once.
>>
>> And I take back that the problem doesn't arise when I call the
>> single test. I just wasn't "lucky enough". Got a failure now (without
>> the fix), in the 7th attempt, while 40 or 60 (I did batches of 20)
>> earlier succeeded (with the fix, of course).
>>
>> So, maybe it's not a problem of a NEW system by itself, just a
>> FASTER system + 1-second resolution in 'autom4te'.
>>
>> --
>> Regards - Bogdan ('bogdro') D. (GNU/Linux & FreeDOS)
>> X86 assembly (DOS, GNU/Linux):http://bogdro.evai.pl/index-en.php
>> Soft(EN): http://bogdro.evai.pl/soft http://bogdro.evai.pl/soft4asm
>> www.Xiph.org www.TorProject.org www.LibreOffice.org www.GnuPG.org
>>
>>
>>
Re: rhel8 test failure confirmation?
Karl Berry , Sat Mar 04 2023 00:00:56 GMT+0100
(Central European Standard Time)
Note that 'config.h' is older (4 seconds) than './configure', which
shouldn't be the case as it should get updated with new values.
Indeed. That is the same sort of thing as I was observing with nodef.
But what (at any level) could be causing that to happen?
Files just aren't getting updated as they should be.
I haven't yet tried older releases of automake to see if their tests
succeed on the systems that are failing now. That's next on my list.
[...]
Another tip, maybe: cache again. When I compare which files are
newer than the only trace file I get in the failing 'backcompat2' test
('autom4te.cache/traces.0'), I see that 'configure.ac' is older than
this file in the succeeding run, but it's newer in the failing run.
This could explain why 'configure' doesn't get updated to put new
values in config.h (in my case) - 'autom4te' thinks it's up-to-date.
The root cause may be in 'autom4te', sub 'up_to_date':
# The youngest of the cache files must be older than the oldest of
# the dependencies.
# FIXME: These timestamps have only 1-second resolution.
# Time::HiRes fixes this, but assumes Perl 5.8 or later.
(lines 913-916 in my version).
Perhaps 'configure.ac' in the case that fails is created "not late
enough" (still within 1 second) when compared to the cache, and the
cached values are taken, generating the old version of 'configure'
which, in turn, generates old versions of the output files.
Still a guess, but maybe a bit more probable now.
Does it work when you add '-f' to '$AUTOCONF'? It does for me -
again, about 20 sequential runs of the same set of tests and about 5
parallel with 4 threads. Zero failures.
I'd probably get the same result if I did a 'rm -fr autom4te.cache'
before each '$AUTOCONF' invocation.
If it does work for you, then maybe it would be better to add '-f'
to the 'AUTOCONF' variable where it's defined? Tests with a single
'$AUTOCONF' invocation won't notice (they will create their cache
anyway), while tests with multiple calls can maybe get fixed all at once.
And I take back that the problem doesn't arise when I call the
single test. I just wasn't "lucky enough". Got a failure now (without
the fix), in the 7th attempt, while 40 or 60 (I did batches of 20)
earlier succeeded (with the fix, of course).
So, maybe it's not a problem of a NEW system by itself, just a
FASTER system + 1-second resolution in 'autom4te'.
--
Regards - Bogdan ('bogdro') D. (GNU/Linux & FreeDOS)
X86 assembly (DOS, GNU/Linux):http://bogdro.evai.pl/index-en.php
Soft(EN): http://bogdro.evai.pl/soft http://bogdro.evai.pl/soft4asm
www.Xiph.org www.TorProject.org www.LibreOffice.org www.GnuPG.org
Re: rhel8 test failure confirmation?
Note that 'config.h' is older (4 seconds) than './configure', which shouldn't be the case as it should get updated with new values. Indeed. That is the same sort of thing as I was observing with nodef. But what (at any level) could be causing that to happen? Files just aren't getting updated as they should be. I haven't yet tried older releases of automake to see if their tests succeed on the systems that are failing now. That's next on my list. Thanks for looking into it, Karl
Re: rhel8 test failure confirmation?
Bogdan , Fri Mar 03 2023 14:56:07 GMT+0100 (Central
European Standard Time)
Bogdan , Fri Mar 03 2023 11:21:25 GMT+0100 (Central
European Standard Time)
Karl Berry , Wed Mar 01 2023 20:01:26 GMT+0100
(Central European Standard Time)
Does anyone have access to an RHEL 8-based machine? Alma Linux,
Rocky Linux,
original RHEL, or even (sort of) CentOS 8? It would be nice if someone
could run a make check there (from automake dev).
git clone -q git://git.savannah.gnu.org/automake.git
cd automake
./bootstrap
./configure && make >
make -j8 VERBOSE=1 check keep_testdirs=yes >
(choose whatever -j value you like)
[...]
This, along with my observation (below), MAY actually point to
some issue with Autoconf. Old systems work, new don't.
I take that back (for now). See below.
[...]
When I run
make check TESTS='t/nodef.sh t/backcompat2.sh t/backcompat3.sh
t/get-sysconf.sh t/lex-depend.sh
t/remake-aclocal-version-mismatch.sh t/subdir-add2-pr46.sh
t/testsuite-summary-reference-log.sh'
I get failures in t/backcompat2.sh. Sometimes, like once-twice in
ten runs.
I've added a simple 'debug' in the form of the 'stat' command and:
--
[...]
--
Note that 'config.h' is older (4 seconds) than './configure',
which shouldn't be the case as it should get updated with new values.
What's funny, when I run just the failing test alone, it works 10
times in a row. A "Schroedinger bug" - when you try to observe, you
change the environment and the bug disappears :).
I'll try to do some more digging later.
[...]
Anyway, to the point: I did a simple change - un-define the value on
the "else" path. Diff attached (not pretty, because it's just for
testing).
Since the change, I've done 22 test runs of the same test set (the
one with 8 tests, not the single one which always passed) and saw
completely no failures.
If it works for everybody, then perhaps some _AM_UNSET_OPTION macro
should be created in options.m4, to keep the code nice, and that
committed to the main code instead of my patch.
If it works, there may be other places worth checking.
Never mind. The tests are failing again. Sorry for the noise.
--
Regards - Bogdan ('bogdro') D. (GNU/Linux & FreeDOS)
X86 assembly (DOS, GNU/Linux):http://bogdro.evai.pl/index-en.php
Soft(EN): http://bogdro.evai.pl/soft http://bogdro.evai.pl/soft4asm
www.Xiph.org www.TorProject.org www.LibreOffice.org www.GnuPG.org
Re: rhel8 test failure confirmation?
Bogdan , Fri Mar 03 2023 11:21:25 GMT+0100 (Central
European Standard Time)
Karl Berry , Wed Mar 01 2023 20:01:26 GMT+0100
(Central European Standard Time)
Does anyone have access to an RHEL 8-based machine? Alma Linux,
Rocky Linux,
original RHEL, or even (sort of) CentOS 8? It would be nice if someone
could run a make check there (from automake dev).
git clone -q git://git.savannah.gnu.org/automake.git
cd automake
./bootstrap
./configure && make >
make -j8 VERBOSE=1 check keep_testdirs=yes >
(choose whatever -j value you like)
[...]
This, along with my observation (below), MAY actually point to some
issue with Autoconf. Old systems work, new don't.
I take that back (for now). See below.
[...]
When I run
make check TESTS='t/nodef.sh t/backcompat2.sh t/backcompat3.sh
t/get-sysconf.sh t/lex-depend.sh t/remake-aclocal-version-mismatch.sh
t/subdir-add2-pr46.sh t/testsuite-summary-reference-log.sh'
I get failures in t/backcompat2.sh. Sometimes, like once-twice in
ten runs.
I've added a simple 'debug' in the form of the 'stat' command and:
--
[...]
--
Note that 'config.h' is older (4 seconds) than './configure', which
shouldn't be the case as it should get updated with new values.
What's funny, when I run just the failing test alone, it works 10
times in a row. A "Schroedinger bug" - when you try to observe, you
change the environment and the bug disappears :).
I'll try to do some more digging later.
Done a bit more digging into how AM_INIT_AUTOMAKE works. No idea how
or if caching of M4 'defines' works, but from a programming point of
view, I see setting the [no-define] option conditionally:
m4_ifval([$3], [_AM_SET_OPTION([no-define])])dnl
and checking it unconditionally:
_AM_IF_OPTION([no-define]
and I ask myself: what if on the next run the value is still
(un)defined when it shouldn't be? It shouldn't be defined, because we
start with the empty value as the third parameter in the
t/backcompat2.sh test, but...
Anyway, to the point: I did a simple change - un-define the value on
the "else" path. Diff attached (not pretty, because it's just for
testing).
Since the change, I've done 22 test runs of the same test set (the
one with 8 tests, not the single one which always passed) and saw
completely no failures.
If it works for everybody, then perhaps some _AM_UNSET_OPTION macro
should be created in options.m4, to keep the code nice, and that
committed to the main code instead of my patch.
If it works, there may be other places worth checking.
--
Regards - Bogdan ('bogdro') D. (GNU/Linux & FreeDOS)
X86 assembly (DOS, GNU/Linux):http://bogdro.evai.pl/index-en.php
Soft(EN): http://bogdro.evai.pl/soft http://bogdro.evai.pl/soft4asm
www.Xiph.org www.TorProject.org www.LibreOffice.org www.GnuPG.org
diff --git a/m4/init.m4 b/m4/init.m4
index 9b809ad5b..a79500695 100644
--- a/m4/init.m4
+++ b/m4/init.m4
@@ -62,7 +62,7 @@ dnl Distinguish between old-style and new-style calls.
m4_ifval([$2],
[AC_DIAGNOSE([obsolete],
[$0: two- and three-arguments forms are deprecated.])
-m4_ifval([$3], [_AM_SET_OPTION([no-define])])dnl
+m4_ifval([$3], [_AM_SET_OPTION([no-define])], [m4_ifdef(_AM_MANGLE_OPTION([no-define]), [m4_undefine(_AM_MANGLE_OPTION([no-define])])])dnl
AC_SUBST([PACKAGE], [$1])dnl
AC_SUBST([VERSION], [$2])],
[_AM_SET_OPTIONS([$1])dnl
Re: rhel8 test failure confirmation?
Karl Berry , Wed Mar 01 2023 20:01:26 GMT+0100
(Central European Standard Time)
Does anyone have access to an RHEL 8-based machine? Alma Linux, Rocky Linux,
original RHEL, or even (sort of) CentOS 8? It would be nice if someone
could run a make check there (from automake dev).
git clone -q git://git.savannah.gnu.org/automake.git
cd automake
./bootstrap
./configure && make >
make -j8 VERBOSE=1 check keep_testdirs=yes >
(choose whatever -j value you like)
Confirmed here as well (OpenMandriva with Autoconf 2.71). Tested
Automake at the current 'master' and a little older branch.
Even in a pristine environment, on a different physical machines, I find
that a few tests fail, usually taken from the list below. Not always
the same tests fail, but I don't think it's directly related to
parallelism.
It's not. I run tests sequentially and also get failures. Randomly
in time, but always in the same test (and just one).
In the one I've looked into most, t/nodef.sh, it seems to be about disk
writes not taking effect. That test runs the usual autotool sequence
twice, both times writing to the same file named `output' and then
grepping it, e.g.:
grep 'DEFS.*-DVERSION=\\"UnIqUe' output && exit 1
If I put a sleep 1 between the two autotool invocations, it passes. If I
don't, it fails, because the second grep reads the `output' file written
by the first. Or so it seems. Which is crazy.
Certainly looks that way, but for me, it's more like
Autoconf/configure/config.status NOT updating the right files
(config.h in my case).
I could chalk it up to weird (extremely broken) hard disk behavior on
one machine, but not two different machines with completely dissimilar
hardware (including disks). It seems it must be in the os, in my case
Alma Linux:
$ grep ^VERSION= /etc/os-release
VERSION="8.7 (Stone Smilodon)"
(But I'd be very surprised if it is Alma-specific.)
All the tests pass fine on CentOS 7.
This, along with my observation (below), MAY actually point to some
issue with Autoconf. Old systems work, new don't.
Sigh. --thanks, karl.
Approximate lists of tests that "usually" fail:
FAIL: t/backcompat2.sh
FAIL: t/backcompat3.sh
FAIL: t/get-sysconf.sh
FAIL: t/lex-depend.sh
FAIL: t/nodef.sh
FAIL: t/remake-aclocal-version-mismatch.sh
FAIL: t/subdir-add2-pr46.sh
FAIL: t/testsuite-summary-reference-log.sh
When I run
make check TESTS='t/nodef.sh t/backcompat2.sh t/backcompat3.sh
t/get-sysconf.sh t/lex-depend.sh t/remake-aclocal-version-mismatch.sh
t/subdir-add2-pr46.sh t/testsuite-summary-reference-log.sh'
I get failures in t/backcompat2.sh. Sometimes, like once-twice in
ten runs.
I've added a simple 'debug' in the form of the 'stat' command and:
--
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating config.h
config.status: config.h is unchanged
+ cat config.h
/* config.h. Generated from config.h.in by configure. */
/* #undef PACKAGE */
/* #undef VERSION */
+ LC_ALL=C
+ stat ./configure
File: ./configure
Size: 106382 Blocks: 208IO Block: 4096 regular file
Device: 8,6 Inode: 6328331 Links: 1
Access: (0700/-rwx--) Uid: ( 1000/ bogdan) Gid: ( 1006/ bogdan)
Access: 2023-03-02 21:42:36.138654005 +0100
Modify: 2023-03-02 21:42:36.251653430 +0100
Change: 2023-03-02 21:42:36.252653424 +0100
Birth: 2023-03-02 21:42:36.138654005 +0100
+ LC_ALL=C
+ stat config.h
File: config.h
Size: 101 Blocks: 8 IO Block: 4096 regular file
Device: 8,6 Inode: 6328337 Links: 1
Access: (0600/-rw---) Uid: ( 1000/ bogdan) Gid: ( 1006/ bogdan)
Access: 2023-03-02 21:42:32.677671613 +0100
Modify: 2023-03-02 21:42:32.683671582 +0100
Change: 2023-03-02 21:42:32.693671531 +0100
Birth: 2023-03-02 21:42:32.677671613 +0100
+ grep '^ *# *define *PACKAGE *"pkgname" *$' config.h
+ am_exit_trap 1
--
Note that 'config.h' is older (4 seconds) than './configure', which
shouldn't be the case as it should get updated with new values.
What's funny, when I run just the failing test alone, it works 10
times in a row. A "Schroedinger bug" - when you try to observe, you
change the environment and the bug disappears :).
I'll try to do some more digging later.
--
Regards - Bogdan ('bogdro') D. (GNU/Linux & FreeDOS)
X86 assembly (DOS, GNU/Linux):http://bogdro.evai.pl/index-en.php
Soft(EN): http://bogdro.evai.pl/soft http://bogdro.evai.pl/soft4asm
www.Xiph.org www.TorProject.org www.LibreOffice.org www.GnuPG.org
Re: rhel8 test failure confirmation?
Hi Frédéric, While building the trunk directly led to check failures, Confirmation is good. rebuilding the RPM in a mock environment didn't. Puzzling. I'll likely spend more time next week to perform more testing. It may simply be an environment problem: I guess it's possible, though I've tried make check in quite a few different environments by now (latest tools, distro tools, minimal env, maximal env, different kinds of filesystems, etc.), and the make check failures have persisted in all cases. Also, I'd expect an environment issue to cause a test (say, nodef.sh) to consistently succeed or consistently fail. But what I observe is a "probabilistic" failure. If I run nodef on its own 10 times, it might fail 7 times and succeed 3 times, say. Or 6/4 or 8/2 or 5/5 or whatever. That smells like a caching bug somewhere to me. But ... another friend tried make check on a freebsd system and *also* got failures (more or less the same set of failing tests, including nodef.sh). So evidently it's unlikely to be a low-level kernel/disk subsystem timing or caching bug. I'll likely spend more time next week to perform more testing. For sure, any digging will be greatly appreciated! --thanks, karl.
Re: rhel8 test failure confirmation?
Hello, For the record, I made a first run of testing on a rhel 8 system. While building the trunk directly led to check failures, rebuilding the RPM in a mock environment didn't. I'll likely spend more time next week to perform more testing. It may simply be an environment problem: I noticed the RPM buildrequires weren't enough to execute bootstrap, there may be more to that. Frédéric Bérat Red Hat - Platform Tools https://www.redhat.com On Wed, Mar 1, 2023 at 10:55 PM Karl Berry wrote: > FAIL: t/remake-aclocal-version-mismatch.sh > > Thanks for trying it, Nick. > I'm glad it's not just me. > And I sure wonder wtf is going on :(. -k > >
Re: rhel8 test failure confirmation?
FAIL: t/remake-aclocal-version-mismatch.sh Thanks for trying it, Nick. I'm glad it's not just me. And I sure wonder wtf is going on :(. -k
Re: rhel8 test failure confirmation?
On 2023-03-01, Karl Berry wrote: > Does anyone have access to an RHEL 8-based machine? Alma Linux, Rocky > Linux, original RHEL, or even (sort of) CentOS 8? It would be nice if > someone could run a make check there (from automake dev). > git clone -q git://git.savannah.gnu.org/automake.git > cd automake > ./bootstrap > ./configure && make > > make -j8 VERBOSE=1 check keep_testdirs=yes > > (choose whatever -j value you like) % cat /etc/redhat-release Red Hat Enterprise Linux release 8.0 (Ootpa) % uname -r 4.18.0-305.10.2.el8_4.x86_64 I ran it twice, the first time out of my user directory (NFS), with no failures, the second out of /tmp (XFS), with one failure: FAIL: t/remake-aclocal-version-mismatch.sh [...] Testsuite summary for GNU Automake 1.16i # TOTAL: 2935 # PASS: 2808 # SKIP: 87 # XFAIL: 39 # FAIL: 1 # XPASS: 0 # ERROR: 0 See ./test-suite.log Please report to bug-autom...@gnu.org Cheers, Nick
