Your message dated Sun, 2 Mar 2025 23:16:05 -0500
with message-id <Z8UtBeJ7EaB904OH@colossus>
and subject line Re: Bug#1041603: automake-1.16: flaky autopkgtest on arm*:
regularly times out
has caused the Debian Bug report #1041603,
regarding automake-1.16: flaky autopkgtest on arm*: regularly times out
to be marked as done.
This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.
(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact [email protected]
immediately.)
--
1041603: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1041603
Debian Bug Tracking System
Contact [email protected] with problems
--- Begin Message ---
Source: automake-1.16
Version: 1:1.16.5-1.3
Severity: important
User: [email protected]
Usertags: flaky
X-Debbugs-Cc: [email protected]
Control: affects -1 + src:glib2.0 src:vala src:python3-defaults
The automake test suite routinely takes more than 2h30 on slower
CPU architectures (armel and armhf), which is uncomfortably close to
autopkgtest's arbitrary limit of 10k seconds (a bit more than 2h45).
If it runs a bit slower for whatever reason - perhaps extra load on the
testbed machine - then the test fails with a timeout.
This means that when packages that are depended on by the test suite,
such as glib2.0, python3-defaults and vala, are trying to migrate to
testing, the test will randomly pass or fail, causing those packages to
be detected as having caused a regression when in fact they have not.
For example, in the recent history of
<https://ci.debian.net/packages/a/automake-1.16/testing/armel/>, we can
see these:
* testing only (migration reference): pass, 2h50
* testing only (migration reference): fail, 2h51
* testing + src:glib2.0 -3 from unstable: fail, 2h52
* testing + src:glib2.0 -3 from unstable: fail, 2h51
* testing + src:glib2.0 -2 from unstable: pass, 2h37
* testing + src:glib2.0 -2 from unstable: fail, 2h52
* testing + src:vala from unstable: pass, 2h27
* testing + src:vala from unstable: fail, 2h53
and this is getting in the way of glib2.0's migration. I am quite
confident that there is no runtime regression between glib2.0/2.76.4-2
and glib2.0/2.76.4-3: they are the same codebase, with only build-time
changes to work around a Meson bug.
The times shown on ci.debian.net are a bit longer than the test timeout
of 10k seconds, which is normal: they include activities like installing
dependencies and reporting results, which are not included in the test
timeout.
automake maintainers: can this test suite be made a bit faster without
sacrificing too much coverage, perhaps by skipping some repetitive or slow
tests, or by breaking it up into batches that each take 1 hour or less?
CI infrastructure maintainers: can we mitigate this by having debci run
autopkgtest with --timeout-factor=2 on the slower architectures, or by
only testing automake-1.16 on the faster architectures? I see riscv64 is
already using --timeout-factor=2.
smcv
--- End Message ---
--- Begin Message ---
Version: 1:1.17-3
This appears to be resolved in automake 1.17.
* Simon McVittie ([email protected]) wrote:
> Source: automake-1.16
> Version: 1:1.16.5-1.3
> Severity: important
> User: [email protected]
> Usertags: flaky
> X-Debbugs-Cc: [email protected]
> Control: affects -1 + src:glib2.0 src:vala src:python3-defaults
>
> The automake test suite routinely takes more than 2h30 on slower
> CPU architectures (armel and armhf), which is uncomfortably close to
> autopkgtest's arbitrary limit of 10k seconds (a bit more than 2h45).
> If it runs a bit slower for whatever reason - perhaps extra load on the
> testbed machine - then the test fails with a timeout.
>
> This means that when packages that are depended on by the test suite,
> such as glib2.0, python3-defaults and vala, are trying to migrate to
> testing, the test will randomly pass or fail, causing those packages to
> be detected as having caused a regression when in fact they have not.
>
> For example, in the recent history of
> <https://ci.debian.net/packages/a/automake-1.16/testing/armel/>, we can
> see these:
>
> * testing only (migration reference): pass, 2h50
> * testing only (migration reference): fail, 2h51
> * testing + src:glib2.0 -3 from unstable: fail, 2h52
> * testing + src:glib2.0 -3 from unstable: fail, 2h51
> * testing + src:glib2.0 -2 from unstable: pass, 2h37
> * testing + src:glib2.0 -2 from unstable: fail, 2h52
> * testing + src:vala from unstable: pass, 2h27
> * testing + src:vala from unstable: fail, 2h53
>
> and this is getting in the way of glib2.0's migration. I am quite
> confident that there is no runtime regression between glib2.0/2.76.4-2
> and glib2.0/2.76.4-3: they are the same codebase, with only build-time
> changes to work around a Meson bug.
>
> The times shown on ci.debian.net are a bit longer than the test timeout
> of 10k seconds, which is normal: they include activities like installing
> dependencies and reporting results, which are not included in the test
> timeout.
>
> automake maintainers: can this test suite be made a bit faster without
> sacrificing too much coverage, perhaps by skipping some repetitive or slow
> tests, or by breaking it up into batches that each take 1 hour or less?
>
> CI infrastructure maintainers: can we mitigate this by having debci run
> autopkgtest with --timeout-factor=2 on the slower architectures, or by
> only testing automake-1.16 on the faster architectures? I see riscv64 is
> already using --timeout-factor=2.
>
> smcv
--
Eric Dorland <[email protected]>
43CF 1228 F726 FD5B 474C E962 C256 FBD5 0022 1E93
--- End Message ---
