Bug#1059168: Bug#1059165: src:zlib: fails to migrate to testing for too long: triggers autopkgtest issues

[Paul Gevers]

retitle 1059168 flaky autopkgtest
user debian...@lists.debian.org
usertag 1059168 flaky
thanks
BCC: control@b.d.o

Hi,

On 21-12-2023 10:04, Jérémy Lal wrote:
Thank you for https://bugs.debian.org/1059168 



I noticed "Marked as found in versions nodejs/18.19.0+dfsg-6 and reopened."
while the test is flaky, retrying it will make it pass, so the bug is 
not as important as it seems.
And indeed, https://tracker.debian.org/pkg/zlib 
 is no longer blocked by nodejs.


It happens that I typically file bugs about flaky tests with severity 
serious with the following reasoning:

"""
Because the unstable-to-testing migration software now blocks on
regressions in testing, flaky tests, i.e. tests that flip between
passing and failing without changes to the list of installed packages,
are causing people unrelated to your package to spend time on these
tests.
"""

Having said that, I hardly ever fight over the severity if the 
maintainer(s) drop severity. It depends a bit on how flaky the test is, 
passing 1/3 or worse is really bad, passing more than 9/10 can be 
defended as acceptable but ideally should be fixed. My personal boundary 
for filing flaky bugs seems to lay around 1/5 to failures.


Paul


OpenPGP_signature.asc
Description: OpenPGP digital signature

Bug#1059165: [Pkg-javascript-devel] Bug#1059165: src:zlib: fails to migrate to testing for too long: triggers autopkgtest issues

[Jérémy Lal]

Le jeu. 21 déc. 2023 à 08:59, Paul Gevers  a écrit :

> Hi,
>
> On 20-12-2023 23:08, Jérémy Lal wrote:
> > I did the one for burp.
>
> Thanks for that.
>
> > Dolfin has active maintainers, though.
>
> I think I need to file a "flaky autopkgtest" bug report against dolfin.


Thank you for https://bugs.debian.org/1059168

I noticed "Marked as found in versions nodejs/18.19.0+dfsg-6 and reopened."
while the test is flaky, retrying it will make it pass, so the bug is not
as important as it seems.
And indeed, https://tracker.debian.org/pkg/zlib is no longer blocked by
nodejs.

Jérémy

Bug#1059165: [Pkg-javascript-devel] Bug#1059165: src:zlib: fails to migrate to testing for too long: triggers autopkgtest issues

[Paul Gevers]

Hi,

On 20-12-2023 23:08, Jérémy Lal wrote:

I did the one for burp.


Thanks for that.


Dolfin has active maintainers, though.


I think I need to file a "flaky autopkgtest" bug report against dolfin.

Paul


OpenPGP_signature.asc
Description: OpenPGP digital signature

Bug#1059165: [Pkg-javascript-devel] Bug#1059165: src:zlib: fails to migrate to testing for too long: triggers autopkgtest issues

[Jérémy Lal]

Le mer. 20 déc. 2023 à 22:33, Mark Brown  a écrit :

> On Wed, Dec 20, 2023 at 10:14:44PM +0100, Jérémy Lal wrote:
>
> > BURP wrong zlib version check in the failing test - this could be NMUed
>
> > DOLFIN has a single test failure, that is odd and unrelated as well -
> this
> > could be NMUed
>
> For non-technical reasons I can't do these NMUs myself if they're
> warranted/needed.
>

I did the one for burp.
Dolfin has active maintainers, though.

Jérémy

Bug#1059165: [Pkg-javascript-devel] Bug#1059165: src:zlib: fails to migrate to testing for too long: triggers autopkgtest issues

[Mark Brown]

On Wed, Dec 20, 2023 at 10:14:44PM +0100, Jérémy Lal wrote:

> BURP wrong zlib version check in the failing test - this could be NMUed

> DOLFIN has a single test failure, that is odd and unrelated as well - this
> could be NMUed

For non-technical reasons I can't do these NMUs myself if they're
warranted/needed.


signature.asc
Description: PGP signature

Bug#1059165: [Pkg-javascript-devel] Bug#1059165: src:zlib: fails to migrate to testing for too long: triggers autopkgtest issues

[Jérémy Lal]

Le mer. 20 déc. 2023 à 21:18, Mark Brown  a écrit :

> clone 1059165 -1
> reassign -1 nodejs
> retitle -1 autopkgtest failures on i386
> found -1 18.19.0+dfsg-6
> block 1059165 by -1
> kthxbye
>
> On Wed, Dec 20, 2023 at 08:15:31PM +0100, Paul Gevers wrote:
>
> > The Release Team considers packages that are out-of-sync between testing
> and
> > unstable for more than 30 days as having a Release Critical bug in
> testing
> > [1]. Your package src:zlib has been trying to migrate for 32 days [2].
> > Hence, I am filing this bug. The version in unstable triggers autopkgtest
> > failures in multiple packages (although I suspect that the current dolfin
> > issues are due to it being flaky). The failure for burp has already a bug
> > report against that package, which leaves nodejs on i386.
>
> ...
>
> > This bug will trigger auto-removal when appropriate. As with all new
> bugs,
> > there will be at least 30 days before the package is auto-removed.
>
> Not sure that's likely in the case of zlib?
>
> > If you believe your package is unable to migrate to testing due to issues
> > beyond your control, don't hesitate to contact the Release Team.
>
> There are non-technical issues with me doing active work on nodejs
> package but from a quick glance the log does not seem particularly
> plausibly related to zlib, and I note that the failures are
>
>not ok 498 parallel/test-debugger-heap-profiler
>not ok 962 parallel/test-fs-utimes-y2K38 # TODO : Fix flaky test
>
> the second of which especially doesn't inspire confidence that this is
> due to zlib rather than general updates to unstable setting off an
> already flaky test (eg, the kernel changed timing?).  Full log is:
>
>https://ci.debian.net/packages/n/nodejs/testing/i386/41176091/
>
> and looking at:
>
>https://ci.debian.net/packages/n/nodejs/testing/i386/
>
> there seem to be a number of packages triggering what from spot checks
> look to be the same or similar issues in nodejs in testing.
>
> I frankly don't really know what I'm supposed to do with this, the test
> results look like noise as far as zlib is concerned so I don't see
> anything to fix or investigate in the package itself.  AFAICT bugs don't
> get filed for autopkgtest failures like they do for build failures so
> perhaps this was just missed up until now?


NODEJS will probably pass with a retry request
The only test that failed here was parallel/test-debugger-heap-profiler
and it doesn't depend on zlib. It is somewhat flaky but usually not on i386.
I sent a retry request for nodejs/i386 testing to debci. There is a good
chance it'll pass this time.
Meanwhile, next nodejs upload will mark that test as flaky on all platforms.

BURP wrong zlib version check in the failing test - this could be NMUed
./test-suite.log
99%: Checks: 252, Failures: 1, Errors: 0
utest/test_fzp.c:95:F:Core:test_fzp_gzseek:0: Assertion 'fzp_seek(fzp,
d->pos, SEEK_SET)==-1' failed
It fails because the condition is still fulfilled, but it shouldn't be:
  START_TEST(test_fzp_gzseek)
  {
if(version_to_long(ZLIB_VERSION) <= version_to_long("1.2.3"))
  fzp_gzopen_old_zlib_seek_hack=1;
do_seek_tests(fzp_gzopen);
  }
END_TEST

DOLFIN has a single test failure, that is odd and unrelated as well - this
could be NMUed
224s with Timer(task) as t:
4608 
5224s sleep(0.05)
4609 
5224s > assert t.elapsed()[0] >= 0.05
4610 
5224s E assert 0.04 >= 0.05

Jérémy

Bug#1059165: src:zlib: fails to migrate to testing for too long: triggers autopkgtest issues

[Mark Brown]

clone 1059165 -1
reassign -1 nodejs
retitle -1 autopkgtest failures on i386
found -1 18.19.0+dfsg-6
block 1059165 by -1
kthxbye

On Wed, Dec 20, 2023 at 08:15:31PM +0100, Paul Gevers wrote:

> The Release Team considers packages that are out-of-sync between testing and
> unstable for more than 30 days as having a Release Critical bug in testing
> [1]. Your package src:zlib has been trying to migrate for 32 days [2].
> Hence, I am filing this bug. The version in unstable triggers autopkgtest
> failures in multiple packages (although I suspect that the current dolfin
> issues are due to it being flaky). The failure for burp has already a bug
> report against that package, which leaves nodejs on i386.

...

> This bug will trigger auto-removal when appropriate. As with all new bugs,
> there will be at least 30 days before the package is auto-removed.

Not sure that's likely in the case of zlib?

> If you believe your package is unable to migrate to testing due to issues
> beyond your control, don't hesitate to contact the Release Team.

There are non-technical issues with me doing active work on nodejs
package but from a quick glance the log does not seem particularly
plausibly related to zlib, and I note that the failures are

   not ok 498 parallel/test-debugger-heap-profiler
   not ok 962 parallel/test-fs-utimes-y2K38 # TODO : Fix flaky test 

the second of which especially doesn't inspire confidence that this is
due to zlib rather than general updates to unstable setting off an
already flaky test (eg, the kernel changed timing?).  Full log is:

   https://ci.debian.net/packages/n/nodejs/testing/i386/41176091/

and looking at:

   https://ci.debian.net/packages/n/nodejs/testing/i386/

there seem to be a number of packages triggering what from spot checks
look to be the same or similar issues in nodejs in testing.

I frankly don't really know what I'm supposed to do with this, the test
results look like noise as far as zlib is concerned so I don't see
anything to fix or investigate in the package itself.  AFAICT bugs don't
get filed for autopkgtest failures like they do for build failures so
perhaps this was just missed up until now?


signature.asc
Description: PGP signature

Bug#1059165: src:zlib: fails to migrate to testing for too long: triggers autopkgtest issues

[Paul Gevers]

Source: zlib
Version: 1:1.2.13.dfsg-3
Severity: serious
Control: close -1 1:1.3.dfsg-3
Tags: sid trixie
User: release.debian@packages.debian.org
Usertags: out-of-sync
Control: affects -1 src:burp src:dolfin src:nodejs
Control: block -1 by 1057880

Dear maintainer(s),

The Release Team considers packages that are out-of-sync between testing 
and unstable for more than 30 days as having a Release Critical bug in 
testing [1]. Your package src:zlib has been trying to migrate for 32 
days [2]. Hence, I am filing this bug. The version in unstable triggers 
autopkgtest failures in multiple packages (although I suspect that the 
current dolfin issues are due to it being flaky). The failure for burp 
has already a bug report against that package, which leaves nodejs on i386.


If a package is out of sync between unstable and testing for a longer 
period, this usually means that bugs in the package in testing cannot be 
fixed via unstable. Additionally, blocked packages can have impact on 
other packages, which makes preparing for the release more difficult. 
Finally, it often exposes issues with the package and/or
its (reverse-)dependencies. We expect maintainers to fix issues that 
hamper the migration of their package in a timely manner.


This bug will trigger auto-removal when appropriate. As with all new 
bugs, there will be at least 30 days before the package is auto-removed.


I have immediately closed this bug with the version in unstable, so if 
that version or a later version migrates, this bug will no longer affect 
testing. I have also tagged this bug to only affect sid and trixie, so 
it doesn't affect (old-)stable.


If you believe your package is unable to migrate to testing due to 
issues beyond your control, don't hesitate to contact the Release Team.


Paul

[1] https://lists.debian.org/debian-devel-announce/2023/06/msg1.html
[2] https://qa.debian.org/excuses.php?package=zlib



OpenPGP_signature.asc
Description: OpenPGP digital signature