Control: close -1 Control: tag -1 + wontfix On Fri, Aug 03, 2018 at 10:54:44AM +0800, Ralf Treinen wrote: > Package: autopkgtest > Version: 5.4.1 > Severity: normal > > Hi, > > the flaky restriction does not seem to work in case of an unsatisfiable > test dependency. This can be observed on ci.debian.net here: > > https://ci.debian.net/data/autopkgtest/unstable/amd64/w/why3/757615/log.gz > > In *this* particular case, the test dependency is not installable due to > a stupid mistake by myself, but that is besides the point. The offending > test why3+cvc4 is declared in debian/test/control with restriction flaky. > Strangely enough, my local installation of autopkgtest 5.4.1 on my laptop > works fine and treats its correctly: > > autopkgtest [10:32:17]: @@@@@@@@@@@@@@@@@@@@ summary > why3+alt-ergo PASS > why3+cvc3 PASS > why3+cvc4 FLAKY non-zero exit status 1 > why3+spass PASS > why3+z3 PASS > why3+coq PASS > > Maybe it has something to do with how ci.d.n sets up the test bed.
The real bug is the fact that autopkgtest is OK with uninstallable dependencies. `flaky` is to be used when *the tests* are unreliable. Having uninstallable dependencies should still be treated as a failure, i.e. in your example above, The `why3+cvc4` test should actually be a failure. Now, there is the *valid* issue you reported in #905311 about how to handle the case when you have multiple alternatives where each of them might not be available on all architectures, but let's discuss that case separately.
signature.asc
Description: PGP signature
