Hi Rebecca, On 30-07-18 08:57, Rebecca N. Palmer wrote: > On 29/07/18 14:58, Paul Gevers wrote: >> [...]> On 27-07-18 22:49, Rebecca N. Palmer wrote: >>> [...] >>> Simon McVittie wrote: >>>> At some point I might add a way to mark some tests as trivial, with >>>> the intention that migration is never sped up by trivial tests passing, >>>> only slowed down by them failing - but that's a future feature. >>> >>> With the neutral state proposed here, a test can effectively mark itself >>> as trivial by returning 77(skip) if it passes.[...] >>> >>> However, that's also true of flaky (return 77 on fail), and I agree it >>> would be more user-friendly to make this option explicit. >> >> Which option, to use flaky versus skippable? Sorry, could you try to >> phrase that sentence again, because I don't get what you mean here. > > I meant that once you have skippable, you can use it to simulate either > flaky or trivial: > > #equivalent to Restrictions: flaky > Restrictions: skippable > Test-Command: do_the_test || exit 77 > > #equivalent to Restrictions: trivial > Restrictions: skippable > Test-Command: do_the_test && exit 77
Right. Not until now did I really understand what you meant by trivial. > but that this doesn't imply they shouldn't exist explicitly. Got it now. >>> Do you[Simon McVittie] propose to add this[trivial] to >>> autodep8-generated "check that we can >>> import the top-level module" tests? > > His original proposed use case (in > https://salsa.debian.org/ci-team/autopkgtest/merge_requests/19) was: >> I've seriously considered mass-bug-filing patches to add trivial >> autopkgtests to as many shared libraries as I can, because even >> something as trivial as "link an implementation of hello(1) to the >> library using pkg-config and -Wl,--no-as-needed" catches surprisingly >> many embarrassing packaging errors. > which would effectively be an autopkgtest-pkg-c, so probably yes. Not sure if that would be the same value of trivial. I mean, doing this may already be enough to be used for gating. >> A similar idea has come to my mind >> about allow-stderr, so I think this is worth discussing. > > Do you mean setting allow-stderr by default in autodep8-generated tests? That is indeed what I wanted to discuss. > There are packages that would pass only with that (e.g. theano prints a > warning to stderr if one of its Recommends is not installed), but I > don't know how many. Indeed, but right now they shouldn't be using autodep8 then. And I wonder if there are regression in this area if we want to block on that. I.e. deprecation warnings in Python are printed on stderr. Having a new Python version being blocked on this is NOT NICE in my opinion. I think failing on stderr is good by default, but package maintainers have no way to override it with autodep8, making it unusable by this class of packages. (This recently came up in the python3.7 transition). Paul
signature.asc
Description: OpenPGP digital signature
