On Wed, 13 Feb 2019 21:23:00 +0100, Xavier Guimard wrote: > Some suggestions for pkg-js-autopkgtest based on pkg-js-autopkgtest > discussion with autodep8 maintainers: > - tests skipped should return a 77 exit code and all tests marked as > "Restrictions: skippable". It avoids to consider that a test succeeds > if maintainer skipped it, but needs a merge request to autodep8. See > > https://salsa.debian.org/ci-team/autodep8/blob/master/support/nodejs/generate > (changed by MR !11) > - runtime-deps* tests should be tagged as "Restrictions: superficial" > since these tests don't really test package features but just Perl > syntax > > Then with this 2 changes, if "build-deps.d" is skipped, success won't > give the benefit of 3-days-reduce.
Thanks for your work and the pull request [0]! Some thoughts and more questions: For the skippable part: - If I understand this correctly (from your text above and the spec [1]) then a skipped syntax.t and use.t would also lead to losing the benefit of faster migration? Do we want this? Or does it just have no influence? - As for the implementation in [0]: not sure if the "exit 0" in smoke is correct - What about the skipped tests within use.t and syntax.t? Should they or some of them also exit 77? For the superficial part: Hm, yeah, use.t and syntax.t don't test that everything in the package is fully functional; still, this "superficial" feels a bit weird. But probably it's correct according to [1]. In general I still don't have the full picture of what benefits and penalties for testing migration will result from which combination of the changes under which circumstances. Cheers, gregor [0] https://salsa.debian.org/perl-team/modules/packages/pkg-perl-tools/merge_requests/2 [1] https://salsa.debian.org/ci-team/autopkgtest/raw/master/doc/README.package-tests.rst -- .''`. https://info.comodo.priv.at -- Debian Developer https://www.debian.org : :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D 85FA BB3A 6801 8649 AA06 `. `' Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe `- NP: Cat Power: The Greatest
signature.asc
Description: Digital Signature