Bug#976037: ms-gsl's autopkg tests are broken by design
On 12/1/20 10:55 AM, Nicholas Guriev wrote: > Hello! > On Sat, 28 Nov 2020 17:21:05 +0100 Matthias Klose > wrote: >> Now, there's at least one test (the clang-12 one), which never will > succeed. So >> what's the value of these tests when you always have failures due to >> non-existing versions? Please just test for the default gcc and > clang. > > The main idea is to let me discover issues as early as possible, before > new toolchains become default. When a new compiler goes into archive, > this CI test is triggered, although it does not block migrating when the > compiler is not available yet due to "skip-not-installable" restriction. > > Testing against default GCC and Clang is already done on buildd trough > the d/rules file and thanks to debhelper(7) which calls to `make test` > during package build. Does it make sense to duplicate the work and do > the same trough autopkgtest(1)? but the problem is that you don't get any valid test result for testing. The test will always fail in testing, and the migration to testing is not blocked because this is not seen as a regression. It doesn't matter if the tests will fail in unstable or not. So no, autopkg tests should not depend on packages which are not available in testing.
Bug#976037: ms-gsl's autopkg tests are broken by design
Hello! On Sat, 28 Nov 2020 17:21:05 +0100 Matthias Klose wrote: > Now, there's at least one test (the clang-12 one), which never will succeed. So > what's the value of these tests when you always have failures due to > non-existing versions? Please just test for the default gcc and clang. The main idea is to let me discover issues as early as possible, before new toolchains become default. When a new compiler goes into archive, this CI test is triggered, although it does not block migrating when the compiler is not available yet due to "skip-not-installable" restriction. Testing against default GCC and Clang is already done on buildd trough the d/rules file and thanks to debhelper(7) which calls to `make test` during package build. Does it make sense to duplicate the work and do the same trough autopkgtest(1)?
Bug#976037: ms-gsl's autopkg tests are broken by design
Control: reassign -1 src:ms-gsl 3.1.0-3 On Sb, 28 nov 20, 17:21:05, Matthias Klose wrote: > Source: src:ms-gsl > Version: 3.1.0-3 > Severity: serious > > $ cat debian/tests/control > Tests: with-gcc8-std14 with-gcc8-std17 > Depends: cmake, g++-8, libgtest-dev, pkg-config > Restrictions: allow-stderr, skip-not-installable > > Tests: with-gcc9-std14 with-gcc9-std17 > Depends: cmake, g++-9, libgtest-dev, pkg-config > Restrictions: allow-stderr, skip-not-installable > > Tests: with-gcc10-std17 with-gcc10-std20 > Depends: cmake, g++-10, libgtest-dev, pkg-config > Restrictions: allow-stderr, skip-not-installable > > Tests: with-clang8-std14 with-clang8-std17 > Depends: cmake, clang-8, libc++-8-dev, libc++abi-8-dev, libgtest-dev, > pkg-config > Restrictions: allow-stderr, skip-not-installable > > Tests: with-clang9-std14 with-clang9-std17 > Depends: cmake, clang-9, libc++-9-dev, libc++abi-9-dev, libgtest-dev, > pkg-config > Restrictions: allow-stderr, skip-not-installable > > Tests: with-clang10-std14 with-clang10-std17 > Depends: cmake, clang-10, libc++-10-dev, libc++abi-10-dev, libgtest-dev, > pkg-config > Restrictions: allow-stderr, skip-not-installable > > Tests: with-clang11-std17 with-clang11-std20 > Depends: cmake, clang-11, libc++-11-dev, libc++abi-11-dev, libgtest-dev, > pkg-config > Restrictions: allow-stderr, skip-not-installable > > Tests: with-clang12-std17 with-clang12-std20 > Depends: cmake, clang-12, libc++-12-dev, libc++abi-12-dev, libgtest-dev, > pkg-config > Restrictions: allow-stderr, skip-not-installable > > Now, there's at least one test (the clang-12 one), which never will succeed. > So > what's the value of these tests when you always have failures due to > non-existing versions? Please just test for the default gcc and clang. Reassigning to correct package. Kind regards, Andrei -- Looking after bugs assigned to unknown or inexistent packages signature.asc Description: PGP signature
Bug#976037: ms-gsl's autopkg tests are broken by design
Source: src:ms-gsl Version: 3.1.0-3 Severity: serious $ cat debian/tests/control Tests: with-gcc8-std14 with-gcc8-std17 Depends: cmake, g++-8, libgtest-dev, pkg-config Restrictions: allow-stderr, skip-not-installable Tests: with-gcc9-std14 with-gcc9-std17 Depends: cmake, g++-9, libgtest-dev, pkg-config Restrictions: allow-stderr, skip-not-installable Tests: with-gcc10-std17 with-gcc10-std20 Depends: cmake, g++-10, libgtest-dev, pkg-config Restrictions: allow-stderr, skip-not-installable Tests: with-clang8-std14 with-clang8-std17 Depends: cmake, clang-8, libc++-8-dev, libc++abi-8-dev, libgtest-dev, pkg-config Restrictions: allow-stderr, skip-not-installable Tests: with-clang9-std14 with-clang9-std17 Depends: cmake, clang-9, libc++-9-dev, libc++abi-9-dev, libgtest-dev, pkg-config Restrictions: allow-stderr, skip-not-installable Tests: with-clang10-std14 with-clang10-std17 Depends: cmake, clang-10, libc++-10-dev, libc++abi-10-dev, libgtest-dev, pkg-config Restrictions: allow-stderr, skip-not-installable Tests: with-clang11-std17 with-clang11-std20 Depends: cmake, clang-11, libc++-11-dev, libc++abi-11-dev, libgtest-dev, pkg-config Restrictions: allow-stderr, skip-not-installable Tests: with-clang12-std17 with-clang12-std20 Depends: cmake, clang-12, libc++-12-dev, libc++abi-12-dev, libgtest-dev, pkg-config Restrictions: allow-stderr, skip-not-installable Now, there's at least one test (the clang-12 one), which never will succeed. So what's the value of these tests when you always have failures due to non-existing versions? Please just test for the default gcc and clang.
