Hello, Thanks for the response. Indeed unit tests are already running during package build on my local machine and I use autopkgtest to test the installed packages for proper functioning. However this only allows me to test on x86_64. My concern was more that I was under the impression that Debian packages get automatically tested on various architectures, sometimes a bit esoteric, like Sparc, etc. I am already trying to determine which tests in upstream are sensitive to different versions of BLAS etc and to skip them or make them non-fatal. There are about a thousand tests that are run and while they all generally pass, between devs on the project there have been varying reports depending on operating systems for example, so I am expecting surprises ;)
(To be clear, I would like to _catch_ all failed tests, for reporting to upstream, but I also don't want to block the package just because the output of one particular solver is slightly different than expected when run e.g. on ARM or something.) But you are saying then that if it builds and tests just fine locally then I should not expect different conditions under ci.debian.net, is this correct? Currently I use cowbuilder with `gbp buildpackage` for package building and schroot for autopkgtest, I don't know if this is optimal but it seems to work well. thanks, Steve On Thu, Oct 25, 2018 at 12:44 AM Stuart Prescott <[email protected]> wrote: > > Hi Steve, > > You can run the test suite yourself on your local machine with either > autopkgtest (using a chroot/schroot/pbuilder/vm/etc) or without isolation > using sadt from devscripts. > > Running the tests yourself only tests against the hardware you have available > and so isn't probing different architectures, but then ci.debian.net doesn't > either. > > It is possible to build and run tests from gitlab CI. You can write your own > code for that or see an existing project for it: > > https://salsa.debian.org/salsa-ci-team/pipeline > > (I'm not sure I like the way it has been implemented in that project but it's > an option for you) > > Gitlab CI is still not probing many architectures. > > Established maintainers can get access to porterboxes with various > architectures to do their own testing and debugging on different hardware. I > don't know whether that's going to be an option for you, however. > > To the source of your problem -- what sort of comparisons are you having > trouble with? It's generally best that numerical tests use some sort of > "almost equals" test probing only an acceptable number of significant figures; > that should then pass on all architectures. Would it help to point to the > specific code at this stage? > > Cheers > Stuart > > > -- > Stuart Prescott http://www.nanonanonano.net/ [email protected] > Debian Developer http://www.debian.org/ [email protected] > GPG fingerprint 90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7
