Re: CI cross builds failing, advice needed
Niels Möller writes: > I'm migrating nettle's CI setup on this branch, > https://gitlab.com/gnutls/nettle/-/tree/ci-nettle-build-images?ref_type=heads, > which is now partly working, both for native and cross builds. Everything seems to be up and working now, and merged to the master branch. I'm considering doing some further refactoring of the ci config, and then I want to get valgrind side-channel tests in. > New failures: One report from the static analyzer, that appear valid > (but harmless). Fixed. > Test failures of poly1305 on powerpc64el, which I haven't been able to > reproduce in a local cross build + qemu. (poly1305 happens to be the > only ppc assembly depending om power9 features). I initially depended on qemu-user-binfmt when running the cross tests, but that appears to not be reliable (it depends on getting the binfmt registrations into the kernel, but since the ci-jobs are run in a container, they use the host's kernel). After switching to starting qemu explicitly, make check EMULATOR=qemu-xxx, all the cross tests pass. I don't understand why the poly1305 test in particular failed initially. Regards, /Niels -- Niels Möller. PGP key CB4962D070D77D7FCB8BA36271D8F1FF368C6677. Internet email is subject to wholesale government surveillance. ___ nettle-bugs mailing list -- nettle-bugs@lists.lysator.liu.se To unsubscribe send an email to nettle-bugs-le...@lists.lysator.liu.se
Re: CI cross builds failing, advice needed
Simon Josefsson writes: > I think that it would be better for nettle to maintain separate > build-images and not rely on other project's build-images. We could add > 'gnutls/nettle-build-images' for this if you want? Or is there some > other more canonical home for nettle on gitlab already? This is now setup, at https://git.lysator.liu.se/nettle/build-images, mirrored to https://gitlab.com/gnutls/nettle-build-images, and resulting images published at https://gitlab.com/gnutls/nettle-build-images/container_registry/5633274 Thanks to Simon and Daiki for help setting this up. Currently one with tools for native builds (including the gnutls build), one qiwth cross compilers and qemu for cross builds (with cross-compilers and qemu), and one with clang tools, for it's static analyzer. Maybe the gnutls build should use its own image, or reuse something from gnutls/build-images, since it has lots of additional dependencies. I'm migrating nettle's CI setup on this branch, https://gitlab.com/gnutls/nettle/-/tree/ci-nettle-build-images?ref_type=heads, which is now partly working, both for native and cross builds. New failures: One report from the static analyzer, that appear valid (but harmless). Test failures of poly1305 on powerpc64el, which I haven't been able to reproduce in a local cross build + qemu. (poly1305 happens to be the only ppc assembly depending om power9 features). Regards, /Niels -- Niels Möller. PGP key CB4962D070D77D7FCB8BA36271D8F1FF368C6677. Internet email is subject to wholesale government surveillance. ___ nettle-bugs mailing list -- nettle-bugs@lists.lysator.liu.se To unsubscribe send an email to nettle-bugs-le...@lists.lysator.liu.se
Re: CI cross builds failing, advice needed
Niels Möller writes: > 1. Switch to the cross-fedora38 image > > (https://gitlab.com/gnutls/build-images/-/blob/master/docker-cross-fedora38/Dockerfile?ref_type=heads). > > 2. Use some official debian image (not maintained by gnutls), and >install needed cross compilers as part of each job. > > 3. Maintain a separate image (or images) for use by Nettle ci builds. >E.g., I'm considering adding more tests depending on valgrind, so >then it would be nice to include valgrind in the image. And it would >be nice if all apt commands in the ci scripts could be eliminated. I think that it would be better for nettle to maintain separate build-images and not rely on other project's build-images. We could add 'gnutls/nettle-build-images' for this if you want? Or is there some other more canonical home for nettle on gitlab already? > Efficiency could matter: extra work, like installing lots of packages as > part of each ci job, makes the ci more expensive, and I don't know what > limits there are to the gnutls project on gitlab, but by default. So > advice appreciated. Compared to gnutls pipeline, I don't think the nettle pipeline will be significant in the resource consumption. The build/gnutls job takes a while, and maybe it could be speed up somehow. My experience is that it can help to have some build image for stable released well-known OSs but not create one for every single variant. However looking at build times most of the jobs are 1-3 minutes which is quite quick already. I usually don't bother until the jobs start to take more than 5 minutes to complete. /Simon > PS. On the topic on valgrind: Is it possible/practical to use valgrind > in a cross setting, e.g., an x86_64 host, cross compiling for arm, and > then use qemu-arm to run an arm valgrind binary in turn running the arm > binary under test. Or is there some other way to run a cross valgrind? What do you want to achieve? If valgrind on arm is the goal, rather than testing the cross-build setup, it may be simpler to have a macOS build job with valgrind. /Simon signature.asc Description: PGP signature ___ nettle-bugs mailing list -- nettle-bugs@lists.lysator.liu.se To unsubscribe send an email to nettle-bugs-le...@lists.lysator.liu.se
CI cross builds failing, advice needed
Hi, I just noticed that Nettle cross ci builds fail, e.g., https://gitlab.com/gnutls/nettle/-/jobs/4791865769 This one fails with $ apt-get update -q Get:1 http://deb.debian.org/debian stable InRelease [151 kB] Get:2 http://deb.debian.org/debian-security stable-security InRelease [48.0 kB] Get:3 http://deb.debian.org/debian stable-updates InRelease [52.1 kB] Reading package lists... E: Repository 'http://deb.debian.org/debian stable InRelease' changed its 'Codename' value from 'bullseye' to 'bookworm' E: Repository 'http://deb.debian.org/debian-security stable-security InRelease' changed its 'Codename' value from 'bullseye-security' to 'bookworm-security' E: Repository 'http://deb.debian.org/debian stable-updates InRelease' changed its 'Codename' value from 'bullseye-updates' to 'bookworm-updates' related to recent debian release. And when trying to find how to upgrade the image used, BUILD_IMAGES_PROJECT: gnutls/build-images DEBIAN_CROSS_BUILD: buildenv-debian-cross [...] image: $CI_REGISTRY/$BUILD_IMAGES_PROJECT:$DEBIAN_CROSS_BUILD I found the recent commit https://gitlab.com/gnutls/build-images/-/commit/268e1060cce9f3532439b7f0d1b9a9eec4afce2b deleting the corresponding docker file. I'm not that familiar with docker images, but I guess some of the options are 1. Switch to the cross-fedora38 image (https://gitlab.com/gnutls/build-images/-/blob/master/docker-cross-fedora38/Dockerfile?ref_type=heads). 2. Use some official debian image (not maintained by gnutls), and install needed cross compilers as part of each job. 3. Maintain a separate image (or images) for use by Nettle ci builds. E.g., I'm considering adding more tests depending on valgrind, so then it would be nice to include valgrind in the image. And it would be nice if all apt commands in the ci scripts could be eliminated. Efficiency could matter: extra work, like installing lots of packages as part of each ci job, makes the ci more expensive, and I don't know what limits there are to the gnutls project on gitlab, but by default. So advice appreciated. Regards, /Niels PS. On the topic on valgrind: Is it possible/practical to use valgrind in a cross setting, e.g., an x86_64 host, cross compiling for arm, and then use qemu-arm to run an arm valgrind binary in turn running the arm binary under test. Or is there some other way to run a cross valgrind? -- Niels Möller. PGP key CB4962D070D77D7FCB8BA36271D8F1FF368C6677. Internet email is subject to wholesale government surveillance. ___ nettle-bugs mailing list -- nettle-bugs@lists.lysator.liu.se To unsubscribe send an email to nettle-bugs-le...@lists.lysator.liu.se
