Hi! Long reply with many facets. I’d appreciate if all of it could be read, even if it’s 231 lines (with quotes and spacing).
On Sun, 13 Sep 2026, Holger Levsen wrote: >On Sat, Sep 12, 2026 at 06:39:18AM +0200, Thorsten Glaser wrote: >> I’ve recently found out that reproducible-builds nowadays fails >> packages which produce different output when 'nocheck' is set vs. >> unset. > >just to be clear, are you talking about https://reproduce.debian.net/ >or https://tests.reproducible-builds.org/debian/ ? … uhm. The one that tests for Debian… i.e. that shows up on DDPO. That would be the latter given the link from DDPO. On Sun, 13 Sep 2026, Christoph Berg wrote: >This sounds like you have a whole lot of packages with that problem. >Care to share concrete cases? The most pressing one is mksh, where this is needed to provide something to the users that has the best quality. To explain, I’ll need to describe how and what the mksh source package builds for normal native builds. First, an mksh binary is built for /bin/mksh using the system glibc. Then, mksh-static binaries are built for all other supported libcs (dietlibc, musl, klibc) if they exist, otherwise one with glibc (basically only on the Hurd; trust me you do NOT want to build this statically against glibc if you can at all avoid it). These are *all* installed, but the ones that test badly are chmod -x’d. Having them still present is *extremely* useful in tracking down bugs in mksh, GCC, binutils, the libc in question, qemu, and linux. (I’m not kidding here.) These are installed into subdirectories. The musl build also has to run small test executables in order to find out whether the combination of musl version, architecture and kernel, *and* toolchain and compiler, supports static-PIE yet and enables it if found, to improve security. Then, the “best” libc from that is selected according to an order (currently, IIRC, I prefer musl over klibc, dietlibc last, as that produces the most reliable one), and then an lksh binary (basically mksh but with “long” as arithmetic type, a printf(1) builtin for early boot, and auto-enabling POSIX mode when called as sh) is built with that. (If none are found, lksh is built dynamically against glibc.) The mksh-static from the “best” libc is also linked as mksh-static into the PATH. And finally, an mksh-g is built against system glibc with ASan/UBSan. In a cross or nocheck build, you get all of mksh, mksh-static, lksh and mksh-g built against glibc, which for mksh-static is… not nice. But also, see below: it is very common for packages to include long full testsuite output in their binaries, and this is a very useful property to have. In both cases, the difference in binary packages does not impact the “package API”: the presence of test results does not change the way things behave, and the various mksh-static and lksh binaries built with different libcs also behave the same (in normal circum‐ stances; the glibc-built static ones may need the very identical nss solibs at runtime when 'cd ~user/' is used, which is why they are less optimal for mksh-static); the build profile wiki page mentions the exposed surface of packages as a driving concern. I do agree that packages whose exposed “API” significantly differs (e.g. leaving out an executable or library functions) if nocheck is given are buggy, but the cases I showed don’t do that. On Sun, 13 Sep 2026, Holger Levsen wrote: >> If (if!) a 'nocheck' build *profile* is supported by the package, >> then, if that build *profile* is set, the binary packages may not >> differ. >> >> But the 'nocheck' build option *is* allowed to have binary >> differences, and this is something important for various corner >> cases. > >i'm not sure you are correct here (you might well be!), so i'm >wondering: where is this documented? The not being allowed to have binary differences is a thing for build profiles and defined in the build profiles spec. It is not defined for the nocheck DEB_BUILD_OPTIONS, whose spec in Policy merely says it’s to not run the package testsuite, and which is commonly understood to facilitate cross builds as well. But it has always permitted binary package differences, such as storing the testsuite results into a binary package so the maintainer can inspect the *full* logs as opposed to the summary shown in the buildd log. So it’s not so much a documentation of “nocheck DEB_BUILD_OPTIONS is permitted to have differing binaries” but instead a “the nocheck profile is explicitly specified to not allow that, but profiles and DEB_BUILD_OPTIONS are in different spaces, and the DEB_BUILD_OPTIONS does not specify that for these packages, and it is currently in use and has always been and is in use for very good reasons”. On Sun, 13 Sep 2026, Christoph Berg wrote: >> This is wrong. > >I believe testing for that is very valuable and it should stay. I agree, but: the correct one. The build profile, not the option. >> So, please add it to DEB_BUILD_OPTIONS on either no or all builds, >> and investigate into setting the 'nocheck' build *profile* on only >> one build *iff* the package supports it. I’m not exactly sure how >> to actually test for that, but if it only gets added to the variable >> DEB_BUILD_PROFILES but not DEB_BUILD_OPTIONS, then it can just be >> ignored by packages not supporting the 'nocheck' *profile*. > >So the fix is to just set DEB_BUILD_PROFILES=nocheck along and then >the complaint is gone? :) That… probably won’t do. If you build any of the affected packages without and with that profile *and they do not support the profile*, then the build without will include check-dependent files (mksh-klibc in mksh’s case, test logs in others). I think we should figure out a way to make this work for all packages. >> If (if!) a 'nocheck' build *profile* is supported by the package, >> then, if that build *profile* is set, the binary packages may not >> differ. > >Unless I missed something, packages do not announce profiles they >support. You could only look at debian/rules and guess which profiles >make any difference. Oh, hmm. This is… interesting. >> But the 'nocheck' build option *is* allowed to have binary >> differences, and this is something important for various corner >> cases. > >Even if that is true, I think t.rb.o should still test for the >problem. In most cases, the difference should be unintentional and >then people can work on fixing it. I do not even disagree here: I do concur that a difference for the nocheck DEB_BUILD_OPTIONS can be unintentional and that it’d be good to check for that. However we need to do this in a way that packages that do make use of the being allowed to differ in binaries can continue to do so, *especially* in the face of build reproducibility becoming a testing migration requirement¹. ① I’d like to see if this allows me to bump prerequisite bugs, like lack of reproducibility in cmake-generated .tar/.zip, to RC… The probably-easiest way would be an allowlist on the reprotest side, but that wouldn’t scale and it would age badly. I could work with having to touch packages that need to keep the distinction allowed somehow. There are several ways we could do that: • a field in d/control, e.g. “XS-nocheck-option-differs: yes”, and if this is present, the nocheck DEB_BUILD_OPTIONS is not set by reprotest ⇒ we would need to figure out how to signal not-support for the nocheck *profile*, too • a new DEB_BUILD_OPTIONS with a long name that reprotest sets that allows d/rules to avoid the non-differing ones: ⓐ it could make the presence of the new option equivalen to nocheck and thus omit the check-dependent things from the first run as well; for this, it would need to be set in *both* runs ⓑ it would make the presence of the new option mean it should ignore(!) the presence of nocheck in the second run (and thus needs to be set in the second run only) Out of these two, to be honest I’d prefer option b because I believe that that gives us overall the better QA/test coverage, even though this may affront sensibilities. My reasoning here is that this new option in variant b signals to the package that it is being reprotested, and that the presence of nocheck does not have the standard Policy meaning for this build. Option b also has the benefit that packages cannot just make amends for the presence of that new flag for the first run and do other things that result in the package being reproducible only when that option is set but not in normal buildd builds. So, reprotest could add 'nocheck reprotest-do-check-if-you-must' to DEB_BUILD_OPTIONS for the second run only, and packages can then detect reprotest-do-check-if-you-must and, if present, drop nocheck from DEB_BUILD_OPTIONS *iff* they actually need it. Hm. Could even vary between 'nocheck reprotest-do-check-if-you-must' for the second run only and 'reprotest-prepare-for-nocheck' in both runs, so that some reprotest runs test with nocheck and some without, if this is deemed important. As long as affected packages have a way to run both consistently while other packages vary. I’d be fine with either of the two approaches, no preference between 1 and 2b, or if you figure out something else, we can discuss it. ⚠ BUT ⚠ could we please, at least while this discussion is still underfood, revert the addition of nocheck until we find a solution, so that the testing migration of packages is not yet affected? As to build profiles… hm. I think we need to consult the people driving that spec with regards to profile discoverability, and not set the profile until then (the profile requires the option) or in case of 1 (XS-nocheck-option-differs), only set the profile if not present, or in case of 2b let packages ignore the profile as well… that would even work, because if a package has <!nocheck> B-Ds, it’s signalling support for the profile, so dpkg assuming the profile is present would not matter. Thanks for your consideration, //mirabilos -- 11:56⎜«liwakura:#!/bin/mksh» also, i wanted to add mksh to my own distro │ i was disappointed that there is no makefile │ but somehow the Build.sh is the least painful built system i've ever seen │ honours CC, {CPP,C,LD}FLAGS properly │ looks cleary like done by someone who knows what they are doing
