Bug#1006471: ruby3.0: reproducible builds: embeds path to various binaries
Control: tag -1 + pending On Wed, Jul 27, 2022 at 02:52:47PM +0100, Simon McVittie wrote: > Is the Ruby build intentionally putting EGREP into rbconfig.rb for use by > some other component, or is it just populating that file with everything > that Autoconf happens to have discovered, on the off chance that it > might become necessary at some point? If the latter, then that seems > like it will cause unpredictable action-at-a-distance if Autoconf stops > needing to discover some particular thing (for instance if Autoconf's > maintainers decide that they are only going to support systems where > the first grep in PATH is POSIX.1-2001 compliant, and stop checking for a > possibly-better-quality grep elsewhere). The later. Thanks for the explanation. This has already been fixed for ruby3.1, and is fixed in git for ruby3.0. signature.asc Description: PGP signature
Bug#1006471: ruby3.0: reproducible builds: embeds path to various binaries
On Wed, 27 Jul 2022 at 08:49:27 -0300, Antonio Terceiro wrote: > > On Fri, 25 Feb 2022 at 15:26:51 -0800, Vagrant Cascadian wrote: > > > The paths to various binaries, which differs on a usrmerge > > > vs. non-usrmerge system, are embedded in rbconfig.rb: > > > > > > /usr/lib/x86_64-linux-gnu/ruby/3.0.0/rbconfig.rb > > > > > > CONFIG["EGREP"]·=·"/bin/grep·-E" > > > vs. > > > CONFIG["EGREP"]·=·"/usr/bin/grep·-E" > > > > If these CONFIG variables are used for something at runtime, then this > > will become a practical problem as soon as Debian starts using merged-/usr > > buildds. > > Those variables are read from config.status during the builds. Maybe > this should be fixed centrally in autoconf instead? autoconf is designed to support arbitrarily bad host OSs, including those that are non-POSIX or otherwise defective, where the only fully-functional grep might be /opt/sw/addons/misc/gnu/grep or something; so it has a tendency to discover a known-good absolute path and save that. This is great if you're building Ruby on an awful 1990s Unix machine and the result of AC_PROG_EGREP will only be used during build, or if it is used at runtime but you only plan to run the resulting Ruby binaries on that same machine, but it goes wrong when facts about the build system start to diverge from facts about the host system. In this case, the fact that is different is the merged-/usr status of the build system and the host system, but it could be almost anything. The macros that Ruby uses to find these commands are probably AC_PROG_GREP, AC_PROG_EGREP, etc., which are not explicitly documented to output an absolute path (the documentation just says "whatever is chosen"), but looking at their implementation, it seems they do: they are like AC_PATH_PROG rather than AC_CHECK_PROG. It's entirely possible that Ruby is not doing this deliberately, those macros might well be a dependency for something else. Is the Ruby build intentionally putting EGREP into rbconfig.rb for use by some other component, or is it just populating that file with everything that Autoconf happens to have discovered, on the off chance that it might become necessary at some point? If the latter, then that seems like it will cause unpredictable action-at-a-distance if Autoconf stops needing to discover some particular thing (for instance if Autoconf's maintainers decide that they are only going to support systems where the first grep in PATH is POSIX.1-2001 compliant, and stop checking for a possibly-better-quality grep elsewhere). smcv
Bug#1006471: ruby3.0: reproducible builds: embeds path to various binaries
Control: clone -1 -2 Control: reassign -2 src:ruby3.1 Control: retitle -2 ruby3.1: reproducible builds: embeds path to various binaries Hi, On Sun, Jul 17, 2022 at 12:04:45PM +0100, Simon McVittie wrote: > Control: severity -1 serious > > On Fri, 25 Feb 2022 at 15:26:51 -0800, Vagrant Cascadian wrote: > > The paths to various binaries, which differs on a usrmerge > > vs. non-usrmerge system, are embedded in rbconfig.rb: > > > > > > https://tests.reproducible-builds.org/debian/rb-pkg/bookworm/amd64/diffoscope-results/ruby3.0.html > > > > /usr/lib/x86_64-linux-gnu/ruby/3.0.0/rbconfig.rb > > > > CONFIG["EGREP"]·=·"/bin/grep·-E" > > vs. > > CONFIG["EGREP"]·=·"/usr/bin/grep·-E" > > If these CONFIG variables are used for something at runtime, then this > will become a practical problem as soon as Debian starts using merged-/usr > buildds. The problem scenario is: > > - ruby3.0 is built on a merged-/usr buildd > - /usr/bin/grep is recorded in rbconfig.rb > - this build of ruby3.0 is installed on a non-merged-/usr system during > the upgrade from Debian 11 to Debian 12 > - whatever feature uses CONFIG["EGREP"] will not work, because > non-merged-/usr systems only have /bin/grep > > Technical Committee resolution > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=994388#110 recommends > that this class of bug is treated as release-critical, so I'm raising the > severity of this bug report. > > If none of the affected CONFIG variables are actually used for anything > on installed systems, then the severity of this bug can be downgraded > to non-RC (but it would be better to fix it anyway, because reproducible > builds are a useful goal for other reasons). Those variables are read from config.status during the builds. Maybe this should be fixed centrally in autoconf instead? > > Patch attached which passes variables to configure to use the > > non-usrmerge locations, as usrmerge installations typically have > > compatibility symlinks, but not vice-versa. > > To clarify: in Debian, merged-/usr installations are *guaranteed* to > have these compatibility symlinks. The patch looks appropriate to me, > although I have not tested it. Sure. signature.asc Description: PGP signature
Bug#1006471: ruby3.0: reproducible builds: embeds path to various binaries
Control: severity -1 serious On Fri, 25 Feb 2022 at 15:26:51 -0800, Vagrant Cascadian wrote: > The paths to various binaries, which differs on a usrmerge > vs. non-usrmerge system, are embedded in rbconfig.rb: > > > https://tests.reproducible-builds.org/debian/rb-pkg/bookworm/amd64/diffoscope-results/ruby3.0.html > > /usr/lib/x86_64-linux-gnu/ruby/3.0.0/rbconfig.rb > > CONFIG["EGREP"]·=·"/bin/grep·-E" > vs. > CONFIG["EGREP"]·=·"/usr/bin/grep·-E" If these CONFIG variables are used for something at runtime, then this will become a practical problem as soon as Debian starts using merged-/usr buildds. The problem scenario is: - ruby3.0 is built on a merged-/usr buildd - /usr/bin/grep is recorded in rbconfig.rb - this build of ruby3.0 is installed on a non-merged-/usr system during the upgrade from Debian 11 to Debian 12 - whatever feature uses CONFIG["EGREP"] will not work, because non-merged-/usr systems only have /bin/grep Technical Committee resolution https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=994388#110 recommends that this class of bug is treated as release-critical, so I'm raising the severity of this bug report. If none of the affected CONFIG variables are actually used for anything on installed systems, then the severity of this bug can be downgraded to non-RC (but it would be better to fix it anyway, because reproducible builds are a useful goal for other reasons). > Patch attached which passes variables to configure to use the > non-usrmerge locations, as usrmerge installations typically have > compatibility symlinks, but not vice-versa. To clarify: in Debian, merged-/usr installations are *guaranteed* to have these compatibility symlinks. The patch looks appropriate to me, although I have not tested it. smcv
Bug#1006471: ruby3.0: reproducible builds: embeds path to various binaries
Source: ruby3.0 Severity: normal Tags: patch User: reproducible-bui...@lists.alioth.debian.org Usertags: usrmerge shell X-Debbugs-Cc: reproducible-b...@lists.alioth.debian.org The paths to various binaries, which differs on a usrmerge vs. non-usrmerge system, are embedded in rbconfig.rb: https://tests.reproducible-builds.org/debian/rb-pkg/bookworm/amd64/diffoscope-results/ruby3.0.html /usr/lib/x86_64-linux-gnu/ruby/3.0.0/rbconfig.rb CONFIG["EGREP"]·=·"/bin/grep·-E" vs. CONFIG["EGREP"]·=·"/usr/bin/grep·-E" Patch attached which passes variables to configure to use the non-usrmerge locations, as usrmerge installations typically have compatibility symlinks, but not vice-versa. The patch also sets variables to ensure consistent values for bash, which can be triggered when /bin/sh points to bash. This patch alone does not fix all reproducibility issues (e.g. build paths on unstable and experimental), but should build reproducibly once it migrates to bookworm/testing! Thanks for maintaining ruby3.0! live well, vagrant From d0a1365cba685b8ab22be92463d28fb79d55a03b Mon Sep 17 00:00:00 2001 From: Vagrant Cascadian Date: Fri, 25 Feb 2022 23:17:08 + Subject: [PATCH] debian/rules: Pass variables to configure to make the package build reproducibly regardless of usrmerge. The variables EGREP, GREP, MAKEDIRS, MKDIR_P and SHELL should all point to their non-usrmerge locations. https://tests.reproducible-builds.org/debian/issues/paths_vary_due_to_usrmerge_issue.html --- debian/rules | 8 1 file changed, 8 insertions(+) diff --git a/debian/rules b/debian/rules index 76fa0b6..2d2c086 100755 --- a/debian/rules +++ b/debian/rules @@ -64,6 +64,14 @@ configure_options += --with-compress-debug-sections=no export DEB_BUILD_MAINT_OPTIONS = hardening=+bindnow optimize=-lto configure_options += $(shell dpkg-buildflags --export=configure) +# Pass variables to ensure consistent values when built on a usrmerge +# or non-usrmerge system. +configure_options += EGREP='/bin/grep -E' +configure_options += GREP='/bin/grep' +configure_options += MAKEDIRS='/bin/mkdir -p' +configure_options += MKDIR_P='/bin/mkdir -p' +configure_options += SHELL='/bin/sh' + # For more info see #999351 ifneq (,$(filter $(DEB_HOST_ARCH),alpha)) export DEB_CFLAGS_MAINT_APPEND += -O1 -- 2.30.2 signature.asc Description: PGP signature
