Bug#1033663: linux: reproducible-builds: Embedded build path in various binaries

2023-07-14 Thread Vagrant Cascadian 
On 2023-07-14, Ben Hutchings wrote:
> On Wed, 2023-03-29 at 09:58 -0700, Vagrant Cascadian wrote:
>> The build path is embedded in various binaries:
>> 
>>   
>> https://tests.reproducible-builds.org/debian/rb-pkg/unstable/i386/diffoscope-results/linux.html
>> 
>>   /usr/sbin/bpftool
>> 
>>   /build/1st/linux-6.1.20/tools/lib/bpf/libbpf.c:805
>>   vs.
>>   /build/2/linux-6.1.20/2nd/tools/lib/bpf/libbpf.c:805  
>> 
>> This *seems* like a regression from earlier versions, though I have not
>> tracked down when the tools started embedding the build paths...
> [...]
>
> Please see
> .
>
> The embedded paths are mostly fixed by:
> .
>
> Perhaps I should finalise that MR even though it doesn't fix
> everything?

Very glad to see that merge request, thanks!

With my reproducible builds hat on, I would welcome partial fixes at
this point, especially for such core packages! It helps to troubleshoot
outstanding issues and also avoids duplication of work.

live well,
  vagrant


signature.asc
Description: PGP signature

Bug#1033663: linux: reproducible-builds: Embedded build path in various binaries

2023-07-14 Thread Ben Hutchings 
On Wed, 2023-03-29 at 09:58 -0700, Vagrant Cascadian wrote:
> Source: linux
> Severity: normal
> Tags: patch
> User: reproducible-bui...@lists.alioth.debian.org
> Usertags: buildpath
> X-Debbugs-Cc: reproducible-b...@lists.alioth.debian.org
> 
> The build path is embedded in various binaries:
> 
>   
> https://tests.reproducible-builds.org/debian/rb-pkg/unstable/i386/diffoscope-results/linux.html
> 
>   /usr/sbin/bpftool
> 
>   /build/1st/linux-6.1.20/tools/lib/bpf/libbpf.c:805
>   vs.
>   /build/2/linux-6.1.20/2nd/tools/lib/bpf/libbpf.c:805  
> 
> This *seems* like a regression from earlier versions, though I have not
> tracked down when the tools started embedding the build paths...
[...]

Please see
.

The embedded paths are mostly fixed by:
.

Perhaps I should finalise that MR even though it doesn't fix
everything?

Ben.

-- 
Ben Hutchings
Hoare's Law of Large Problems:
   Inside every large problem is a small problem struggling to get out.



signature.asc
Description: This is a digitally signed message part

Bug#1033663: linux: reproducible-builds: Embedded build path in various binaries

2023-07-14 Thread Bastian Blank 
On Thu, Jul 13, 2023 at 11:53:40PM -0700, Vagrant Cascadian wrote:
> Any objections to me merging these patches? Prefer if they remain
> separate or squash them into a single patch?

Hmm, I have to check, but dpkg-buildflags should already provide this
flag and all the modified locations are for userspace tooling.

Maybe you could look into that as well.

Regards,
Bastian

-- 
It is a human characteristic to love little animals, especially if
they're attractive in some way.
-- McCoy, "The Trouble with Tribbles", stardate 4525.6

Bug#1033663: linux: reproducible-builds: Embedded build path in various binaries

2023-07-14 Thread Vagrant Cascadian 
On 2023-03-29, Vagrant Cascadian wrote:
> The build path is embedded in various binaries:
>
>   
> https://tests.reproducible-builds.org/debian/rb-pkg/unstable/i386/diffoscope-results/linux.html
>
>   /usr/sbin/bpftool
>
>   /build/1st/linux-6.1.20/tools/lib/bpf/libbpf.c:805
>   vs.
>   /build/2/linux-6.1.20/2nd/tools/lib/bpf/libbpf.c:805  
>
> This *seems* like a regression from earlier versions, though I have not
> tracked down when the tools started embedding the build paths...
>
> The attached patches fix this for some of the binary packages and
> corresponding dbgsym packages (bpftool, hyperv-daemons, *cpupower*) by
> explicitly passing the -ffile-prefix-map argument (to avoid embedding
> the build path) in the CFLAGS or EXTRA_CFLAGS variables in
> debian/rules.d/.

Any objections to me merging these patches? Prefer if they remain
separate or squash them into a single patch?


> From 7dcacfb3415c5ec4caea8933dd2c12fb401bea52 Mon Sep 17 00:00:00 2001
> From: Vagrant Cascadian 
> Date: Tue, 28 Mar 2023 14:49:01 -0700
> Subject: [PATCH 1/4] debian/rules.d/tools/bpf/bpftool/Makefile: Pass
>  -ffile-prefix-map via EXTRA_CFLAGS.
>
> ---
>  debian/rules.d/tools/bpf/bpftool/Makefile | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/debian/rules.d/tools/bpf/bpftool/Makefile 
> b/debian/rules.d/tools/bpf/bpftool/Makefile
> index 17c27c104..9d3017e42 100644
> --- a/debian/rules.d/tools/bpf/bpftool/Makefile
> +++ b/debian/rules.d/tools/bpf/bpftool/Makefile
> @@ -5,7 +5,7 @@ MAKE_BPFTOOL += prefix=/usr
>  MAKE_BPFTOOL += mandir=/usr/share/man
>  MAKE_BPFTOOL += V=1
>  MAKE_BPFTOOL += ARCH=$(KERNEL_ARCH)
> -MAKE_BPFTOOL += EXTRA_CFLAGS='$(CFLAGS) $(CPPFLAGS)'
> +MAKE_BPFTOOL += EXTRA_CFLAGS='$(CFLAGS) $(CPPFLAGS) 
> -ffile-prefix-map=$(top_srcdir)=.'
>  MAKE_BPFTOOL += EXTRA_LDFLAGS='$(LDFLAGS)'
>  
>  # dynamically linking with libbfd is not allowed in Debian
> -- 
> 2.39.2
>
> From cb11c490a41a0afbdeb00f272fa8a71cc3c69be1 Mon Sep 17 00:00:00 2001
> From: Vagrant Cascadian 
> Date: Tue, 28 Mar 2023 15:48:05 -0700
> Subject: [PATCH 2/4] debian/rules.d/tools/power/cpupower/Makefile: Pass
>  -ffile-prefix-map via MAKE_CPUPOWER CFLAGS.
>
> ---
>  debian/rules.d/tools/power/cpupower/Makefile | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/debian/rules.d/tools/power/cpupower/Makefile 
> b/debian/rules.d/tools/power/cpupower/Makefile
> index e4bd5202d..28ccf7004 100644
> --- a/debian/rules.d/tools/power/cpupower/Makefile
> +++ b/debian/rules.d/tools/power/cpupower/Makefile
> @@ -1,6 +1,6 @@
>  include $(top_rulesdir)/Makefile.inc
>  
> -MAKE_CPUPOWER := CFLAGS='$(CFLAGS) $(CPPFLAGS)' LDFLAGS='$(LDFLAGS)' $(MAKE) 
> O=$(CURDIR) CPUFREQ_BENCH=false V=true mandir=/usr/share/man
> +MAKE_CPUPOWER := CFLAGS='$(CFLAGS) -ffile-prefix-map=$(top_srcdir)=. 
> $(CPPFLAGS)' LDFLAGS='$(LDFLAGS)' $(MAKE) O=$(CURDIR) CPUFREQ_BENCH=false 
> V=true mandir=/usr/share/man
>  
>  MAKE_CPUPOWER += DEBUG=$(if $(filter noopt,$(DEB_BUILD_OPTIONS)),true,)
>  
> -- 
> 2.39.2
>
> From 2cf67982526879524548e37df0efac5f8b37e0d3 Mon Sep 17 00:00:00 2001
> From: Vagrant Cascadian 
> Date: Tue, 28 Mar 2023 16:57:01 -0700
> Subject: [PATCH 3/4] debian/rules.d/tools/power/x86/*/Makefile: Add
>  -ffile-prefix-map to CFLAGS.
>
> ---
>  debian/rules.d/tools/power/x86/turbostat/Makefile  | 2 ++
>  debian/rules.d/tools/power/x86/x86_energy_perf_policy/Makefile | 2 ++
>  2 files changed, 4 insertions(+)
>
> diff --git a/debian/rules.d/tools/power/x86/turbostat/Makefile 
> b/debian/rules.d/tools/power/x86/turbostat/Makefile
> index eb5124d3a..019fcbde6 100644
> --- a/debian/rules.d/tools/power/x86/turbostat/Makefile
> +++ b/debian/rules.d/tools/power/x86/turbostat/Makefile
> @@ -7,3 +7,5 @@ include $(top_rulesdir)/Makefile.inc
>  CPPFLAGS += -I"$(top_srcdir)/tools/include" 
> -DMSRHEADER='"$(top_srcdir)/arch/x86/include/asm/msr-index.h"' 
> -DINTEL_FAMILY_HEADER='"$(top_srcdir)/arch/x86/include/asm/intel-family.h"'
>  
>  LDLIBS += -lcap -lrt
> +
> +CFLAGS += -ffile-prefix-map=$(top_srcdir)=.
> diff --git a/debian/rules.d/tools/power/x86/x86_energy_perf_policy/Makefile 
> b/debian/rules.d/tools/power/x86/x86_energy_perf_policy/Makefile
> index b9ec56c89..a86da0f1e 100644
> --- a/debian/rules.d/tools/power/x86/x86_energy_perf_policy/Makefile
> +++ b/debian/rules.d/tools/power/x86/x86_energy_perf_policy/Makefile
> @@ -5,3 +5,5 @@ installdir = /usr/sbin
>  include $(top_rulesdir)/Makefile.inc
>  
>  CPPFLAGS += -I"$(top_srcdir)/tools/include" 
> -DMSRHEADER='"$(top_srcdir)/arch/x86/include/asm/msr-index.h"'
> +
> +CFLAGS += -ffile-prefix-map=$(top_srcdir)=.
> -- 
> 2.39.2
>
> From 36e49dee5adf4ba4b566cb87bea35dd0158f8ff3 Mon Sep 17 00:00:00 2001
> From: Vagrant Cascadian 
> Date: Tue, 28 Mar 2023 21:44:05 -0700
> Subject: [PATCH 4/4] debian/rules.d/tools/hv/Makefile: Add -ffile-prefix-map
>  to CFLAGS.
>
> ---
>  debian/rules.d/tools/hv/Makefile | 2 ++
>  1 file changed, 2 insertions(+)
>
> diff --git

Bug#1033663: linux: reproducible-builds: Embedded build path in various binaries

2023-03-29 Thread Vagrant Cascadian 
Source: linux
Severity: normal
Tags: patch
User: reproducible-bui...@lists.alioth.debian.org
Usertags: buildpath
X-Debbugs-Cc: reproducible-b...@lists.alioth.debian.org

The build path is embedded in various binaries:

  
https://tests.reproducible-builds.org/debian/rb-pkg/unstable/i386/diffoscope-results/linux.html

  /usr/sbin/bpftool

  /build/1st/linux-6.1.20/tools/lib/bpf/libbpf.c:805
  vs.
  /build/2/linux-6.1.20/2nd/tools/lib/bpf/libbpf.c:805  

This *seems* like a regression from earlier versions, though I have not
tracked down when the tools started embedding the build paths...

The attached patches fix this for some of the binary packages and
corresponding dbgsym packages (bpftool, hyperv-daemons, *cpupower*) by
explicitly passing the -ffile-prefix-map argument (to avoid embedding
the build path) in the CFLAGS or EXTRA_CFLAGS variables in
debian/rules.d/.

I unsuccessfully tried applying similar patches for other tools packages
(rtla, perf, usbip) but was not able to convince the tools to take the
passed flags...

It would, of course, be better to figure out a way to get the various
tools to respect the default CFLAGS, as -ffile-prefix-map is included in
the default CFLAGS.

Unfortunately, these patches do not fix all reproducibility issues, but
applying these patches could reduce some of the noise and make it easier
to debug the remaining issues.

Thanks for maintaining linux!

live well,
  vagrant
From 7dcacfb3415c5ec4caea8933dd2c12fb401bea52 Mon Sep 17 00:00:00 2001
From: Vagrant Cascadian 
Date: Tue, 28 Mar 2023 14:49:01 -0700
Subject: [PATCH 1/4] debian/rules.d/tools/bpf/bpftool/Makefile: Pass
 -ffile-prefix-map via EXTRA_CFLAGS.

---
 debian/rules.d/tools/bpf/bpftool/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/debian/rules.d/tools/bpf/bpftool/Makefile b/debian/rules.d/tools/bpf/bpftool/Makefile
index 17c27c104..9d3017e42 100644
--- a/debian/rules.d/tools/bpf/bpftool/Makefile
+++ b/debian/rules.d/tools/bpf/bpftool/Makefile
@@ -5,7 +5,7 @@ MAKE_BPFTOOL += prefix=/usr
 MAKE_BPFTOOL += mandir=/usr/share/man
 MAKE_BPFTOOL += V=1
 MAKE_BPFTOOL += ARCH=$(KERNEL_ARCH)
-MAKE_BPFTOOL += EXTRA_CFLAGS='$(CFLAGS) $(CPPFLAGS)'
+MAKE_BPFTOOL += EXTRA_CFLAGS='$(CFLAGS) $(CPPFLAGS) -ffile-prefix-map=$(top_srcdir)=.'
 MAKE_BPFTOOL += EXTRA_LDFLAGS='$(LDFLAGS)'
 
 # dynamically linking with libbfd is not allowed in Debian
-- 
2.39.2

From cb11c490a41a0afbdeb00f272fa8a71cc3c69be1 Mon Sep 17 00:00:00 2001
From: Vagrant Cascadian 
Date: Tue, 28 Mar 2023 15:48:05 -0700
Subject: [PATCH 2/4] debian/rules.d/tools/power/cpupower/Makefile: Pass
 -ffile-prefix-map via MAKE_CPUPOWER CFLAGS.

---
 debian/rules.d/tools/power/cpupower/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/debian/rules.d/tools/power/cpupower/Makefile b/debian/rules.d/tools/power/cpupower/Makefile
index e4bd5202d..28ccf7004 100644
--- a/debian/rules.d/tools/power/cpupower/Makefile
+++ b/debian/rules.d/tools/power/cpupower/Makefile
@@ -1,6 +1,6 @@
 include $(top_rulesdir)/Makefile.inc
 
-MAKE_CPUPOWER := CFLAGS='$(CFLAGS) $(CPPFLAGS)' LDFLAGS='$(LDFLAGS)' $(MAKE) O=$(CURDIR) CPUFREQ_BENCH=false V=true mandir=/usr/share/man
+MAKE_CPUPOWER := CFLAGS='$(CFLAGS) -ffile-prefix-map=$(top_srcdir)=. $(CPPFLAGS)' LDFLAGS='$(LDFLAGS)' $(MAKE) O=$(CURDIR) CPUFREQ_BENCH=false V=true mandir=/usr/share/man
 
 MAKE_CPUPOWER += DEBUG=$(if $(filter noopt,$(DEB_BUILD_OPTIONS)),true,)
 
-- 
2.39.2

From 2cf67982526879524548e37df0efac5f8b37e0d3 Mon Sep 17 00:00:00 2001
From: Vagrant Cascadian 
Date: Tue, 28 Mar 2023 16:57:01 -0700
Subject: [PATCH 3/4] debian/rules.d/tools/power/x86/*/Makefile: Add
 -ffile-prefix-map to CFLAGS.

---
 debian/rules.d/tools/power/x86/turbostat/Makefile  | 2 ++
 debian/rules.d/tools/power/x86/x86_energy_perf_policy/Makefile | 2 ++
 2 files changed, 4 insertions(+)

diff --git a/debian/rules.d/tools/power/x86/turbostat/Makefile b/debian/rules.d/tools/power/x86/turbostat/Makefile
index eb5124d3a..019fcbde6 100644
--- a/debian/rules.d/tools/power/x86/turbostat/Makefile
+++ b/debian/rules.d/tools/power/x86/turbostat/Makefile
@@ -7,3 +7,5 @@ include $(top_rulesdir)/Makefile.inc
 CPPFLAGS += -I"$(top_srcdir)/tools/include" -DMSRHEADER='"$(top_srcdir)/arch/x86/include/asm/msr-index.h"' -DINTEL_FAMILY_HEADER='"$(top_srcdir)/arch/x86/include/asm/intel-family.h"'
 
 LDLIBS += -lcap -lrt
+
+CFLAGS += -ffile-prefix-map=$(top_srcdir)=.
diff --git a/debian/rules.d/tools/power/x86/x86_energy_perf_policy/Makefile b/debian/rules.d/tools/power/x86/x86_energy_perf_policy/Makefile
index b9ec56c89..a86da0f1e 100644
--- a/debian/rules.d/tools/power/x86/x86_energy_perf_policy/Makefile
+++ b/debian/rules.d/tools/power/x86/x86_energy_perf_policy/Makefile
@@ -5,3 +5,5 @@ installdir = /usr/sbin
 include $(top_rulesdir)/Makefile.inc
 
 CPPFLAGS += -I"$(top_srcdir)/tools/include" -DMSRHEADER='"$(top_srcdir)/arch/x86/include/asm/msr-index.h"'
+
+CFLAGS += -ffile-prefix-map=$(top_srcdir)=.
--