When building the Linux kernel, the linker generates a hash of all versions of tools involved in a build called BuildID in ELF header. This breaks reproducibility accross different buildhosts eventhough OpenWrt builds the toolchain from source -- the build-id hash ends up to be the only thing which differs in the resulting builds.
The cause is most likely a result of the build hosts' architectures, OSs and standard C libraries being different. While in theory it is true that tools may produce a different output depending on archtecture, OS and libc of the buildhost, in practice this is (fortunately) hardly ever the case and hence it contradicts ld(1) which states: 'The "md5" and "sha1" styles produces an identifier that is always the same in an identical output file, but will be unique among all nonidentical output files.' (the kernel is using sha1 style build-id, rebuilding the kernel on a different buildhost results in everything being identical **except** for the build-id) Hence, to achieve reproducible builds we will either have to resort to identical containers/VMs for building or get rid of the BuildID hash alltogether (or use a different build-id style) At this point, this seems to be what Debian is doing as well in order to achieve reproducible kernel builds, see[1]. [1]: https://wiki.debian.org/SameKernel#How_this_works Signed-off-by: Daniel Golle <dan...@makrotopia.org> --- To investigate the issue of non-reproducible kernel images accross buildhosts I compated the files build by OpenWrt's buildbots with Paul's rebuilder script running on his (Aarch64) Mac. The resulting bzImage binaries were compared by first using scripts/extract-vmlinux to extract the ELF contents from bzImages and then compared using dffoscope: Format-specific differences are supported for ELF binaries but no file-specific differences were detected; falling back to a binary diff. @@ -1254946,16 +1254946,16 @@ 01326210: 0400 0000 0800 0000 0c00 0000 5865 6e00 ............Xen. 01326220: 0000 0000 0080 ffff 0400 0000 0800 0000 ................ 01326230: 0400 0000 5865 6e00 0000 0000 0000 0000 ....Xen......... 01326240: 0400 0000 2000 0000 0500 0000 474e 5500 .... .......GNU. 01326250: 0100 01c0 0400 0000 df00 0000 0000 0000 ................ 01326260: 0200 01c0 0400 0000 0700 0000 0000 0000 ................ 01326270: 0400 0000 1400 0000 0300 0000 474e 5500 ............GNU. -01326280: b737 4be0 66fd dc7f 9fc8 24b6 6354 d8f9 .7K.f.....$.cT.. -01326290: a42c b698 0600 0000 0100 0000 0001 0000 .,.............. +01326280: 2008 eb34 e442 f784 6573 d75b 9bd2 b23f ..4.B..es.[...? +01326290: 536e a33e 0600 0000 0100 0000 0001 0000 Sn.>............ 013262a0: 4c69 6e75 7800 0000 0000 0000 0400 0000 Linux........... 013262b0: 0800 0000 1200 0000 5865 6e00 b004 0001 ........Xen..... 013262c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 013262d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 013262e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 013262f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 01326300: 0000 0000 0000 0000 0000 0000 0000 0000 ................ (all the remaining ELF is bit-by-bit identical) Using file(1) revealed that the difference is exactly the build-id: linux1.elf: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, BuildID[sha1]=b7374be066fddc7f9fc824b66354d8f9a42cb698, stripped linux2.elf: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, BuildID[sha1]=2008eb34e442f7846573d75b9bd2b23f536ea33e, stripped include/kernel-defaults.mk | 1 + 1 file changed, 1 insertion(+) diff --git a/include/kernel-defaults.mk b/include/kernel-defaults.mk index 1e82f7d739..9c8d5fbe97 100644 --- a/include/kernel-defaults.mk +++ b/include/kernel-defaults.mk @@ -46,6 +46,7 @@ else if [ -d $(LINUX_DIR)/user_headers ]; then \ rm -rf $(LINUX_DIR)/user_headers; \ fi + $(SED) -i $(LINUX_DIR)/Makefile -e 's/--build-id=.*/--build-id=none/g' endef endif -- 2.35.1 _______________________________________________ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
