On Mon, 2021-11-22 at 14:12 +0100, Jacob Kroon wrote: > On 11/22/21 09:00, Jacob Kroon via lists.openembedded.org wrote: > > Hi, > > > > Some days ago there was a patch in OE-Core that updated the > > Upstream-Status tags in a couple of the gcc patches. I figured this > > wouldn't cause too much of rebuilding thanks to hashequiv, but it did, > > and all my target packages got rebuilt. > > > > It turns out that gcc embeds a checksum of its build artifacts, which > > includes .a archives, and those will most likely differ between > > gcc-cross builds since the timestamps of the .o files are included as > > meta information by default. There is a way to configure binutils in > > such a way that ar and ranlib defaults to being deterministic > > (--enable-deterministic-archives), but I don't think distros like Fedora > > do that. > > > > The target gcc reproduces because it is built with binutils-cross, and > > binutils-cross *is* configured with --enable-deterministic-archives. > > gcc-cross however is built with the host's binutils. > > > > I proposed a gcc patch > > (https://gcc.gnu.org/pipermail/gcc-patches/2021-November/585060.html) > > for building gcc with the 'D' flags passed to ar/ranlib, but it turns > > out it is not portable to all platforms. But it did mean that gcc-cross > > reproduced. > > > > Next thing I tried was to let gcc-cross depend on binutils-native, so I > > made sure the tools were not removed during do_install(), added the > > deterministic configure switch and added the dependency to gcc-cross, > > and again gcc-cross reproduced. However, RP raised some concerns about > > circular dependencies and increased build times. > > > > Also, it might be better if *all* cross/native recipes would use > > deterministic ar/ranlib, so next thing was to try and pass "-D" in > > BUILD_AR and BUILD_RANLIB. This causes problems with recipes that do > > "$(AR) cru ...". This becomes "ar D cru ..." which is a fatal error in > > ar, see > > https://sourceware.org/git/?p=binutils-gdb.git;a=blob;f=binutils/ar.c;h=8885585ef7537450f0f0990a5eeea7eb16bcad8f;hb=HEAD#l814. > > > > So next idea was to add wrappers around ar/ranlib (similar to what is > > done today for chown/chgrp for native recipes) that filters the options > > and makes them behave as if they were built with > > --enable-deterministic-archives. This also reproduces gcc-cross, but > > introducing more wrapper scripts is perhaps not so great. > > > > Here are the fragile wrappers I used: > > > > openembedded-core/scripts/cross-intercept/ar > > #!/bin/bash > > args=$(echo $1 | sed -e "s/u//g") > > shift > > PATH=$(echo $PATH | sed -e "s,$(dirname $(realpath -s $0)),,g") > > exec ar D $args $* > > > > openembedded-core/scripts/cross-intercept/ranlib > > #!/bin/bash > > PATH=$(echo $PATH | sed -e "s,$(dirname $(realpath -s $0)),,g") > > exec ranlib -D $* > > > > Thoughts anyone ? > > > > I've attached an oe-core patch that adds the wrappers above to native > and cross builds.
I think this would be something really useful to fix. For ranlib, we can probably just patch BUILD_RANLIB to add the flag. For ar, we need to remove the u option so it can only really be done with an intercept. I did try this on the autobuilder with your ar intercept and the ranlib flag tweak with: https://autobuilder.yoctoproject.org/typhoon/#/builders/15/builds/4671/steps/11/logs/stdio It broke as the above needs to be: exec ar D$args $* with no space else it breaks, at least on ubuntu1804. I'll try again with that fixed. As mentioned on irc, I am a little worried about the fork overhead in the wrapper too as fork overhead does seem to be a build pain point for us and the hit on a more complex program like python may be worth it, not sure. Cheers, Richard
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1355): https://lists.openembedded.org/g/openembedded-architecture/message/1355 Mute This Topic: https://lists.openembedded.org/mt/87230716/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-architecture/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
