On 8/5/21 6:21 AM, Richard Purdie wrote:
> On Wed, 2021-08-04 at 20:13 -0400, Randy MacLeod wrote:
>> On 2021-07-27 9:34 a.m., Tony Battersby wrote:
>>> Currently, enabling Link Time Optimization via:
>>>
>>> require conf/distro/include/lto.inc
>>> DISTRO_FEATURES_append = " lto"
>>>
>>> makes the build non-reproducible (as in
>>> https://reproducible-builds.org/). This patch set fixes some of the
>>> problems, but is not yet a complete solution.
>>>
>>> This patch set addresses the following bugs:
>>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=14481
>>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=14490
>>>
>>> openembedded-core patches
>>> 1/4: lto.inc: disable LTO for grub
>>> 2/4: gcc: Backport patch to make LTO builds more reproducible
>>> 3/4: bitbake.conf: add DEBUG_PREFIX_MAP to TARGET_LDFLAGS
>>>
>>> openembedded-devel patches
>>> 4/4: net-snmp: fix QA Issue after LDFLAGS change
>>>
>>> Patch 3 adds DEBUG_PREFIX_MAP to TARGET_LDFLAGS (it was already in
>>> TARGET_CFLAGS). This could possibly be done differently, such as by
>>> removing it from TARGET_CFLAGS and adding it to TOOLCHAIN_OPTIONS,
>>> TARGET_CC_ARCH, or TUNE_CCARGS, but I haven't tested those. Let me know
>>> if there is a better/preferred way to do it.
>>>
>>> The TARGET_LDFLAGS change broke net-snmp from meta-networking, so I have
>>> included a patch to fix that. Other packages might have been broken as
>>> well, although I haven't found any others.
>>>
>>> With all these patches, there are still many shared libraries in /lib
>>> and /usr/lib that are still not reproducible with LTO enabled because
>>> x86_64-poky-linux-libtool filters out the -f*-prefix-map flags added by
>>> patch 3 when linking shared libraries. /usr/lib/libbz2.so.1.0.6 from
>>> the libbz2 package from the bzip2 recipe is a good example. I do not
>>> have a patch for that problem. I will leave that to someone else.
>>>
>>> So this is what we need to achieve reproducible LTO builds:
>>> *) This patch set
>>> *) A fix for libtool
>>> *) Add LTO to reproducible builds testing if appropriate (e.g.
>>> meta/lib/oeqa/selftest/cases/reproducible.py, CI on Autobuilder)
>>> *) Fixes for individual packages, if needed
>>> *) Once everything works, update the documentation again (I previously
>>> requested a documentation change for this bug, which will eventually
>>> show up at
>>> http://docs.yoctoproject.org/test-manual/reproducible-builds.html).
>> This is certainly a few steps in the right direction with a good outline
>> of what's next.
>>
>> I suspect that Richard has been busy with other things like overrides
>> and the update to glibc-2.34.
>>
>> Do you have any suggestions or questions about what we should do
>> with this patch series Richard?
> Thanks for the reminder, I've been meaning to queue/test this which I will
> do. The gcc version has been bumped but I ported the patch forward to it
> and the patches are in master-next for testing. It is great to see this
> support moving forward.
>
> Much as it pains me to admit having any libtool knowledge, I may be able to
> help with that bit. There is an untested patch below which may address that
> issue, or at least give a good hint on how to do it if it doesn't work. I
> don't have an environment to test easily right now.
>
> Cheers,
>
> Richard
I tested your libtool patch in master-next with the bzip2 recipe. It
works as intended; libbz2.so.1.0.6 and all the other output files are
now reproducible with LTO enabled. I also verified that libtool is
passing the -f*-prefix-map flags to the link command in log.do_compile
as intended.
I also verified that core-image-minimal builds successfully with the
libtool patch.
Tony Battersby
>
> >From 7681b0a1461f22034e1f70328a060db34474c60f Mon Sep 17 00:00:00 2001
> From: Richard Purdie <richard.pur...@linuxfoundation.org>
> Date: Thu, 5 Aug 2021 11:18:20 +0100
> Subject: libtool: Fix lto option passing?
>
> Signed-off-by: Richard Purdie <richard.pur...@linuxfoundation.org>
> ---
> meta/recipes-devtools/libtool/libtool-2.4.6.inc | 1 +
> meta/recipes-devtools/libtool/libtool/lto-prefix.patch | 16 ++++++++++++++++
> 2 files changed, 17 insertions(+)
> create mode 100644 meta/recipes-devtools/libtool/libtool/lto-prefix.patch
>
> diff --git a/meta/recipes-devtools/libtool/libtool-2.4.6.inc
> b/meta/recipes-devtools/libtool/libtool-2.4.6.inc
> index c1cbceb51f..6748d7468e 100644
> --- a/meta/recipes-devtools/libtool/libtool-2.4.6.inc
> +++ b/meta/recipes-devtools/libtool/libtool-2.4.6.inc
> @@ -23,6 +23,7 @@ SRC_URI = "${GNU_MIRROR}/libtool/libtool-${PV}.tar.gz \
>
> file://0001-libtool-Check-for-static-libs-for-internal-compiler-.patch \
>
> file://0001-Makefile.am-make-sure-autoheader-run-before-autoconf.patch \
>
> file://0001-Makefile.am-make-sure-autoheader-run-before-automake.patch \
> + file://lto-prefix.patch \
> "
>
> SRC_URI[md5sum] = "addf44b646ddb4e3919805aa88fa7c5e"
> diff --git a/meta/recipes-devtools/libtool/libtool/lto-prefix.patch
> b/meta/recipes-
> devtools/libtool/libtool/lto-prefix.patch
> new file mode 100644
> index 0000000000..f788b3fc27
> --- /dev/null
> +++ b/meta/recipes-devtools/libtool/libtool/lto-prefix.patch
> @@ -0,0 +1,16 @@
> +Index: libtool-2.4.6/build-aux/ltmain.in
> +===================================================================
> +--- libtool-2.4.6.orig/build-aux/ltmain.in
> ++++ libtool-2.4.6/build-aux/ltmain.in
> +@@ -5424,9 +5424,10 @@ func_mode_link ()
> + # --sysroot=* for sysroot support
> + # -O*, -g*, -flto*, -fwhopr*, -fuse-linker-plugin GCC link-time
> optimization
> + # -stdlib=* select c++ std lib with clang
> ++ # -f*-prefix-map* needed for lto linking
> + -64|-mips[0-9]|-r[0-9][0-9]*|-xarch=*|-xtarget=*|+DA*|+DD*|-q*|-m*| \
> +
> -t[45]*|-txscale*|-p|-pg|--coverage|-fprofile-*|-F*|@*|-tp=*|--sysroot=*| \
> +-
> -O*|-g*|-flto*|-fwhopr*|-fuse-linker-plugin|-fstack-protector*|-stdlib=*)
> ++
> -O*|-g*|-flto*|-fwhopr*|-fuse-linker-plugin|-fstack-protector*|-stdlib=*|-f*-prefix-map*)
> + func_quote_for_eval "$arg"
> + arg=$func_quote_for_eval_result
> + func_append compile_command " $arg"
>
>
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154525):
https://lists.openembedded.org/g/openembedded-core/message/154525
Mute This Topic: https://lists.openembedded.org/mt/84481652/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
