when using ubuntu 18.04 host tcsh fails now see [1] gcc: error: unrecognized command line option ‘-fmacro-prefix-map=TOPDIR/build/tmp/work/core2-32-yoe-linux/tcsh/6.22.04-r0=/usr/src/debug/tcsh/6.22.04-r0’ Makefile:450: recipe for target 'gethost' failed make: *** [gethost] Error 1
[1] https://errors.yoctoproject.org/Errors/Details/600645/ On Mon, Aug 9, 2021 at 10:18 AM Khem Raj <[email protected]> wrote: > > > > On 8/9/21 7:39 AM, Tony Battersby wrote: > > On 8/6/21 10:19 PM, Khem Raj wrote: > >> I am seeing bunch of failures on meta-oe and meta-atmel > >> > >> https://errors.yoctoproject.org/Errors/Details/600064/ > >> https://errors.yoctoproject.org/Errors/Details/600065/ > >> https://errors.yoctoproject.org/Errors/Details/600071/ > >> > >> > > I sent patches for meta-openembedded, but I am not sure how to build > > meta-atmel: > > > > bitbake libegl > > ERROR: Layer atmel is not compatible with the core layer which only > > supports these series: honister (layer is compatible with hardknott) > > you can use my branch which has fixes for override problem > > https://github.com/YoeDistro/meta-atmel/tree/yoe/mut > > > > > > But the fix is probably the same as the ones I sent for the other > > packages (which I copied from > > meta-openembedded/meta-networking/recipes-protocols/net-snmp/net-snmp_5.9.1.bb). > > > > Tony Battersby > > > > > >> On Fri, Aug 6, 2021 at 2:04 PM Tony Battersby <[email protected]> > >> wrote: > >>> On 8/6/21 12:44 PM, Tony Battersby wrote: > >>>> On 8/6/21 1:33 AM, Richard Purdie wrote: > >>>>> On Thu, 2021-08-05 at 17:54 -0400, Tony Battersby wrote: > >>>>>> On 8/5/21 6:21 AM, Richard Purdie wrote: > >>>>>>> On Wed, 2021-08-04 at 20:13 -0400, Randy MacLeod wrote: > >>>>>>>> On 2021-07-27 9:34 a.m., Tony Battersby wrote: > >>>>>>>>> Currently, enabling Link Time Optimization via: > >>>>>>>>> > >>>>>>>>> require conf/distro/include/lto.inc > >>>>>>>>> DISTRO_FEATURES_append = " lto" > >>>>>>>>> > >>>>>>>>> makes the build non-reproducible (as in > >>>>>>>>> https://reproducible-builds.org/). This patch set fixes some of the > >>>>>>>>> problems, but is not yet a complete solution. > >>>>>>>>> > >>>>>>>>> This patch set addresses the following bugs: > >>>>>>>>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=14481 > >>>>>>>>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=14490 > >>>>>>>>> > >>>>>>>>> openembedded-core patches > >>>>>>>>> 1/4: lto.inc: disable LTO for grub > >>>>>>>>> 2/4: gcc: Backport patch to make LTO builds more reproducible > >>>>>>>>> 3/4: bitbake.conf: add DEBUG_PREFIX_MAP to TARGET_LDFLAGS > >>>>>>>>> > >>>>>>>>> openembedded-devel patches > >>>>>>>>> 4/4: net-snmp: fix QA Issue after LDFLAGS change > >>>>>>>>> > >>>>>>>>> Patch 3 adds DEBUG_PREFIX_MAP to TARGET_LDFLAGS (it was already in > >>>>>>>>> TARGET_CFLAGS). This could possibly be done differently, such as by > >>>>>>>>> removing it from TARGET_CFLAGS and adding it to TOOLCHAIN_OPTIONS, > >>>>>>>>> TARGET_CC_ARCH, or TUNE_CCARGS, but I haven't tested those. Let me > >>>>>>>>> know > >>>>>>>>> if there is a better/preferred way to do it. > >>>>>>>>> > >>>>>>>>> The TARGET_LDFLAGS change broke net-snmp from meta-networking, so I > >>>>>>>>> have > >>>>>>>>> included a patch to fix that. Other packages might have been > >>>>>>>>> broken as > >>>>>>>>> well, although I haven't found any others. > >>>>>>>>> > >>>>>>>>> With all these patches, there are still many shared libraries in > >>>>>>>>> /lib > >>>>>>>>> and /usr/lib that are still not reproducible with LTO enabled > >>>>>>>>> because > >>>>>>>>> x86_64-poky-linux-libtool filters out the -f*-prefix-map flags > >>>>>>>>> added by > >>>>>>>>> patch 3 when linking shared libraries. /usr/lib/libbz2.so.1.0.6 > >>>>>>>>> from > >>>>>>>>> the libbz2 package from the bzip2 recipe is a good example. I do > >>>>>>>>> not > >>>>>>>>> have a patch for that problem. I will leave that to someone else. > >>>>>>>>> > >>>>>>>>> So this is what we need to achieve reproducible LTO builds: > >>>>>>>>> *) This patch set > >>>>>>>>> *) A fix for libtool > >>>>>>>>> *) Add LTO to reproducible builds testing if appropriate (e.g. > >>>>>>>>> meta/lib/oeqa/selftest/cases/reproducible.py, CI on Autobuilder) > >>>>>>>>> *) Fixes for individual packages, if needed > >>>>>>>>> *) Once everything works, update the documentation again (I > >>>>>>>>> previously > >>>>>>>>> requested a documentation change for this bug, which will eventually > >>>>>>>>> show up at > >>>>>>>>> http://docs.yoctoproject.org/test-manual/reproducible-builds.html). > >>>>>>>> This is certainly a few steps in the right direction with a good > >>>>>>>> outline > >>>>>>>> of what's next. > >>>>>>>> > >>>>>>>> I suspect that Richard has been busy with other things like overrides > >>>>>>>> and the update to glibc-2.34. > >>>>>>>> > >>>>>>>> Do you have any suggestions or questions about what we should do > >>>>>>>> with this patch series Richard? > >>>>>>> Thanks for the reminder, I've been meaning to queue/test this which I > >>>>>>> will > >>>>>>> do. The gcc version has been bumped but I ported the patch forward to > >>>>>>> it > >>>>>>> and the patches are in master-next for testing. It is great to see > >>>>>>> this > >>>>>>> support moving forward. > >>>>>>> > >>>>>>> Much as it pains me to admit having any libtool knowledge, I may be > >>>>>>> able to > >>>>>>> help with that bit. There is an untested patch below which may > >>>>>>> address that > >>>>>>> issue, or at least give a good hint on how to do it if it doesn't > >>>>>>> work. I > >>>>>>> don't have an environment to test easily right now. > >>>>>>> > >>>>>>> Cheers, > >>>>>>> > >>>>>>> Richard > >>>>>> I tested your libtool patch in master-next with the bzip2 recipe. It > >>>>>> works as intended; libbz2.so.1.0.6 and all the other output files are > >>>>>> now reproducible with LTO enabled. I also verified that libtool is > >>>>>> passing the -f*-prefix-map flags to the link command in log.do_compile > >>>>>> as intended. > >>>>>> > >>>>>> I also verified that core-image-minimal builds successfully with the > >>>>>> libtool patch. > >>>>> Thanks for testing that, sounds like it helps that issue. I'll queue > >>>>> that patch. > >>>>> > >>>>> Unfortunately even with the ruby fix I queued, we still see > >>>>> reproduciblity > >>>>> issues with the LDFLAGS change in curl-dev, gettext-ptest, ruby and > >>>>> ruby-dbg > >>>>> packages: > >>>>> > >>>>> https://autobuilder.yoctoproject.org/typhoon/#/builders/115/builds/515/steps/12/logs/stdio > >>>>> > >>>>> We'll need to resolve those before I can merge the flags change. > >>>>> > >>>>> Diffoscope output: > >>>>> > >>>>> https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20210805-8jfd55ki/packages/diff-html/ > >>>>> > >>>>> Basically LDFLAGS get encoded into a couple of scripts and the ruby > >>>>> binary. > >>>>> > >>>>> Cheers, > >>>>> > >>>>> Richard > >>>>> > >>>> It looks like the ruby problem is being caused by MJIT_DLDFLAGS in > >>>> build/tmp/work/core2-64-poky-linux/ruby/3.0.2-r0/build/mjit_config.h. > >>>> There is already a patch > >>>> poky/meta/recipes-devtools/ruby/ruby/0001-template-Makefile.in-do-not-write-host-cross-cc-item.patch > >>>> that removes some of these items for the sake of reproducibiltiy. But I > >>>> do not know ruby, so I do not know the implications of doing the same > >>>> thing for MJIT_DLDFLAGS. > >>>> > >>>> Tony Battersby > >>>> > >>>> > >>>> > >>> Richard, > >>> > >>> I noticed that you already fixed the problems in curl and gettext in > >>> master-next. Here is a patch for ruby if you haven't fixed that already. > >>> > >>> Tony Battersby > >>> > >>> From 9cfb9777ff6ef95a27ed6a03719c807365d0f116 Mon Sep 17 00:00:00 2001 > >>> From: Tony Battersby <[email protected]> > >>> Date: Fri, 6 Aug 2021 16:47:14 -0400 > >>> Subject: [PATCH] ruby: Fix reproducibility issue with LDFLAGS > >>> > >>> If we add DEBUG_PREFIX_MAP into LDFLAGS, ruby and ruby-dbg are no > >>> longer reproducible. Fix this. > >>> > >>> Signed-off-by: Tony Battersby <[email protected]> > >>> --- > >>> ...-Makefile.in-filter-out-f-prefix-map.patch | 42 +++++++++++++++++++ > >>> meta/recipes-devtools/ruby/ruby_3.0.2.bb | 1 + > >>> 2 files changed, 43 insertions(+) > >>> create mode 100644 > >>> meta/recipes-devtools/ruby/ruby/0002-template-Makefile.in-filter-out-f-prefix-map.patch > >>> > >>> diff --git > >>> a/meta/recipes-devtools/ruby/ruby/0002-template-Makefile.in-filter-out-f-prefix-map.patch > >>> > >>> b/meta/recipes-devtools/ruby/ruby/0002-template-Makefile.in-filter-out-f-prefix-map.patch > >>> new file mode 100644 > >>> index 0000000000..9387506c26 > >>> --- /dev/null > >>> +++ > >>> b/meta/recipes-devtools/ruby/ruby/0002-template-Makefile.in-filter-out-f-prefix-map.patch > >>> @@ -0,0 +1,42 @@ > >>> +Subject: [PATCH] template/Makefile.in: filter out -f*prefix-map > >>> + > >>> +If we add DEBUG_PREFIX_MAP into LDFLAGS, ruby and ruby-dbg are no longer > >>> +reproducible. Fix this. > >>> + > >>> +Upstream-Status: Inapproppriate [oe-core specific] > >>> +Signed-off-by: Tony Battersby <[email protected]> > >>> +--- > >>> +--- a/tool/mjit_archflag.sh > >>> ++++ b/tool/mjit_archflag.sh > >>> +@@ -7,6 +7,20 @@ quote() { > >>> + echo > >>> + } > >>> + > >>> ++quote_filtered() { > >>> ++ printf "#${indent}define $1" > >>> ++ while shift && [ "$#" -gt 0 ]; do > >>> ++ case "$1" in > >>> ++ -ffile-prefix-map=*|-fdebug-prefix-map=*|-fmacro-prefix-map=*) > >>> ++ ;; > >>> ++ *) > >>> ++ printf ' "%s"'$sep "$1" > >>> ++ ;; > >>> ++ esac > >>> ++ done > >>> ++ echo > >>> ++} > >>> ++ > >>> + archs="" > >>> + arch_flag="" > >>> + > >>> +--- a/template/Makefile.in > >>> ++++ b/template/Makefile.in > >>> +@@ -666,7 +666,7 @@ mjit_config.h: > >>> + quote "MJIT_OPTFLAGS " $(MJIT_OPTFLAGS); \ > >>> + quote "MJIT_DEBUGFLAGS " $(MJIT_DEBUGFLAGS); \ > >>> + quote "MJIT_LDSHARED " ; \ > >>> +- quote "MJIT_DLDFLAGS MJIT_ARCHFLAG" $(MJIT_DLDFLAGS); \ > >>> ++ quote_filtered "MJIT_DLDFLAGS MJIT_ARCHFLAG" $(MJIT_DLDFLAGS); > >>> \ > >>> + quote "MJIT_LIBS " $(LIBRUBYARG_SHARED); \ > >>> + quote 'PRELOADENV "@PRELOADENV@"'; \ > >>> + indent=$${archs:+' '}; \ > >>> diff --git a/meta/recipes-devtools/ruby/ruby_3.0.2.bb > >>> b/meta/recipes-devtools/ruby/ruby_3.0.2.bb > >>> index 0b9c342def..38e594a59e 100644 > >>> --- a/meta/recipes-devtools/ruby/ruby_3.0.2.bb > >>> +++ b/meta/recipes-devtools/ruby/ruby_3.0.2.bb > >>> @@ -6,6 +6,7 @@ SRC_URI += " \ > >>> file://remove_has_include_macros.patch \ > >>> file://run-ptest \ > >>> > >>> file://0001-template-Makefile.in-do-not-write-host-cross-cc-item.patch \ > >>> + > >>> file://0002-template-Makefile.in-filter-out-f-prefix-map.patch \ > >>> " > >>> > >>> SRC_URI[sha256sum] = > >>> "5085dee0ad9f06996a8acec7ebea4a8735e6fac22f22e2d98c3f2bc3bef7e6f1" > >>> -- > >>> 2.25.1 > >>> > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#154691): https://lists.openembedded.org/g/openembedded-core/message/154691 Mute This Topic: https://lists.openembedded.org/mt/84481652/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
