On Mon, 2022-08-29 at 13:09 +0200, Peter Bergin wrote:
> Hi Richard,
>
> (also adding Khem Raj to CC as maintainer for gcc in oe-core)
>
> On 2022-08-25 17:25, Richard Purdie wrote:
> > On Thu, 2022-08-25 at 14:03 +0200, Peter Bergin wrote:
> > > Hi Richard,
> > >
> > > On 2022-08-25 10:21, Richard Purdie wrote:
> > > > <snip>
> > > > > I've tried locally to reproduce something. I've built and tested
> > > > > genericx86 and qemuarm64 now on core-image-sato sdk that I saw was the
> > > > > target for the autobuilder. Both tests passes. The failure I see in
> > > > > the
> > > > > autobuilder logs is that the build script can not be executed. On my
> > > > > machine I have that file and it can clearly be executed:
> > > > >
> > > > > $ find
> > > > > tmp/work/genericx86-poky-linux/core-image-sato/1.0-r0/testimage-sdk/hello/target/
> > > > > -name build-script-build
> > > > > tmp/work/genericx86-poky-linux/core-image-sato/1.0-r0/testimage-sdk/hello/target/debug/build/hello-4dbf26d86b93a892/build-script-build
> > > > >
> > > > > $
> > > > > tmp/work/genericx86-poky-linux/core-image-sato/1.0-r0/testimage-sdk/hello/target/debug/build/hello-4dbf26d86b93a892/build-script-build
> > > > >
> > > > > $ file
> > > > > tmp/work/genericx86-poky-linux/core-image-sato/1.0-r0/testimage-sdk/hello/target/debug/build/hello-4dbf26d86b93a892/build-script-build
> > > > >
> > > > > tmp/work/genericx86-poky-linux/core-image-sato/1.0-r0/testimage-sdk/hello/target/debug/build/hello-4dbf26d86b93a892/build-script-build:
> > > > > ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically
> > > > > linked, interpreter
> > > > > /storage/yocto/esp5-platform/build/genericx86/tmp/work/genericx86-poky-linux/core-image-sato/1.0-r0/testimage-sdk/sysroots/x86_64-pokysdk-linux/lib/ld-linux-x86-64.so.2,
> > > > > BuildID[sha1]=c5d5e70657f8342addf5343d0206c77d9d767fd8, for GNU/Linux
> > > > > 3.2.0, with debug_info, not stripped
> > > > >
> > > > > Found one interesting link here
> > > > > https://github.com/rust-lang/cargo/issues/3553. Unfortunately without
> > > > > answer. But also checked the interpreter in my build which looks
> > > > > correct?
> > > > >
> > > > > $ readelf -a
> > > > > tmp/work/genericx86-poky-linux/core-image-sato/1.0-r0/testimage-sdk/hello/target/debug/build/hello-4dbf26d86b93a892/build-script-build
> > > > > > grep interpreter
> > > > > [Requesting program interpreter:
> > > > > /storage/yocto/esp5-platform/build/genericx86/tmp/work/genericx86-poky-linux/core-image-sato/1.0-r0/testimage-sdk/sysroots/x86_64-pokysdk-linux/lib/ld-linux-x86-64.so.2]
> > > > >
> > > > >
> > > > > So there are some differences between my machine and the autobuilder
> > > > > setup that I can't get. I would need help here as I'm not that
> > > > > familiar
> > > > > with the autobuilder setup. Can it still be some host dependency? I'm
> > > > > running on Ubuntu 22.04. Is it possible to check in a autobuilder
> > > > > setup
> > > > > if the file 'build-script-build' is present and possible to execute?
> > > > After staring at this for an hour, I think the pattern is that is it
> > > > failing on builds with:
> > > >
> > > > SDKMACHINE = "i686"
> > > >
> > > > which probably means the linker isn't linking against the libc and
> > > > loader in the SDK properly.
> > > >
> > > > (i686 SDK binaries should run on x86_64 hosts since we provide our own
> > > > loader and libc)
> > > Thanks a lot for the reproducer. You are correct, with SDKMACHINE i686 I
> > > was able to reproduce. I want to share some weird stuff and see if
> > > someone can get a clue for a solution.
> > >
> > > Did same analyze on the build-script-build file as I did on a working one:
> > As I suspected, the dynamic loader on binaries built using the
> > toolchain in the SDK isn't working correctly.
> >
> > "/usr/local/oe-sdk-hardcoded-buildpath/sysroots/i686-pokysdk-
> > linux/lib/ld-linux.so.2"
> >
> > is the value used at build time. This is supposed to be relocated into
> > the SDK's install location at runtime.
> >
> > The question is therefore where is rust getting this value from? You
> > can specify it to the linker at compile time along the lines of:
> >
> > -Wl,--dynamic-linker=/path/to/ld-linux.so.2
> >
> > or it can be the default that comes from toolchain. It is important it
> > points at the one in the SDK, not on the host system.
> >
> > The bit I don't understand is whether:
> >
> > a) the compiler in the SDK is generating the wrong loader (in which
> > case buildtools-extended-tarball would break which it doesn't)
> > b) rust is hardcoding some dynamic-linker option somewhere it shouldn't
> > c) whether the toolchain is relocating paths correctly or some path
> > isn't relocating and is being used here.
> >
> > I'd suggest first checking that a binary generated by nativesdk-gcc in
> > the SDK has the right loader. If that is the case, see what compiler
> > options are being used to create the build tool and go from there.
> >
> > FWIW you can see/change the interpreter with patchelf.
> >
> > Cheers,
> >
> > Richard
>
> I've tried to narrow this down further and I can say now that this error
> when building the native rust build script is not directly related to
> cargo/rust but to the native gcc compiler shipped in the SDK. I will
> next try to dig down in the gcc build process but before spending time
> there I would like to share my finding and see if someone have ideas.
>
> I have two SDKs one with SDKMACHINE="x86_64" and one with
> SDKMACHINE="i686". I have the hosts gcc compiler installed in them and
> use them to compile a minimal c code example with hello world. The one
> for "x86_64" works well and is using the program interpreter from the
> SDK. The executable compiled and linked with "i686" toolchain will have
> a program interpreter pointing to
> "/usr/local/oe-sdk-hardcoded-buildpath/sysroots/i686-pokysdk-linux/lib/ld-linux.so.2".
>
> I will paste in output from the two experiments here below:
>
> x86_64:
>
> $
> <sdk-install>/sysroots/x86_64-pokysdk-linux/usr/bin/x86_64-pokysdk-linux-gcc
> main.c
> $ ./a.out
> Hello world!
> $ readelf -a a.out | grep interpreter
> [Requesting program interpreter:
> <sdk-install>/sysroots/x86_64-pokysdk-linux/lib/ld-linux-x86-64.so.2]
> $
> <sdk-install>/sysroots/x86_64-pokysdk-linux/usr/bin/x86_64-pokysdk-linux-gcc
> -dumpspecs | grep -A 1 *dynamic_linker
>
> i686:
>
> $
> <sdk-install>/sysroots/i686-pokysdk-linux/usr/bin/i686-pokysdk-linux-gcc
> main.c
> $ ./a.out
> -bash: ./a.out: No such file or directory
> $ readelf -a a.out | grep interpreter
> [Requesting program interpreter:
> /usr/local/oe-sdk-hardcoded-buildpath/sysroots/i686-pokysdk-linux/lib/ld-linux.so.2]
>
>
> $
> <sdk-install>/sysroots/i686-pokysdk-linux/usr/bin/i686-pokysdk-linux-gcc
> -dumpspecs | grep -A 1 *dynamic_linker
> *dynamic_linker:
> %{muclibc:%rld-uClibc.so.0;:%{mbionic:/system/bin/linker;:%{mmusl:/usr/local/oe-sdk-hardcoded-buildpath/sysroots/i686-pokysdk-linux/lib/ld-musl-i386.so.1;:/usr/local/oe-sdk-hardcoded-buildpath/sysroots/i686-pokysdk-linux/lib/ld-linux.so.2}}}
>
>
>
> Noted above is that the spec-file (-dumpspecs) for i686 have a sections
> with "*dynamic_linker" but that one is missing in x86_64.
>
> So my conclusion around the results above is that something seems
> missconfigured in the nativesdk-gcc build for i686. When cargo/rust is
> using the native compiler to build its build script it is set to use
> wrong program interpreter and when executed it fails just as my a.out
> above does. I hope this can give you some more ideas where to look next
> and ultimately how to solve this.
>
I couldn't remeber the details of how this worked and there has been a
lot going on so I only just got around to experimenting. This might be
really simple fix:
diff --git a/meta/recipes-devtools/gcc/gcc-multilib-config.inc
b/meta/recipes-devtools/gcc/gcc-multilib-config.inc
index 26bfed9507b..2dbbc23c940 100644
--- a/meta/recipes-devtools/gcc/gcc-multilib-config.inc
+++ b/meta/recipes-devtools/gcc/gcc-multilib-config.inc
@@ -154,7 +154,7 @@ python gcc_multilib_setup() {
gcc_header_config_files = {
'x86_64' : ['gcc/config/linux.h', 'gcc/config/i386/linux.h',
'gcc/config/i386/linux64.h'],
'i586' : ['gcc/config/linux.h', 'gcc/config/i386/linux.h',
'gcc/config/i386/linux64.h'],
- 'i686' : ['gcc/config/linux.h', 'gcc/config/i386/linux64.h'],
+ 'i686' : ['gcc/config/linux.h', 'gcc/config/i386/linux.h',
'gcc/config/i386/linux64.h'],
'mips' : ['gcc/config/linux.h', 'gcc/config/mips/linux.h',
'gcc/config/mips/linux64.h'],
'mips64' : ['gcc/config/linux.h', 'gcc/config/mips/linux.h',
'gcc/config/mips/linux64.h'],
'powerpc' : ['gcc/config/linux.h', 'gcc/config/rs6000/linux64.h'],
which changes the linux.h header to have a %r instead of a
SYSTEMLIBS_DIR hardcoded definition. That will the relocate with the
toolchain.
I've not tested this yet but seems like the right thing to fix.
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#170246):
https://lists.openembedded.org/g/openembedded-core/message/170246
Mute This Topic: https://lists.openembedded.org/mt/93200332/21656
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
