On 03/10/18 05:58, Richard Purdie wrote:
On Thu, 2018-09-27 at 12:18 +1200, Douglas Royds wrote:
When I build glibc in two different places, I get non-reproducible
results - a 4-byte difference:
$ cmp -l ~/workspace/upstream[12]/build/tmp/work/armv5e-tait-linux-
gnueabi/glibc/2.28-r0/package/lib/libc-2.28.so
1259181 71 172
1259182 27 304
1259183 152 77
1259184 363 243
These 4 bytes are the checksum that objcopy --add-gnu-debuglink adds
to the binary when the .debug info is split out at do_package time,
see package.bbclass +416
So why is the debug info different? We add -fdebug-prefix-map to our
CFLAGS, which ensures that all our intra-component debug paths are
prefixed with /usr/src/debug/glibc/, for instance, but this isn't
working in this case. The difference is happening in csu/crt1.o,
which is being linked into libc.so (and others):
$ ../recipe-sysroot-native/usr/bin/arm-tait-linux-gnueabi/arm-tait-
linux-gnueabi-objdump -t csu/crt1.o
...
00000000 l df *ABS* 00000000
/home/douglas/workspace/upstream1/build/tmp/work/armv5e-tait-linux-
gnueabi/glibc/2.28-r0/build-arm-tait-linux-gnueabi/csu/abi-note.o
00000000 l df *ABS* 00000000
/home/douglas/workspace/upstream1/build/tmp/work/armv5e-tait-linux-
gnueabi/glibc/2.28-r0/build-arm-tait-linux-gnueabi/csu/start.o
This abi-note.o symbol is finding its way into libc.so:
$ ../recipe-sysroot-native/usr/bin/arm-tait-linux-gnueabi/arm-tait-
linux-gnueabi-objdump -t libc.so
...
00000000 l df *ABS* 00000000
/home/douglas/workspace/upstream1/build/tmp/work/armv5e-tait-linux-
gnueabi/glibc/2.28-r0/build-arm-tait-linux-gnueabi/csu/abi-note.o
No it isn't, it turns out. The non-reproducibility of crt1.o and friends
is a problem, as they get copied into the recipe-sysroot of all other
components, causing the 780ish different packages observed by Juro in
https://bugzilla.yoctoproject.org/show_bug.cgi?id=12743
There is a work-around: turn off the debug packaging:
INHIBIT_PACKAGE_DEBUG_SPLIT_pn-glibc = "1"
I don't have a solution for this. Suggestions?
I did some digging.
I tried what I suggested using relative paths:
http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/t222&id=22189be4bf508851950f72654870c4eebd4b73d9
and whilst it helps remove some references, there is a much wider
problem. I therefore went and had a look at the linker itself to
understand why its injecting this path. I found this code:
http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/t222&id=f9aca51204990fbdbdfa7442f1dcc938e97bf782
which if disabled as per this hack, makes the binaries reproducible and
drops the problematic references.
We're swapping some misleading debug output for reproducibility with
that hack.
At this point we probably need to talk to some toolchain people about
what 'real' fixes may be possible...
Excellent. Your patch in 22189be is much tidier than my own Makefile
hack that I was too embarassed to publish on this list (it may or may
not have involved sedding a binary file).
It turns out that we have two binary difference issues: The crt1.o (and
friends) one discussed above, and a similar problem linking all the
shared libs, eg. libm.so
$
../recipe-sysroot-native/usr/bin/arm-tait-linux-gnueabi/arm-tait-linux-gnueabi-objdump
-t math/libm.so | grep armv5e
00000000 l df *ABS* 00000000
/home/douglas/workspace/upstream1/build/tmp/work/armv5e-tait-linux-gnueabi/glibc/2.28-r0/build-arm-tait-linux-gnueabi/csu/abi-note.o
00000000 l df *ABS* 00000000
/home/douglas/workspace/upstream1/build/tmp/work/armv5e-tait-linux-gnueabi/glibc/2.28-r0/build-arm-tait-linux-gnueabi/csu/crti.o
00000000 l df *ABS* 00000000
/home/douglas/workspace/upstream1/build/tmp/work/armv5e-tait-linux-gnueabi/glibc/2.28-r0/build-arm-tait-linux-gnueabi/csu/crtn.o
From temp/log.do_compile:
arm-tait-linux-gnueabi-gcc
-march=armv5e -marm
...
-B/home/douglas/workspace/upstream1/build/tmp/work/armv5e-tait-linux-gnueabi/glibc/2.28-r0/build-arm-tait-linux-gnueabi/csu/
...
-o
/home/douglas/workspace/upstream1/build/tmp/work/armv5e-tait-linux-gnueabi/glibc/2.28-r0/build-arm-tait-linux-gnueabi/math/libm.so
...
/home/douglas/workspace/upstream1/build/tmp/work/armv5e-tait-linux-gnueabi/glibc/2.28-r0/build-arm-tait-linux-gnueabi/csu/abi-note.o
...
This link command ignores -fdebug-prefix-map even if we add it (ld
doesn't accept it). The fully-qualified paths to crti.o and crtn.o come
from the -B option. If we use relative paths to these files and to
abi-note.o, the problem goes away:
$ cd git/math/
$ arm-tait-linux-gnueabi-gcc \
-march=armv5e -marm \
...
-B../../build-arm-tait-linux-gnueabi/csu/ \
...
-o
/home/douglas/workspace/upstream1/build/tmp/work/armv5e-tait-linux-gnueabi/glibc/2.28-r0/build-arm-tait-linux-gnueabi/math/libm.so
\
...
../../build-arm-tait-linux-gnueabi/csu/abi-note.o \
...
$ cd ../../build-arm-tait-linux-gnueabi/
$
../recipe-sysroot-native/usr/bin/arm-tait-linux-gnueabi/arm-tait-linux-gnueabi-objdump
-t math/libm.so | grep csu
00000000 l df *ABS* 00000000
../../build-arm-tait-linux-gnueabi/csu/abi-note.o
00000000 l df *ABS* 00000000
../../build-arm-tait-linux-gnueabi/csu/crti.o
00000000 l df *ABS* 00000000
../../build-arm-tait-linux-gnueabi/csu/crtn.o
Make is invoked from the build-arm-tait-linux-gnueabi/ directory, whose
Makefile recursively invokes the Makefile in ../git/, passing in a
fully-qualified "objdir":
all .DEFAULT:
$(MAKE) -r PARALLELMFLAGS="$(PARALLELMFLAGS)" -C $(srcdir)
*objdir=`pwd`* $@
Unfortunately, the makefiles are written on the assumption that the
objdir is fully-qualified, so simply hacking this line doesn't work.
The -B path comes from git/Makerules +612, and the fully-qualified path
to abi-note.o from line +665, in both cases via the csu-objpfx variable.
$(extra-B-$(@F:lib%.so=%).so) -B$(csu-objpfx) \
...
$(csu-objpfx)abi-note.o $(build-shlib-objlist)
I added "realpath" to the HOSTTOOLS, and precomputed a relative path to
the csu/ dir in git/Makerules.
csu-objpfx-relative = $(shell realpath --relative-to=`pwd`
$(csu-objpfx))/
Seems to do the trick. I'll send a patch(ish) separately.
--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
