There's an interesting reproducibility problem triggered when a recipe
has a file in $S that is manipulated and then written into $B. As a
concrete example, zlib has zconf.h in the source tree which is then
processed by configure to create zconf.h in the build tree.
When we populate the zlib-src package we have to work from the mapped
source paths in the binaries (of the form /usr/src/BP) and try to
identify where the file came from. To do this we iterate through the
mapping paths looking for files, and using cpio in a loop we end up with
the newest file with the name we're after in the -src package, which
tends to be the right thing to do.
However, when building zlib from scratch we create the -src package by
hard linking files from the source and build trees. Then as the final
act of do_package the sstate archive is created, and this clamps the
file timestamps to SOURCE_DATE_EPOCH (see fixtimestamp in sstate.bbclass).
The fundamental problem is that we clamp the mtimes in sstate_package in
a way that changes the source files. There are several paths through the
code:
1. Full build from clean
In this case B/zconf.h is freshly generated and has a current timestamp,
and the final zlib-src package in sstate has the correct file.
2. Repackage from existing tree
If we need to repackage but there is an existing build tree then we've
previously clamped the timestamp of zconf.h. When iterating the map paths
for files we first copy from $S and then $B, but as the file in $M has
an identical timestamp it isn't used. This results in the wrong file in
the final zlib-src package in sstate.
3. Repackage from sstate
Depending on whether the sstate has the correct or incorrect file, the
output will have the same contents.
The long-term fix to this is to not clamp the actual files but instead
to clamp then when writing the tarballs, but this isn't trivial as you
need to handle the sstate and non-sstate paths separately. What I think
is an acceptable workaround is to reorder the prefix map list and move
${B} to the front, on the assumption that generated files should be
preferred over non-generated. This results in ${B}/zconf.h having priority
over ${S}/zconf.h when they both have identical timestamps.
[ YOCTO #15491 ]
Signed-off-by: Ross Burton <[email protected]>
---
meta/lib/oe/package.py | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py
index c213a9a3ca6..8ec00b96b69 100644
--- a/meta/lib/oe/package.py
+++ b/meta/lib/oe/package.py
@@ -1010,7 +1010,15 @@ def copydebugsources(debugsrcdir, sources, d):
bb.utils.mkdirhier(basepath)
cpath.updatecache(basepath)
- for pmap in prefixmap:
+ # Re-order the prefix list so that we sweep ${B} first, as the sstate
creation will
+ # clamp mtimes of hardlinks in the -src package, and that will change
the source.
+ # (see #15491)
+ b = d.getVar("B")
+ prefixmap_keys = list(prefixmap)
+ prefixmap_keys.remove(b)
+ prefixmap_keys.insert(0, b)
+
+ for pmap in prefixmap_keys:
# Ignore files from the recipe sysroots (target and native)
cmd = "LC_ALL=C ; sort -z -u '%s' | egrep -v -z
'((<internal>|<built-in>)$|/.*recipe-sysroot.*/)' | " % sourcefile
# We need to ignore files that are not actually ours
--
2.34.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#206301):
https://lists.openembedded.org/g/openembedded-core/message/206301
Mute This Topic: https://lists.openembedded.org/mt/109194063/21656
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
