Hi Richard, Thank you for the comment! Well, the environment described in reproducible guides currently does not provide binary reproducability due to extfs implementation. Moreover, building on some FS (for example mounted with noatime or not supporting crtime field at all like UFS) makes the extfs creation process totally not reproducible in spite of any modifications on the source files.
I've checked the behaviour under multiple conditions, double checking that I've set as much reproducible stuff as possible, and it comes out that in certain circumstances rootfs files have incorrect timestamps -- they're set to the build process time. So I'm sure that if the user set SDE to a particular timestamp, we should also modify atime, ctime and crtime. P.S. overriding only the time later than SDE could have also been an option, but in reality we can not surely gather stat info for each file without debugfs or similar tools and this will tremendously degradate performance of already pretty slow image creation process. Due to this I suggest the patch I've sent. Many thanks! -- With best wishes, [cid:[email protected]] Sergei Zhmylev Engineering consultant OS development department В Чт, 03/11/2022 в 16:05 +0000, Richard Purdie пишет: On Thu, 2022-11-03 at 18:26 +0300, Sergey Zhmylev wrote: From: Sergei Zhmylev <[email protected]<mailto:[email protected]>> Ext2/3/4 FS contains not only mtime, but also ctime, atime and crtime. Currently, all the files are being added into the rootfs image using mkfs -d functionality which affects all the timestamps excluding mtime. This patch ensures all the timestamps inside the FS image equal to the SOURCE_DATE_EPOCH if it is set. Signed-off-by: Sergei Zhmylev <[email protected]<mailto:[email protected]>> --- scripts/lib/wic/partition.py | 27 ++++++++++++++++++++++----- 1 file changed, 22 insertions(+), 5 deletions(-) diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py index bc889bdeb9..b7965267cb 100644 --- a/scripts/lib/wic/partition.py +++ b/scripts/lib/wic/partition.py @@ -294,17 +294,34 @@ class Partition(): f.write("cd etc\n") f.write("rm fstab\n") f.write("write %s fstab\n" % (self.updated_fstab_path)) - if os.getenv('SOURCE_DATE_EPOCH'): - fstab_time = int(os.getenv('SOURCE_DATE_EPOCH')) - for time in ["atime", "mtime", "ctime"]: - f.write("set_inode_field fstab %s %s\n" % (time, hex(fstab_time))) - f.write("set_inode_field fstab %s_extra 0\n" % (time)) debugfs_cmd = "debugfs -w -f %s %s" % (debugfs_script_path, rootfs) exec_native_cmd(debugfs_cmd, native_sysroot) mkfs_cmd = "fsck.%s -pvfD %s" % (self.fstype, rootfs) exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo) + if os.getenv('SOURCE_DATE_EPOCH'): + sde_time = hex(int(os.getenv('SOURCE_DATE_EPOCH'))) + debugfs_script_path = os.path.join(cr_workdir, "debugfs_script") + files = [] + for root, dirs, others in os.walk(rootfs_dir): + base = root.replace(rootfs_dir, "").rstrip(os.sep) + files += [ "/" if base == "" else base ] + files += [ base + "/" + n for n in dirs + others ] + with open(debugfs_script_path, "w") as f: + f.write("set_current_time %s\n" % (sde_time)) + f.write("set_super_value hash_seed %s\n" % (self.fsuuid)) + for file in set(files): + for time in ["atime", "mtime", "ctime", "crtime"]: + f.write("set_inode_field \"%s\" %s %s\n" % (file, time, sde_time)) + f.write("set_inode_field \"%s\" %s_extra 0\n" % (file, time)) + for time in ["wtime", "mkfs_time", "lastcheck"]: + f.write("set_super_value %s %s\n" % (time, sde_time)) + for time in ["mtime", "first_error_time", "last_error_time"]: + f.write("set_super_value %s 0\n" % (time)) + debugfs_cmd = "debugfs -w -f %s %s" % (debugfs_script_path, rootfs) + exec_native_cmd(debugfs_cmd, native_sysroot) + self.check_for_Y2038_problem(rootfs, native_sysroot) def prepare_rootfs_btrfs(self, rootfs, cr_workdir, oe_builddir, rootfs_dir, I don't think this is the correct behaviour as it will effectively remove all timestamp information. Many of the timestamps are ok to use and are reproducible (e.g anything from the packaging backend). There will only be recent entries like the ftab one that specifically need to be set to SDE. Some changes here are good (like the first/last error time and mkfs time and so on) but I think changing all of atime, mtime, ctime etc. is not needed. Could you see if there is a way we could only change the needed values. We could also consider a clamp approach where we only change timestamps later than SDE as that is successful in some other places. Cheers, Richard
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#172650): https://lists.openembedded.org/g/openembedded-core/message/172650 Mute This Topic: https://lists.openembedded.org/mt/94758789/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
