On Fri, 2022-11-04 at 11:16 +0000, Sergey Zhmylev wrote: > On Thu, 2022-11-03 at 21:51 +0000, Richard Purdie wrote: > > «Внимание! Данное письмо от внешнего адресата!» > > > > On Thu, 2022-11-03 at 18:53 +0000, Sergey Zhmylev wrote: > > > Hi Richard, > > > > > > Thank you for the comment! > > > Well, the environment described in reproducible guides currently > > > does > > > not provide binary reproducability due to extfs implementation. > > > > Agreed, there is an issue here. I just don't think the patch looks > > quite right. > > > > > Moreover, building on some FS (for example mounted with noatime or > > > not supporting crtime field at all like UFS) makes the extfs > > > creation > > > process totally not reproducible in spite of any modifications on > > > the > > > source files. > > > > You're arguing we need to set crtime, ctime and atime which I can > > believe. The mtime however should be fine from the package manager so > > I > > don't think this patch should be changing it. > > > > There may be a case for just setting the others to mtime instead of > > SDE > > too? > > Unfortunately, there is no other way to properly get mtime of the files > in order to clone it into atime/ctime/crtime, than create a large > listing of the files and ask debugfs to print stat information, and > then parse the long output creating a debugfs script. > This process can lead to additional bugs and will take a time.
What about adding a new command to debugfs which basically does this for us? There are two options I can see, one to clone the fs info, the other could be a clamp operation, process all the files and change any file with a timestamp later than SDE, to SDE. I'd imagine upstream could understand a usecase for this. > > > I've checked the behaviour under multiple conditions, double > > > checking > > > that I've set as much reproducible stuff as possible, and it comes > > > out that in certain circumstances rootfs files have incorrect > > > timestamps -- they're set to the build process time. > > > > Which timestamps though? Isn't mtime ok? > > mtime "mostly" looks ok, while the others were definitely bad. > There were several files, not only fstab before my previous patch, but > also rpmdb.sqlite and some dnf-related stuff, and so on, on which mtime > was set improperly anyways. > > According to https://reproducible-builds.org/docs/source-date-epoch/ if > the user specified SDE, the "last modification of something" (aka > mtime) must be set to this value. That link is a guideline, not a hard requirement. We go to quite some lengths to preserve file timestamps throughout our build process so I don't feel that destroying them all when creating the filesystem is a particularly great solution. I'd like to see mtime preserved and only values greater than SDE being clamped. > Taking all the above into account, there is nothing bad in changing all > the inodes timestamps: mtime (SDE requirement) and atime/ctime/crtime > (to overcome extfs building issues) to the same value -- specified by > the user. Yes, there is. It isn't what the package managers created or expected and the timestamps don't match the packages. SDE is a tool but the reproducible builds people would not advocate for setting all timestamps to SDE, only those that otherwise wouldn't be reproducible. > > > > > So I'm sure that if the user set SDE to a particular timestamp, we > > > should also modify atime, ctime and crtime. > > > > > > P.S. overriding only the time later than SDE could have also been > > > an > > > option, but in reality we can not surely gather stat info for each > > > file without debugfs or similar tools and this will tremendously > > > degradate performance of already pretty slow image creation > > > process. > > > Due to this I suggest the patch I've sent. > > > > That stat information should already be present? > > Right, but as I said, it's pretty tricky to obtain it for later use. I think we should think about a slightly different approach with more involvement with the tools. It may be worth talking to upstream to see what may or may not be acceptable. In particular, I don't want to destroy all the mtime data. Cheers, Richard
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#172710): https://lists.openembedded.org/g/openembedded-core/message/172710 Mute This Topic: https://lists.openembedded.org/mt/94758789/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
